Embedded RDMA (Remote Direct Memory Access) system and method for multi-source sensor access scene

Abstract

The invention relates to an embedded RDMA (Remote Direct Memory Access) system and method for a multi-source sensor access scene, which can be used for encapsulating transmission data into an RDMA message format and correctly realizing hardware adaptation of RDMA in the multi-source sensor access scene. According to the invention, a software communication interface and software message analysis response processing are provided at a software side, a sensor communication interface and DDR read-write management are provided at a sensor side, and transmission control unloading is provided for an RDMA message processing logic module; the technical problems of standard compatibility, software and hardware collaboration, reliable transmission and the like existing in application of RDMA in a multi-source sensor access scene are solved, and high-performance network transmission with high bandwidth, low delay and low CPU overhead can be provided for sensing data transmission.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology and relates to an embedded RDMA system and method for a multi-source sensor access scenario. Background Art

[0002] RDMA (Remote Direct Memory Access) relies on its design concepts of zero-copy, kernel bypass, and CPU offloading. By offloading the protocol stack to the hardware network card, reliable network communication with high bandwidth, low latency, and low CPU overhead can be achieved. Currently, it has been widely used in fields such as high-performance computing and data centers. With the rapid development of technologies such as autonomous driving and industrial Internet of Things, unmanned equipment needs to comprehensively sense the environment and respond quickly through multi-source sensors, posing higher requirements for the performance of collected data transmission. In the current collection and communication methods, point-to-point communication methods such as MIPI and USB3 Vision have insufficient scalability and limited bandwidth. Bus-type communication methods such as CAN and MOST have low bandwidth. Software network protocol stacks using Ethernet-type communication methods such as the RTP / GigE Vision protocol have high overhead. On the one hand, the processing delay is high, and on the other hand, a large amount of CPU resources are occupied. Further expanding RDMA to the multi-source sensor access scenario is conducive to meeting the reliable communication requirements of high bandwidth and low latency, while occupying less CPU computing resources.

[0003] However, existing RDMA solutions require a general-purpose CPU to support the operation of the RDMA protocol stack and are difficult to meet the requirements of the weak computing power embedded scenario for multi-source sensor access. In terms of standard compatibility, the current embedded RDMA technology route customizes protocol modifications based on the IB protocol standard formulated and maintained by the InfiniBand Trade Association (IBTA) for acquisition adaptation and integration, making it difficult to interoperate with commercial RDMA network cards such as the NVIDIA ConnectX series, and actual deployment and application are difficult. In terms of software and hardware cooperation, the RDMA protocol stack deployment for the multi-source sensor access scenario often uses an embedded CPU core with weak computing power. In the case of high hardware working frequency and fast processing speed, this weak CPU core needs to participate in the startup control before each data transmission and the CQE (Completion Queue Entry) processing after the transmission is completed, which will significantly affect the RDMA transmission performance. The current technology route for the multi-source sensor access scenario has not further optimized this problem. In terms of reliability, most of the current technology routes adopt the RDMA UD (Unreliable Datagram) transmission method to simplify the hardware implementation. However, the UD transmission method can transmit a maximum of only 4KB of data per single transmission, which is not suitable for large-block sensing data transmission and cannot guarantee transmission reliability.

[0004] In addition, in the scenario of multi-source sensor access, the accessed sensing data has different traffic characteristics. Temperature and humidity sensors with relatively small amounts of data may package periodically collected data and perform block-based transmission in units of fixed-size blocks, while audio and video stream data such as cameras is often transmitted in the form of continuous byte streams. For reliable transmission assurance, the embedded RDMA architecture and system for the multi-source sensor access scenario need to resolve the contradiction between continuous writing of sensing data and reliable reading required by RDMA. Currently, there is no mature technology that can effectively guarantee the reliability of RDMA transmission in the streaming transmission scenario. Especially when dealing with special situations such as RDMA retransmission and overwriting when the storage space for streaming data is full, it is difficult to ensure the requirements of data integrity and real-time performance. Summary of the Invention

[0005] In view of the problems existing in the above-mentioned traditional technologies, the present invention proposes an embedded RDMA system for the multi-source sensor access scenario and an embedded RDMA method for the multi-source sensor access scenario, which can provide high-bandwidth, low-latency, and low-CPU-overhead high-performance network transmission for sensing data transmission.

[0006] To achieve the above object, the embodiments of the present invention adopt the following technical solutions: On the one hand, an embedded RDMA system for the multi-source sensor access scenario is provided, including a sensor communication interface, an RDMA peer communication interface, a software communication interface, a DDR controller, a software configuration message parsing and response module, an RDMA message processing logic module, an RDMA transmission control offloading module, and a DDR read / write management module; The DDR read / write management module is respectively connected to the sensor communication interface, the DDR controller, and the RDMA transmission control offloading module. The RDMA message processing logic module is respectively connected to the RDMA transmission control offloading module, the RDMA peer communication interface, the software communication interface, the software configuration message parsing and response module, and the DDR controller. The software configuration message parsing and response module is respectively connected to the software communication interface and the RDMA transmission control offloading module. The software communication interface is used to connect to the embedded RDMA software end; The sensor communication interface is used to acquire sensor data. The RDMA peer communication interface is used to interact with the RDMA communication peer for RDMA messages, two - end link - establishment messages, and communication control messages. The software communication interface is used to transmit software configuration messages, link - establishment messages, and ARP request - response messages. The DDR controller is used to control the peripheral DDR memory. The software configuration message parsing and response module is used to parse and process software message types and forward them, and is also used to configure the RDMA message processing logic module and the RDMA transmission control offloading module. The RDMA message processing logic module is used to encapsulate RDMA messages, parse RDMA messages, and implement packet loss re - transmission. The RDMA transmission control offloading module is used to offload the RDMA transmission start and completion processing corresponding to the standard RDMA protocol stack to hardware for execution. The DDR read - write management module is used to maintain and update the DDR read - write pointers according to the writing of sensing data and the reading of the RDMA message processing logic module.

[0007] On the other hand, an embedded RDMA method for a multi - source sensor access scenario is also provided, which is applied to an embedded RDMA system for a multi - source sensor access scenario. The embedded RDMA system includes a sensor communication interface, an RDMA peer communication interface, a software communication interface, a DDR controller, a software configuration message parsing and response module, an RDMA message processing logic module, an RDMA transmission control offloading module, and a DDR read - write management module. The DDR read - write management module is respectively connected to the sensor communication interface, the DDR controller, and the RDMA transmission control offloading module. The RDMA message processing logic module is respectively connected to the RDMA transmission control offloading module, the RDMA peer communication interface, the software communication interface, the software configuration message parsing and response module, and the DDR controller. The software configuration message parsing and response module is respectively connected to the software communication interface and the RDMA transmission control offloading module. The software communication interface is used to connect to the embedded RDMA software side. The sensor communication interface is used to acquire sensor data. The RDMA peer communication interface is used to interact with the RDMA communication peer for RDMA messages, two - end link - establishment messages, and communication control messages. The software communication interface is used to transmit software configuration messages, link - establishment messages, and ARP request - response messages. The DDR controller is used to control the peripheral DDR memory. The software configuration message parsing and response module is used to parse and process software message types and forward them, and is also used to configure the RDMA message processing logic module and the RDMA transmission control offloading module. The RDMA message processing logic module is used to encapsulate RDMA messages, parse RDMA messages, and implement packet loss re - transmission. The RDMA transmission control offloading module is used to offload the RDMA transmission start and completion processing corresponding to the standard RDMA protocol stack to hardware for execution. The DDR read - write management module is used to maintain and update the DDR read - write pointers according to the writing of sensing data and the reading of the RDMA message processing logic module.

[0008] The embedded RDMA method includes the steps of: Power on the system, negotiate the physical link for the hardware network interface, and sequentially receive ARP request messages, metadata configuration messages, link establishment messages, and transmission control messages from the software communication interface; among them, the software configuration message parsing and response module performs preliminary parsing, forwarding, or configuration of the messages, and constructs and aggregates the response messages corresponding to each request and provides them to the embedded RDMA software; The RDMA transmission control offloading module parses the transmission control message and configures it according to the parsed transmission control parameters; After receiving the software acquisition start configuration, sensor data is merged through the sensor communication interface, and the DDR read / write management module converts the sensor data into an AXIS stream and writes it into the DDR memory through the DDR controller; The RDMA transmission control offloading module constructs WQEs according to the software configuration parameters and the prefetch pointer information provided by the DDR read / write management module and writes them into the on-chip cache SQ; among them, the RDMA transmission control offloading module issues SQ Doorbell information to the RDMA message processing logic module according to the software configuration parameters and the number of WQEs; The RDMA message processing logic module reads the corresponding WQEs in the on-chip cache according to the SQ Doorbell information, obtains sensor data through the DDR controller, constructs and sends an RDMA write request message; After all the response messages corresponding to the request messages are received, the RDMA message processing logic module constructs CQEs and stores them in the on-chip cache CQ; The RDMA transmission control offloading module monitors the CQE write signal of the RDMA message processing logic module, reads and parses the CQEs, and provides CQN information to the DDR read / write management module to update the read pointer; at the same time, the RDMA transmission control offloading module issues a CQ Doorbell to the RDMA message processing logic module to update the CQ status.

[0009] One of the above technical solutions has the following advantages and beneficial effects: The above-mentioned embedded RDMA system and method for the multi-source sensor access scenario can encapsulate the transmission data into the RDMA message format and correctly implement the hardware adaptation of RDMA in the multi-source sensor access scenario. By providing a software communication interface and software message parsing and response processing on the software side, providing a sensor communication interface and DDR read / write management on the sensor side, and providing transmission control offloading for the RDMA message processing logic module, it solves the technical problems such as standard compatibility, software and hardware cooperation, and reliable transmission existing in the application of RDMA in the multi-source sensor access scenario, and can provide high-bandwidth, low-latency, and low-CPU-overhead high-performance network transmission for sensor data transmission. Description of the Drawings

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0011] Figure 1 It is a schematic diagram of the architecture of an embedded RDMA system for a multi-source sensor access scenario in an embodiment; Figure 2 It is a schematic diagram of the internal design of a packet parsing sub-module in an embodiment; Figure 3 It is a schematic diagram of the internal design of a CQE polling sub-module in an embodiment; Figure 4 It is a schematic diagram of the internal design of a DDR read / write management module in an embodiment; Figure 5 It is a schematic diagram of the flow of an embedded RDMA method for a multi-source sensor access scenario in an embodiment. Detailed implementation manners

[0012] In order to make the purpose, technical solutions and advantages of the present invention more clear and understandable, the following further details the present invention in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used in the description of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0013] It should be noted that referring to "embodiment" herein means that the specific features, structures or characteristics described in connection with the embodiment may be included in at least one embodiment of the present invention. Displaying this phrase at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art can understand that the embodiments described herein can be combined with other embodiments. The term "and / or" used herein refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0014] The following will detail the embodiments of the present invention in combination with the drawings in the embodiments of the present invention.

[0015] It can be understood that to solve the existing problems in the background technology, the present invention will provide an embedded RDMA system for multi-source sensor access scenarios, which is used to solve problems in aspects such as standard compatibility, software and hardware cooperation, and reliable transmission (especially for streaming transmission) when applying RDMA in multi-source sensor access scenarios, and to meet the high-performance network transmission requirements for high bandwidth, low latency, and low CPU overhead in multi-source sensor access scenarios. The embedded RDMA architecture and system are compatible with the standard RDMA protocol, and a method with software-configurable transmission parameters provides a flexible access method for sensing data acquisition. Through transmission control hardware offloading and DDR hardware storage management methods, a reliable transmission service of embedded RDMA with high performance and low network processing overhead is achieved.

[0016] In one embodiment, as Figure 1 shown, an embedded RDMA system for multi-source sensor access scenarios includes a sensor communication interface 10, an RDMA peer communication interface 20, a software communication interface 30, a DDR controller 40, a software configuration message parsing and response module 50, an RDMA message processing logic module 60, an RDMA transmission control offloading module 70, and a DDR read / write management module 80. The DDR read / write management module 80 is respectively connected to the sensor communication interface 10, the DDR controller 40, and the RDMA transmission control offloading module 70. The RDMA message processing logic module 60 is respectively connected to the RDMA transmission control offloading module 70, the RDMA peer communication interface 20, the software communication interface 30, the software configuration message parsing and response module 50, and the DDR controller 40. The software configuration message parsing and response module 50 is respectively connected to the software communication interface 30 and the RDMA transmission control offloading module 70. The software communication interface 30 is used to connect to the embedded RDMA software (endpoint).

[0017] The sensor communication interface 10 is used to obtain sensor data, and it can perform deserialization and decoding processing using relevant Xilinx SDI IP cores. The RDMA peer communication interface 20 is used to interact with the RDMA communication peer for RDMA messages, two-end link establishment messages, and communication control messages (such as RDMA messages, ARP messages, and UDP messages), and it can use the Xilinx UltraScale (FPGA) + 100GEthernet Subsystem IP core. The software communication interface 30 is used to transmit software configuration messages, link establishment messages, and ARP request / response messages, and it can use the Xilinx AXI 1G Ethernet Subsystem IP core.

[0018] The DDR controller 40 is used to control the peripheral DDR (such as DDR4) memory and can directly call the relevant IP cores of Xilinx. The software configuration message parsing and response module 50 is used to parse and process the software message types and forward and configure the RDMA message processing logic module 60 and the RDMA transmission control offloading module 70. The RDMA message processing logic module 60 is used to encapsulate RDMA messages, parse RDMA messages and implement packet loss retransmission to ensure transmission reliability. The specific design of this module can refer to the relevant processing logic of the Xilinx ERNIC IP core. The RDMA transmission control offloading module 70 is used to construct WQEs and the corresponding SQ Doorbell before the start of RDMA transmission and send them to the RDMA message processing logic module 60, and process the CQEs constructed after the RDMA message processing logic module 60 completes the transmission and send the CQ Doorbell. The DDR read / write management module 80 is used to maintain and update the DDR read / write pointers.

[0019] It can be understood that the overall system architecture of the embedded RDMA system for the multi-source sensor access scenario provided in this embodiment is composed of a peripheral interface, a software configuration message parsing and response, an RDMA message processing logic module, an RDMA transmission control offloading module, and a DDR read / write management module. The main function is to offload the RDMA transmission startup and completion processing operations in hardware while being compatible with the standard RDMA protocol, and ensure data integrity and reliable RDMA transmission through data acquisition and storage management. This embedded RDMA architecture and system can be deployed using the Xilinx UltraScale (FPGA) + VCU118 development board. Its PCIe interface is not enabled, and it also includes a 10 / 100 / 1000 Mbps SGMII Ethernet interface and two QSFP28 interfaces (100 Gbps). In terms of memory resources, it provides 4 MB of on-chip cache resources and two 2.5 GB DDR4 memories.

[0020] The peripheral interface includes a sensor communication interface, a peer communication interface, and a software communication interface, which are used for communication connections to collect sensors (i.e., sensing devices), RDMA communication peers, embedded RDMA software, and DDR memory, respectively. For the collected sensors, signals of the SDI interface input by a camera (other collection interfaces can also be used instead, which does not affect the implementation of the overall mechanism) are connected to the GTX / GTH transceivers on the VCU118 development board through the FMC HPC interface on the ALINX FH12219, and are converted into an AXI4-Stream video stream through relevant Xilinx IP cores and sent into this embedded RDMA system. For the RDMA communication peer, high-bandwidth sensing data communication is performed through a 100G optical port. For the embedded RDMA software, custom configuration messages, etc. are interacted with through a 1000Mbps network port. For the DDR memory, a DDR controller is used to read and write the DDR storage space.

[0021] The RDMA message processing logic module is used to complete the encapsulation of RDMA request messages and the parsing of received messages, and ensure transmission reliability through the Go-back-N (policy) retransmission mechanism. This embodiment mainly conducts an embedded integration design for the multi-source sensor access scenario based on the existing RDMA message processing logic module. In addition, the on-chip cache of the VCU118 development board is used to store RDMA QP (Queue Pair, which is the basic unit for managing and controlling data transmission), so that the RDMA message processing logic module can quickly read the SQ (Send Queue) WQE (Work Queue Entry) and write the corresponding CQ (Completion Queue) CQE (Completion Queue Entry).

[0022] The software configuration message parsing and response module is used to parse the messages received from the embedded RDMA software and send back responses to ensure the reliability of software and hardware interaction. For various types of messages sent by the software, including UDP link establishment messages, transmission start / stop control messages, metadata configuration messages, and ARP request / response messages, etc., this software configuration message parsing and response module identifies different message types and provides corresponding response messages. At the same time, this software configuration message parsing and response module forwards relevant information to other hardware modules for further processing according to the message type.

[0023] The RDMA transmission control offloading module is used to offload the RDMA transmission startup and completion processing corresponding to the standard RDMA protocol stack to hardware (such as Xilinx UltraScale) for execution, so as to avoid weak CPU cores participating in the transmission startup and stop, which may affect the hardware transmission performance. The transmission control offloading specifically includes offloading the WQE construction, SQ Doorbell issuance, CQE processing, and CQ Doorbell issuance to the hardware for implementation, so as to ensure that the RDMA message processing logic module can normally obtain the Doorbell (doorbell) and WQE at the hardware level and relevant CQEs can be correctly processed.

[0024] The DDR read / write management module is used to maintain and update the DDR read / write pointers according to the write of sensing data and the read situation of the RDMA message processing logic module. Taking the video stream as an example, to ensure that the system can operate correctly in abnormal situations such as the DDR being full and RDMA retransmission (it is necessary to avoid being unable to obtain the originally expected retransmission data due to DDR data overwriting), the DDR read / write pointer update design here corresponds one-to-one with the RDMA QP processing status. Specifically, the DDR write pointer is updated according to the write situation of the sensing data and triggers the WQE construction, the DDR pre-read pointer is updated according to the WQE construction situation, and the DDR read pointer is updated according to the CQE completion situation, so as to lock the DDR space corresponding to the WQE that has been constructed in the SQ but has not received the CQE according to the pre-read pointer and the read pointer to avoid incorrect retransmission data.

[0025] In one embodiment, further, the software configuration message parsing and response module 50 includes a message parsing sub-module and a message response sub-module. The data received by the message parsing sub-module from the gigabit network port first passes through message assembly to combine 8-bit data into 512-bit data blocks, and then different types of messages are classified and processed by message shunting. The message response sub-module caches the Ethernet messages, metadata configuration response messages, and transmission parameter configuration response messages from the RDMA peer respectively, and after arbitration according to the designed priority, the 520-bit FIFO data is disassembled into an 8-bit data format supported by AXI4-Stream by data disassembly.

[0026] Specifically, the data received by the message parsing sub-module from the gigabit network port first passes through message assembly (Assemble) 501 to combine 8-bit data into 512-bit data blocks, and then different types of messages are classified and processed by message shunting (Type_Dec) 502. The internal design schematic diagram is as Figure 2As shown. Ethernet message processing such as link establishment and acquisition control responses sent by the software 503 sends them to the RDMA message processing logic module 60 for forwarding to the RDMA communication peer; Token bucket configuration message processing 504 completes the configuration through the configuration register path provided by the RDMA message processing logic module 60; Mbox configuration message processing 505 performs metadata configuration through the Mbox interaction mechanism of the RDMA message processing logic module 60, and the Mbox response message aggregation 506 constructs the corresponding response message and gives it to the message response sub-module; The transmission parameter configuration message processing 507 hands it over to the RDMA transmission control offloading module 70 for further processing. Correspondingly, the message response sub-module caches the Ethernet messages, metadata configuration response messages, transmission parameter configuration response messages, etc. from the RDMA peer and performs arbitration according to the designed priority, and then the data disassembler (Disassemble) disassembles the 520-bit FIFO data into the 8-bit data format supported by AXI4-Stream. Among them, ETH_Pkt_Data represents Ethernet message data, Token_Data represents token data, MboxW / MboxR represents write / read data of the message queue, ACK_Pkt_Data represents response message data, and Reg_Pkt_Data represents transmission parameter configuration messages.

[0027] In one embodiment, further, the RDMA transmission control offloading module 70 includes a Reg_Ctrl sub-module, a WQE construction sub-module, a communication parameter configuration sub-module, an SQ Doorbell construction sub-module, a CQE polling sub-module, and a CQ Doorbell construction sub-module. The Reg_Ctrl sub-module is used to parse the software transmission parameter configuration message obtained from the software configuration message parsing response module and configure it into the corresponding module, and at the same time construct the corresponding response message according to the parsed message field parameters. The WQE construction sub-module is used to construct the WQE according to the transmission parameters provided by the communication parameter configuration sub-module and the write pointer information provided by the DDR read / write management module, and write it into the on-chip cache, and at the same time output the PI pointer required by the SQ Doorbell construction sub-module. The communication parameter configuration sub-module is also used to provide acquisition start / stop control information. The SQ Doorbell construction sub-module is used to inform the RDMA message processing logic module that there is a new WQE to be processed, and perform WQE counting according to the doorbell_rate parameter provided by the transmission parameter configuration message processing, and construct and issue the SQ Doorbell after the WQE count value reaches the value of the doorbell_rate parameter. The CQE polling sub-module is used to monitor the AXI write operation of the RDMA message processing logic module to the on-chip cache and determine whether it is a CQE according to the written address range, and is also used to output the CQE pointer and the CI pointer. The CQ Doorbell construction sub-module is used to inform the RDMA message processing logic module that the constructed CQE has been processed.

[0028] Specifically, in the Reg_Ctrl sub-module, the transmission parameter processing Reg_Dec parses the software transmission parameter configuration message obtained from the software configuration message parsing response module 50 and configures it into the corresponding functional module. At the same time, the Reg_Ack_Con (transmission confirmation response module) constructs the corresponding response message according to the parsed message field parameters. The transmission parameters used are shown in Table 1. The WQE construction sub-module constructs the WQE according to the transmission parameters provided by the communication parameter configuration sub-module and the write pointer information provided by the DDR read / write management module 80, and writes it into the on-chip cache, and at the same time outputs information such as the PI pointer required for the construction of the SQ Doorbell construction sub-module. In addition, the acquisition start / stop control information provided by the communication parameter configuration sub-module is also used to control whether the WQE construction sub-module can continue or pause constructing the WQE. The interface signals of the WQE construction sub-module are shown in Table 2.

[0029] The SQ Doorbell construction sub-module is used to inform the RDMA message processing logic module 60 that there is a new WQE to be processed. This sub-module counts the WQEs according to the doorbell_rate parameter provided by the message processing 507 based on the transmission parameters, and constructs and issues the SQ Doorbell after the WQE count value reaches this parameter value. In the CQE polling sub-module, the CQE_Record (Completion Queue Entry Write Unit) 701 monitors the AXI write operation of the RDMA message processing logic module 60 to the on-chip cache and determines whether it is a CQE based on the written address range. The CQE_Read (Completion Queue Entry Read Unit) 702 reads the CQE data and sends it to the Ctrl_Gen (Control Signal Generation) logic 703. The Ctrl_Gen logic 703 parses the CQE content and outputs information such as the CQE pointer and CI pointer for use by the CQ Doorbell construction sub-module and the DDR read / write management module 80. The internal design schematic diagram of the CQE polling sub-module is as shown in Figure 3 shown. The CQ Doorbell construction sub-module is used to inform the RDMA message processing logic module 60 that the CQE it constructs has been processed. The SQ / CQ Doorbell data structure is shown in Table 3;

[0030] Specifically, the internal design schematic diagram of the DDR read / write management module 80 is as shown in Figure 4 shown, where the RGBtoAXIS (i.e., RGB to AXIS unit) 801 converts the RGB video signal (for the camera) into an AXIS stream for FPGA processing. The data cache (FIFO_64to512) 802 temporarily stores the input sensing data. The AXI_WR (i.e., AXI read / write unit) 803 polls the FIFO (First In First Out) status. When the data volume meets the AXI burst write condition, the data is written into the DDR memory, and at the same time, the write pointer is dynamically adjusted to skip the non-overwriteable area where the WQE has been sent but the corresponding CQE has not been received (by comparing the write pointer, the read pointer, and the pre-read pointer). AXIS is short for AXI4-Stream and belongs to the Xilinx AXI (Advanced eXtensible Interface) protocol family, which is specifically used for high-speed streaming data transmission (such as video and audio streams). Raddr and Waddr represent reading from and writing to the DDR memory respectively, and WQE_Len represents the WQE length.

[0031] Based on the write pointer information provided by AXI_WR 803 and the pre-read pointer and read pointer information maintained by itself, Addr_Ctrl (i.e., the address control unit) 804 determines the unsent area in the DDR memory that has been written but not constructed with CQE (normally, this area is between the pre-read pointer and the write pointer; in the case of DDR memory write full wrap-around, this area is between the pre-read pointer to the end of DDR and the write pointer to the head of DDR). When the unsent area is greater than the Message_Size (message size) parameter provided by the Reg_Ctrl sub-module, it triggers the valid signal of the WQE construction sub-module while updating the pre-read pointer, and updates the read pointer after receiving the completion information provided by the CQE polling sub-module. In addition, when receiving the acquisition pause signal, the update of the pre-read pointer pauses, while the write pointer and the read pointer continue to be updated.

[0032] In some embodiments, to make the technical method of the present invention clearer, the following further describes the detailed operation process of the above-mentioned embedded RDMA system for the multi-source sensor access scenario with reference to the accompanying drawings and in combination with operation examples: When the system is powered on, the hardware network interface negotiates the physical link, and sequentially receives ARP (Address Resolution) request messages, metadata configuration messages, link establishment messages, transmission control messages, etc. from the software communication interface 30. The message parsing sub-module in the software configuration message parsing and response module 50 performs preliminary message parsing and further forwarding (or configuration), and the message response sub-module constructs and aggregates the response messages corresponding to each request and provides them to the embedded RDMA software. The communication parameter configuration sub-module in the RDMA transmission control offloading module 70 parses the transmission control message and sends the transmission control parameters to sub-modules such as WQE construction and SQ Doorbell construction for configuration.

[0033] After receiving the software acquisition start configuration, the sensing data imported by the sensing device is imported through the sensor communication interface. After being converted into an AXIS stream by the RGBtoAXIS sub-module of the DDR read / write management module 80, it is written into the DDR4 memory by the AXI_WR sub-module through the DDR controller 40.

[0034] The WQE construction sub-module in the RDMA transmission control offloading module 70 constructs WQE according to the software configuration parameters and the pre-read pointer information provided by the Addr_Ctrl (address control) sub-module in the DDR read / write management module 80, and writes it into the on-chip cache SQ. The SQ Doorbell construction sub-module issues SQ Doorbell information to the RDMA message processing logic module 60 according to the software configuration parameters and the number of WQE.

[0035] The RDMA message processing logic module 60 reads the corresponding WQE in the on-chip cache according to the SQ Doorbell information, obtains the sensing data through the DDR controller 40, and constructs and sends an RDMA WRITE (or WRITE with imm, that is, uses the immediate number imm as an operand to perform a write operation) request message.

[0036] One WQE may correspond to multiple request messages. After all the response messages corresponding to the request messages are received, the RDMA message processing logic module 60 constructs a CQE and stores it in the on-chip cache CQ.

[0037] The CQE polling sub-module in the RDMA transmission control offloading module 70 monitors the CQE write signal of the RDMA message processing logic module 60, reads and parses the CQE, provides information such as CQN to the Addr_Ctrl sub-module to update the read pointer, and at the same time triggers the CQ Doorbell construction sub-module to send a CQ Doorbell to the RDMA message processing logic module 60 to update the CQ status.

[0038] After that, the sensing data can continue to be imported from the sensing device into the DDR memory, and the DDR write pointer update operation starts to loop and execute the next RDMA transmission process until a collection pause configuration or the end of the system transmission task is received.

[0039] Each module in the above-mentioned embedded RDMA system for the multi-source sensor access scenario can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned components can be embedded in or independent of a device with radar data processing functions in hardware form, or stored in the memory of the aforementioned device in software form, so as to facilitate the processor to call and execute the operations corresponding to each of the above modules. The aforementioned device can be, but is not limited to, various existing computers in the field.

[0040] In one embodiment, an embedded RDMA method for a multi-source sensor access scenario is further provided, which is applied to an embedded RDMA system for a multi-source sensor access scenario. The embedded RDMA system includes a sensor communication interface, an RDMA peer communication interface, a software communication interface, a DDR controller, a software configuration message parsing and response module, an RDMA message processing logic module, an RDMA transmission control offloading module, and a DDR read / write management module. The DDR read / write management module is respectively connected to the sensor communication interface, the DDR controller, and the RDMA transmission control offloading module. The RDMA message processing logic module is respectively connected to the RDMA transmission control offloading module, the RDMA peer communication interface, the software communication interface, the software configuration message parsing and response module, and the DDR controller. The software configuration message parsing and response module is respectively connected to the software communication interface and the RDMA transmission control offloading module. The software communication interface is used to connect to the embedded RDMA software side. The sensor communication interface is used to obtain sensor data. The RDMA peer communication interface is used to interact with the RDMA communication peer for RDMA messages, two-end link establishment messages, and communication control messages. The software communication interface is used to transmit software configuration messages, link establishment messages, and ARP request / response messages. The DDR controller is used to control the peripheral DDR memory. The software configuration message parsing and response module is used to parse and process software message types and forward them, and to configure the RDMA message processing logic module and the RDMA transmission control offloading module. The RDMA message processing logic module is used to encapsulate RDMA messages, parse RDMA messages, and implement packet loss retransmission. The RDMA transmission control offloading module is used to offload the RDMA transmission start and completion processing corresponding to the standard RDMA protocol stack to the hardware for execution. The DDR read / write management module is used to maintain and update the DDR read / write pointers according to the writing of sensing data and the reading of the RDMA message processing logic module.

[0041] As Figure 5 shown, the above-mentioned embedded RDMA method may include the following steps: S10, when the system is powered on, the hardware network interface negotiates the physical link, and sequentially receives an ARP request message, a metadata configuration message, a link establishment message, and a transmission control message from the software communication interface; wherein, the software configuration message parsing and response module performs preliminary parsing, forwarding, or configuration of the messages, and constructs and provides the response messages corresponding to each request to the embedded RDMA software; S12, the RDMA transmission control offloading module parses the transmission control message and configures it according to the parsed transmission control parameters; S14, after receiving the software acquisition start configuration, the sensing data is merged through the sensor communication interface, and the DDR read / write management module converts the sensing data into an AXIS stream and writes it into the DDR memory through the DDR controller; S16. The RDMA transfer control offloading module constructs WQEs according to software configuration parameters and the prefetch pointer information provided by the DDR read / write management module, and writes them into the on-chip buffer SQ. Among them, the RDMA transfer control offloading module issues SQ Doorbell information to the RDMA message processing logic module according to software configuration parameters and the number of WQEs. S18. The RDMA message processing logic module reads the corresponding WQEs in the on-chip buffer according to the SQ Doorbell information, obtains the sensing data through the DDR controller, constructs and sends an RDMA write request message. S20. After all the response messages corresponding to the request messages are received, the RDMA message processing logic module constructs CQEs and stores them in the on-chip buffer CQ. S22. The RDMA transfer control offloading module monitors the CQE write signal of the RDMA message processing logic module, reads and parses the CQEs, and provides CQN information to the DDR read / write management module to update the read pointer. At the same time, the RDMA transfer control offloading module issues a CQ Doorbell to the RDMA message processing logic module to update the CQ status.

[0042] After that, the remaining sensing data transfer operations can continue to be executed starting from step S14.

[0043] The above-mentioned embedded RDMA method for the multi-source sensor access scenario can encapsulate the transmitted data into the RDMA message format and correctly implement the hardware adaptation of RDMA in the multi-source sensor access scenario. By providing a software communication interface and software message parsing and response processing on the software side, providing a sensor communication interface and DDR read / write management on the sensor side, and providing transfer control offloading for the RDMA message processing logic module, it solves the technical problems such as standard compatibility, software and hardware cooperation, and reliable transmission existing in the application of RDMA in the multi-source sensor access scenario, and can provide high-bandwidth, low-latency, and low-CPU-overhead high-performance network transmission for sensing data transmission.

[0044] In one embodiment, the software configuration message parsing and response module includes a message parsing sub-module and a message response sub-module. The data received from the gigabit network interface by the message parsing sub-module first combines 8-bit data into 512-bit data blocks through message assembly, and then different types of messages are classified and processed by message shunting. The message response sub-module caches the Ethernet messages, metadata configuration response messages, and transmission parameter configuration response messages from the RDMA peer respectively, arbitrates according to the designed priority, and then disassembles the 520-bit FIFO data into the 8-bit data format supported by AXI4-Stream through data disassembly.

[0045] In one embodiment, the RDMA transmission control offloading module includes a Reg_Ctrl sub-module, a WQE construction sub-module, a communication parameter configuration sub-module, an SQ Doorbell construction sub-module, a CQE polling sub-module, and a CQ Doorbell construction sub-module. The Reg_Ctrl sub-module is used to parse the software transmission parameter configuration message obtained from the software configuration message parsing response module and configure it into the corresponding module, and at the same time construct the corresponding response message according to the parsed message field parameters. The WQE construction sub-module is used to construct a WQE according to the transmission parameters provided by the communication parameter configuration sub-module and the write pointer information provided by the DDR read / write management module, and write it into the on-chip cache, and at the same time output the PI pointer required by the SQ Doorbell construction sub-module. The communication parameter configuration sub-module is also used to provide acquisition start / stop control information. The SQ Doorbell construction sub-module is used to inform the RDMA message processing logic module that there is a new WQE to be processed, and perform WQE counting according to the doorbell_rate parameter provided by the transmission parameter configuration message processing, and construct and issue an SQ Doorbell after the WQE count value reaches the value of the doorbell_rate parameter. The CQE polling sub-module is used to monitor the AXI write operation of the RDMA message processing logic module to the on-chip cache and determine whether it is a CQE according to the written address range, and is used to output the CQE pointer and the CI pointer. The CQ Doorbell construction sub-module is used to inform the RDMA message processing logic module that the constructed CQE has been processed.

[0046] It should be understood that although the above steps Figure 5 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps of the above process Figure 5 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0047] All the above optional technical solutions can be combined arbitrarily to form alternative embodiments of the present application, which will not be elaborated here one by one.

[0048] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus dynamic random access memory (Rambus DRAM, abbreviated as RDRAM), and interface dynamic random access memory (DRDRAM), etc.

[0049] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0050] The above embodiments only represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the protection scope of the invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and they all belong to the protection scope of the present invention.

Claims

1. An embedded RDMA system for multi-source sensor access scenarios, characterized in that, It includes a sensor communication interface, an RDMA peer communication interface, a software communication interface, a DDR controller, a software configuration message parsing and response module, an RDMA message processing logic module, an RDMA transmission control offloading module, and a DDR read / write management module; The DDR read / write management module is respectively connected to the sensor communication interface, the DDR controller, and the RDMA transmission control offloading module. The RDMA message processing logic module is respectively connected to the RDMA transmission control offloading module, the RDMA peer communication interface, the software communication interface, the software configuration message parsing and response module, and the DDR controller. The software configuration message parsing and response module is respectively connected to the software communication interface and the RDMA transmission control offloading module. The software communication interface is used to connect to the embedded RDMA software side; The sensor communication interface is used to obtain sensor data. The RDMA peer communication interface is used to interact with the RDMA communication peer for RDMA messages, dual-end link establishment messages, and communication control messages. The software communication interface is used to transmit software configuration messages, link establishment messages, and ARP request / response messages. The DDR controller is used to control the peripheral DDR memory. The software configuration message parsing and response module is used to parse and process software message types and forward them, and is also used to configure the RDMA message processing logic module and the RDMA transmission control offloading module. The RDMA message processing logic module is used to encapsulate RDMA messages, parse RDMA messages, and implement packet loss retransmission. The RDMA transmission control offloading module is used to offload the RDMA transmission start and completion processing corresponding to the standard RDMA protocol stack to hardware for execution. The DDR read / write management module is used to maintain and update the DDR read / write pointers according to the writing of sensing data and the reading of the RDMA message processing logic module.

2. The embedded RDMA system for multi-source sensor access scenarios according to claim 1, wherein The software configuration message parsing and response module includes a message parsing sub-module and a message response sub-module. The data received from the gigabit network port by the message parsing sub-module first undergoes message assembly to combine 8-bit data into 512-bit data blocks, and then different types of messages are classified and processed by message shunting; The message response sub-module caches the Ethernet messages, metadata configuration response messages, and transmission parameter configuration response messages from the RDMA peer respectively, and after arbitration according to the designed priority, the 520-bit FIFO data is disassembled into an 8-bit data format supported by AXI4-Stream by data disassembly; 3. The embedded RDMA system for multi-source sensor access scenarios according to claim 1 or 2, characterized in that The RDMA transmission control offloading module includes a Reg_Ctrl sub-module, a WQE construction sub-module, a communication parameter configuration sub-module, an SQ Doorbell construction sub-module, a CQE polling sub-module, and a CQ Doorbell construction sub-module; The Reg_Ctrl sub-module is used to parse the software transmission parameter configuration message obtained from the software configuration message parsing and response module and configure it to the corresponding module, and at the same time construct the corresponding response message according to the parsed message field parameters; The WQE construction sub-module is used to construct a WQE according to the transmission parameters provided by the communication parameter configuration sub-module and the write pointer information provided by the DDR read / write management module, write the WQE into the on-chip cache, and output the PI pointer required by the SQ Doorbell construction sub-module. The communication parameter configuration sub-module is also used to provide acquisition start / stop control information; The SQ Doorbell construction sub-module is used to inform the RDMA message processing logic module that there is a new WQE to be processed, count the WQEs according to the doorbell_rate parameter provided by the transmission parameter configuration message processing, and construct and issue an SQ Doorbell after the WQE count value reaches the value of the doorbell_rate parameter; The CQE polling sub-module is used to monitor the AXI write operation of the RDMA message processing logic module to the on-chip cache and determine whether it is a CQE according to the written address range, and is used to output the CQE pointer and the CI pointer; The CQ Doorbell construction sub-module is used to inform the RDMA message processing logic module that the constructed CQE has been processed.

4. An embedded RDMA method for multi-source sensor access scenarios, characterized in that, Applied to an embedded RDMA system for a multi-source sensor access scenario, the embedded RDMA system includes a sensor communication interface, an RDMA peer communication interface, a software communication interface, a DDR controller, a software configuration message parsing and response module, an RDMA message processing logic module, an RDMA transmission control offloading module, and a DDR read / write management module; The DDR read / write management module is respectively connected to the sensor communication interface, the DDR controller, and the RDMA transmission control offloading module. The RDMA message processing logic module is respectively connected to the RDMA transmission control offloading module, the RDMA peer communication interface, the software communication interface, the software configuration message parsing and response module, and the DDR controller. The software configuration message parsing and response module is respectively connected to the software communication interface and the RDMA transmission control offloading module. The software communication interface is used to connect to the embedded RDMA software side; The sensor communication interface is used to obtain sensor data. The RDMA peer communication interface is used to interact with the RDMA communication peer for RDMA messages, two-end link establishment messages, and communication control messages. The software communication interface is used to transmit software configuration messages, link establishment messages, and ARP request response messages. The DDR controller is used to control the peripheral DDR memory. The software configuration message parsing and response module is used to parse and process the software message type and forward it, and is used to configure the RDMA message processing logic module and the RDMA transmission control offloading module. The RDMA message processing logic module is used to encapsulate RDMA messages, parse RDMA messages, and implement packet loss retransmission. The RDMA transmission control offloading module is used to offload the RDMA transmission start and completion processing corresponding to the standard RDMA protocol stack to the hardware for execution. The DDR read / write management module is used to maintain and update the DDR read / write pointers according to the writing of sensing data and the reading of the RDMA message processing logic module; The embedded RDMA method includes the steps: The system powers on, and the hardware network interface negotiates the physical link, and sequentially receives ARP request messages, metadata configuration messages, link establishment messages, and transmission control messages from the software communication interface; among them, the message preliminary parsing, forwarding, or configuration is performed by the software configuration message parsing and response module, and the response messages corresponding to each request are aggregated and provided to the embedded RDMA software; The RDMA transmission control offloading module parses the transmission control message and configures it according to the parsed transmission control parameters; After receiving the software acquisition start configuration, the sensing data is incorporated through the sensor communication interface, and the DDR read / write management module converts the sensing data into an AXIS stream and writes it into the DDR memory through the DDR controller; The RDMA transmission control offloading module constructs WQEs according to the software configuration parameters and the prefetch pointer information provided by the DDR read / write management module and writes them into the on-chip buffer SQ; among them, the RDMA transmission control offloading module issues SQ Doorbell information to the RDMA message processing logic module according to the software configuration parameters and the number of WQEs; The RDMA message processing logic module reads the corresponding WQEs in the on-chip buffer according to the SQ Doorbell information, obtains the sensing data through the DDR controller, constructs and sends an RDMA write request message; After all the response messages corresponding to the request messages are received, the RDMA message processing logic module constructs CQEs and stores them in the on-chip buffer CQ; The RDMA transmission control offloading module monitors the CQE write signal of the RDMA message processing logic module, reads and parses the CQEs, and provides CQN information to the DDR read / write management module to update the read pointer; at the same time, the RDMA transmission control offloading module issues a CQ Doorbell to the RDMA message processing logic module to update the CQ status.

5. The embedded RDMA method for multi-source sensor access scenarios according to claim 4, wherein The software configuration message parsing and response module includes a message parsing sub-module and a message response sub-module. The data received from the gigabit network interface by the message parsing sub-module is first assembled into 512-bit data blocks by message assembly, and then different types of messages are classified and processed by message shunting; The message response sub-module caches the Ethernet message, metadata configuration response message, and transmission parameter configuration response message of the RDMA peer end respectively, arbitrates according to the designed priority, and then disassembles the 520-bit FIFO data into an 8-bit data format supported by AXI4-Stream by data disassembly; 6. The embedded RDMA method for multi-source sensor access scenarios according to claim 4 or 5, characterized in that The RDMA transmission control offloading module includes a Reg_Ctrl sub-module, a WQE construction sub-module, a communication parameter configuration sub-module, an SQDoorbell construction sub-module, a CQE polling sub-module, and a CQ Doorbell construction sub-module; The Reg_Ctrl sub-module is used to parse the software transmission parameter configuration message obtained from the software configuration message parsing and response module and configure it to the corresponding module, and at the same time construct the corresponding response message according to the parsed message field parameters; The WQE construction sub-module is used to construct a WQE according to the transmission parameters provided by the communication parameter configuration sub-module and the write pointer information provided by the DDR read / write management module, write it into the on-chip cache, and output the PI pointer required by the SQ Doorbell construction sub-module. The communication parameter configuration sub-module is also used to provide acquisition start / stop control information; The SQ Doorbell construction sub-module is used to inform the RDMA message processing logic module that there is a new WQE to be processed, count the WQEs according to the doorbell_rate parameter provided by the transmission parameter configuration message processing, and construct and issue an SQ Doorbell after the WQE count value reaches the value of the doorbell_rate parameter; The CQE polling sub-module is used to monitor the AXI write operation of the RDMA message processing logic module to the on-chip cache and determine whether it is a CQE according to the written address range, and is used to output the CQE pointer and the CI pointer; The CQ Doorbell construction sub-module is used to inform the RDMA message processing logic module that the constructed CQE has been processed.

Images