PCIe-based bidirectional data transmission method, system and device

Abstract

The application provides a PCIe-based bidirectional data transmission method, system and device. The method is applied between a PCIe root complex and a PCIe endpoint configured with a managed memory pool, and includes: the PCIe endpoint and the PCIe root complex complete handshake through a PCIe mapping control area; first and second data transmission logical channels are created; in a first data transmission direction, the PCIe endpoint issues a first control block containing a first to-be-transmitted data source address and length; the PCIe root complex pulls the first to-be-transmitted data from the PCIe endpoint in a PCIe read mode according to the first control block and writes the first to-be-transmitted data into its managed memory pool; in a second data transmission direction, the PCIe root complex issues a second control block containing a second to-be-transmitted data source address and length; and the PCIe endpoint pulls the second to-be-transmitted data from the managed memory pool of the PCIe root complex in a PCIe read mode according to the second control block. The application can realize simultaneous, high-bandwidth, stable and safe bidirectional data transmission on the same PCIe link without modifying the white list configuration of the managed memory pool.

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, and in particular to a method, system and device for bidirectional data transmission based on PCIe. Background Technology

[0002] In the research and testing of embedded real-time control systems such as intelligent driving and intelligent robots, data acquisition and data refeeding are two fundamental and crucial verification methods. The former is responsible for extracting algorithm outputs in real time from various controllers under test, such as autonomous driving domain controllers, robot motion controllers, or perception fusion units, as well as payloads such as sensor intermediate layer data and video streams, for offline analysis and problem localization by the host computer; the latter, by re-injecting historical road data or virtual simulation sequences into the controller under test, drives its application layer software to run again according to the original time sequence, in order to implement version regression, defect reproduction, and extreme condition coverage testing.

[0003] In existing technologies, common physical layer solutions for achieving the aforementioned data interaction include Ethernet, USB bus, dedicated high-speed acquisition cards, and various fieldbuses. However, these solutions generally suffer from bandwidth limitations, high processor utilization, and significant transmission latency jitter when facing high-bitrate data stream transmission. Furthermore, they lack effective resource isolation mechanisms in multi-process concurrent access scenarios and struggle to support bidirectional full-duplex synchronous execution of acquisition and feedback tasks. While some technologies have attempted to introduce the high-speed and low-latency characteristics of the PCIe interface into the aforementioned testing domain, these solutions typically use PCIe as a hardware interface or high-speed bus without considering the collaborative scheduling issues during parallel acquisition and feedback tasks. In practical applications, when uplink and downlink data streams share the same PCIe link, systemic defects commonly exist, including physical memory contention, queue buffer congestion, security risks introduced by direct operation of physical addresses by the service layer, delayed response of recovery mechanisms after link transient failures, and mutual inhibition of bidirectional effective bandwidth.

[0004] Of particular concern is that when the controller under test (DUT) is running a real-time operating system with security constraints, such as QNX, NVIDIA DRIVE OS Safety, or VxWorks, its system-level memory management unit (SMMU / IOMMU) implements static whitelist-based access control for PCIe inbound transactions. Only a specific segment of managed typed memory is open to external PCIe devices, and typically only read permissions are granted. Active writes (CPU MMIO writes or host-side DMA writes) from external PCIe devices to the domain controller's physical memory are silently discarded by the SMMU. This access control policy directly renders the traditional method of external devices actively sending back data to the DUT via PCIe ineffective.

[0005] In summary, there is an urgent need for a bidirectional data transmission solution that can complete data acquisition and backfeeding through a single PCIe link without relying on write permissions on the system under test, and overcome a series of technical challenges such as memory contention, queue congestion, bandwidth crosstalk, and vulnerability to fault recovery during concurrent operation. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a PCIe-based bidirectional data transmission method, system, and device, which can achieve high-bandwidth, stable, and secure bidirectional data transmission, providing efficient, secure, and robust data transmission assurance for data acquisition and backfeeding processes in R&D and testing systems such as intelligent driving.

[0007] The technical solution of this invention is as follows: A bidirectional data transfer method based on PCIe, applied between a PCIe root complex configured with a managed memory pool and a PCIe endpoint, characterized in that the method includes: The PCIe endpoint and the PCIe root complex complete the handshake through the PCIe mapping control area; Create a first data transmission logical channel and a second data transmission logical channel; In the first data transmission direction, the PCIe endpoint publishes a first control block containing the address and length of the first data source to be transmitted; the PCIe root complex, according to the first control block, pulls the first data to be transmitted from the PCIe endpoint in PCIe read mode and writes it into its managed memory pool. In the second data transmission direction, the PCIe root complex publishes a second control block containing the address and length of the second data source to be transmitted; the PCIe endpoint retrieves the second data to be transmitted from the managed memory pool of the PCIe root complex in PCIe read mode according to the second control block.

[0008] Preferably, the first and second data to be transmitted are retrieved through the same PCIe link, and the first and second data transmission directions are independently handled by callback release, backpressure processing, and error recovery.

[0009] Preferably, the first control block and the second control block are arranged in offset regions that are separate from each other in the PCIe mapping control area.

[0010] Preferably, the method further includes: performing at least one of managed memory range verification, length verification, or alignment verification on the second data to be transmitted; after the verification is passed, writing the source address and length information of the data into the second control block.

[0011] Preferably, when the PCIe endpoint and the PCIe root complex handshake, each party calculates a digest value for the configuration data and writes it into the PCIe mapping control area, and then writes a handshake ready identifier. After either party polls the identifier, it compares the local digest with the peer's digest. If they match, the handshake is successful and the data plane transmit / receive state is entered. If they do not match, the connection is rejected.

[0012] Preferably, for the second data to be transmitted, if its length is less than a preset threshold, multiple data items are written in batches to the temporary storage area of ​​the PCIe mapping control area, and the multiple data items are published at once through the second control block; if its length is greater than or equal to the preset threshold, the physical address and length information of the second data to be transmitted are directly submitted.

[0013] The preferred method further includes an activity detection step: the PCIe endpoint and the PCIe root complex continuously monitor the activity of the peer through the heartbeat sub-region of the PCIe mapping control area; the updater splits the heartbeat information into a count field and a generation field, first updates the count field, inserts a memory barrier, and then updates the generation field; the observer polls the count field and the generation field, and determines whether the peer is alive, restarted, or failed based on the changes in the two fields.

[0014] Preferably, the PCIe endpoint and the PCIe root complex execute the following steps independently: providing a logical channel interface to the user process, allowing the user process to submit description information of the data to be transmitted, and configuring the data type, maximum frame size, and backpressure strategy; returning data to the user process in the form of a virtual address, metadata, or error code, and notifying the user process that the data is ready through a callback mechanism; the user process accessing the data through the virtual address in the callback function, and automatically releasing the corresponding buffer space after the callback returns.

[0015] Preferably, the PCIe endpoint allocates a variable-size DMA receive ring space based on the data length of the second data to be transmitted. If the remaining space at the end of the ring is insufficient to accommodate the second data to be transmitted, a skip flag is written and allocation continues from the beginning of the ring. Preferably, the PCIe endpoint advances through a consumer cursor located in a cacheable shared memory region to release the space already consumed in the DMA data region of the write-merge mapping. The cacheable shared memory region and the DMA receive ring space are different memory regions.

[0016] A PCIe-based bidirectional data transmission system includes a PCIe root complex configured with a managed memory pool, PCIe endpoints, a PCIe mapping control area, and a PCIe link connecting the PCIe endpoints and the PCIe root complex, wherein: The PCIe mapping control area includes a handshake sub-region, a first control block, and a second control block. PCIe endpoints include: The first client library is used to create the first data transmission logical channel; The first pull module is used to pull the second data to be transmitted from the PCIe root complex managed memory pool in PCIe read mode. The first daemon process is used to write the source address and length of the first data to be transmitted into the first control block; and to start the first fetch module according to the second control block.

[0017] The PCIe root complex includes: The second client library is used to create a second data transmission logical channel; The second pull module is used to pull the first data to be transmitted from the PCIe endpoint in PCIe read mode and write it into its managed memory pool. The second daemon process is used to create a second data transmission logical channel, write the source address and length of the data to be transmitted in the second direction into the second control block, and start the second pull module according to the first control block.

[0018] Preferably, the first and second data to be transmitted are retrieved through the same PCIe link; the first and second daemons independently perform callback release, backpressure processing, and error recovery.

[0019] Preferably, the first control block and the second control block in the PCIe mapping control area are located in offset regions that are separate from each other.

[0020] Preferably, the system also includes: The handshake module is used to, when the PCIe endpoint and the PCIe root complex handshake, each party calculates a digest value of the configuration data and writes it into the PCIe mapping control area, and then writes the handshake ready identifier; and to, after either party polls for the identifier, compare the local digest with the peer's digest. If they match, the handshake is successful and the data plane transmit / receive state is entered. If they do not match, the connection is rejected.

[0021] Preferably, the PCIe mapping control area further includes a heartbeat sub-region; the system also includes an activity monitoring module, used to continuously monitor the activity of the peer through the heartbeat sub-region; the updater splits the heartbeat information into a count field and a generation field, first updates the count field, inserts a memory barrier, and then updates the generation field; the observer polls the count field and the generation field, and determines whether the peer is alive, restarted, or failed based on the changes in the two fields.

[0022] A PCIe endpoint device based on bidirectional PCIe data transmission includes: A PCIe interface is used to connect the PCIe root complex device via a PCIe link. The first client library is used to create the first data transmission logical channel; The first pull module is used to pull the second data to be transmitted from the PCIe root complex managed memory pool in PCIe read mode. The first daemon process is used to write the source address and length of the first data to be transmitted into the first control block; and to start the first fetch module according to the second control block.

[0023] Preferably, it also includes a PCIe mapping control area, which includes a handshake sub-region, a heartbeat sub-region, a first control block, and a second control block located in different offset regions.

[0024] Preferably, the first client library is also used for: performing callback release, backpressure processing, and error recovery for the first data transmission direction; providing a logical channel interface to the user process, allowing the user process to submit description information of the data to be transmitted, and configuring the data type, maximum frame size, and backpressure strategy; returning data to the user process in the form of a virtual address, metadata, or error code, and notifying the user process that the data is ready through a callback mechanism; enabling the user process to access the data through the virtual address in the callback function, and automatically releasing the corresponding buffer space after the callback returns.

[0025] Preferably, the device further includes: a high-speed storage module, a computing module, a power supply module, and a control plane module.

[0026] Preferably, a variable-size DMA receive ring is used to allocate space according to the data length of the second data to be transmitted. If the remaining space at the end of the ring is insufficient to accommodate the second data to be transmitted, a skip flag is written and allocation continues from the beginning of the ring. Preferably, after the user callback is completed, the first client library advances through a consumer cursor located in a cacheable shared memory region to release the consumed space in the DMA data region of the write-merge mapping; the cacheable shared memory region and the DMA receive ring space are different memory regions.

[0027] A PCIe root complex device based on bidirectional PCIe data transmission includes: PCIe interface, used to connect PCIe endpoint devices via PCIe link; The second client library is used to create a second data transmission logical channel; The second pull module is used to pull the first data to be transmitted from the PCIe endpoint in PCIe read mode and write it into its managed memory pool. The second daemon process writes the source address and length of the data to be transmitted in the second direction into the second control block of the PCIe mapping control area; and starts the second pull module according to the first control block of the PCIe mapping control area. The managed memory pool is used to store the data to be transmitted during the second-direction data transmission and the data received during the first-direction data transmission.

[0028] Preferably, it also includes a verification module; the verification module is used to perform at least one of managed memory range verification, length verification or alignment verification on the second data to be transmitted. After the verification is passed, the source address and length of the data to be transmitted in the second direction are written into the second control block of the PCIe mapping control area.

[0029] Preferably, the second client library is further used for: performing callback release, backpressure processing, and error recovery for the second data transmission direction; providing a logical channel interface to the user process, allowing the user process to submit description information of the data to be transmitted, and configuring the data type, maximum frame size, and backpressure strategy; returning data to the user process in the form of a virtual address, metadata, or error code, and notifying the user process that the data is ready through a callback mechanism; enabling the user process to access the data through the virtual address in the callback function, and automatically releasing the corresponding buffer space after the callback returns.

[0030] Preferably, it further includes a second data preprocessing module for: batch writing multiple second data items with lengths less than a preset threshold into the temporary storage area of ​​the PCIe mapping control area, and publishing the multiple data items at once through the second control block; and directly publishing the physical address and length of the second transmission data with a length greater than or equal to the preset threshold through the second control block.

[0031] The beneficial effects of this invention are as follows: When PCIe endpoints lack write permissions to the managed memory of the root complex, this invention bypasses the write interception of the memory management unit by actively initiating PCIe read transactions on the receiving side. This achieves high-speed, bidirectional, full-duplex synchronous data acquisition and data backhaul transmission without modifying the underlying system configuration or security whitelist, making it suitable for testing scenarios of embedded control devices equipped with secure real-time operating systems. Furthermore, this invention supports bidirectional concurrent service operation based on a single PCIe link, eliminating the redundant hardware and cabling structures required for multi-channel transmission, simplifying the device architecture, and reducing hardware costs and integration complexity.

[0032] This invention partitions and independently stores bidirectional transmission control information and manages it independently, achieving physical and logical dual isolation of control information in both directions. Each direction independently maintains its transmission status and traffic control parameters, avoiding problems such as memory contention, queue congestion, direct access to physical addresses by user processes, uncertain error recovery, and mutual interference in throughput when bidirectional data streams are concurrent in traditional solutions. This improves the real-time performance, determinism, and operational stability of bidirectional data transmission.

[0033] Furthermore, this invention uses a daemon process to uniformly verify managed memory access, ensuring that DMA operations point to a legitimate range of managed memory addresses, intercepting illegal DMA accesses and erroneous operations, and improving the security of data interaction. Upper-layer applications complete data interaction through a standardized logical channel interface, accessing data via virtual addresses without directly manipulating hardware registers or physical addresses, decoupling from the underlying hardware transmission mechanism and reducing application development and maintenance costs. This invention supports backpressure control, abnormal state detection, and link fault recovery capabilities, adapting to long-term, high-load benchtop closed-loop testing scenarios, improving the long-term operational reliability and operational coverage of the testing system. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of the bidirectional data transmission method based on PCIe according to the present invention; Figure 2 This is a diagram of the bidirectional data transmission system architecture based on PCIe according to the present invention; Figure 3 This is a schematic diagram of the bidirectional data transmission process according to the first embodiment of the present invention; Figure 4 This is a schematic diagram of the layout of the PCIe mapping control area in the first embodiment of the present invention. Detailed Implementation

[0036] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0037] Figure 1 This is a flowchart of the bidirectional data transfer method based on PCIe according to the present invention. This method is applied between a PCIe endpoint and a PCIe root complex. The PCIe root complex has managed memory constrained by an inbound whitelist of its memory management unit. This whitelist constraint causes active writes from the PCIe endpoint to the PCIe root complex to be rejected or discarded. Under this constraint, in scenarios such as autonomous driving data testing, conventional techniques cannot be used to achieve data backflow; that is, data cannot be actively written to the PCIe root complex memory from the PCIe endpoint side.

[0038] To address the above problems, this invention proposes a bidirectional data transmission method based on PCIe, comprising: In step SS1, the PCIe endpoint and the PCIe root complex each start a daemon process and complete a handshake through the PCIe mapping control area.

[0039] Step SS2: Create the first data transmission logical channel and the second data transmission logical channel; First data transmission direction: Step SS311: The PCIe endpoint publishes a first control block containing the address and length of the first data source to be transmitted; Step SS312: The PCIe root complex, according to the first control block, pulls the first data to be transmitted from the PCIe endpoint in PCIe read mode and writes it into its managed memory pool. Second data transmission direction: Step SS321: The PCIe root complex publishes a second control block containing the address and length of the second data source to be transmitted; In step SS322, the PCIe endpoint retrieves the second data to be transmitted from the managed memory pool of the PCIe root complex in PCIe read mode according to the second control block.

[0040] Both the data payloads in the first and second data transmission directions are completed via PCIe reads initiated by the data receiving side or the peer, thus enabling bidirectional data transmission even when the PCIe endpoint does not have write permissions to the managed memory of the PCIe root complex, without needing to modify the inbound whitelist configuration of the PCIe root complex.

[0041] Step SS4: The first data transmission and the second data transmission run concurrently on the same PCIe link. The data lengths of the first control block and the second control block are independent of the metadata lengths, supporting the separate transmission of payload and description information. The first data transmission direction and the second data transmission direction maintain independent control blocks, sequence numbers, acknowledgment status, failure status, queues, memory budgets, backpressure status, and release procedures, so the two transmission directions can run concurrently on the same PCIe link.

[0042] Figure 2 This is a system architecture diagram of the present invention based on PCIe bidirectional data transmission. Figure 2 As shown, the system includes: a PCIe endpoint, a PCIe root complex configured with a managed memory pool, a PCIe mapping control area, and a PCIe link connecting the PCIe endpoint and the PCIe root complex.

[0043] The PCIe mapping control area includes a handshake sub-region, a heartbeat sub-region, a first control block, and a second control block. These sub-regions are located in mutually separate offset regions. The PCIe mapping control area is located on the PCIe endpoint device side.

[0044] The PCIe endpoint includes a first client library for creating a first data transmission logical channel; a first pull module for pulling second data to be transmitted from the PCIe root complex managed memory pool in PCIe read mode; a first daemon for writing the source address and length of the first data to be transmitted into a first control block, and starting the first pull module according to the second control block; a control plane module for transmitting start, stop, power, status, time synchronization or configuration control information through a link different from the PCIe data plane, thereby separating the control plane from the PCIe data plane; and also includes a handshake module, an activity monitoring module, a high-speed storage module and a power module, etc.

[0045] The PCIe root complex includes a second client library for creating a second data transmission logical channel; a second pull module for pulling first data to be transmitted from the PCIe endpoint in PCIe read mode and writing it into its managed memory pool; a second daemon process for creating the second data transmission logical channel, writing the source address and length of the data to be transmitted in the second direction into a second control block; and starting the second pull module according to the first control block; a verification module for performing at least one of managed memory range verification, length verification, or alignment verification on the second data to be transmitted, and notifying the PCIe endpoint device of the source address and length information of the data after the verification is passed; a second data to be transmitted preprocessing module for batch notifying the PCIe endpoint device of multiple second data items with lengths less than a preset threshold, and directly notifying the PCIe endpoint device of the physical address and length information of second data to be transmitted with lengths greater than or equal to the preset threshold; and also includes a handshake module, an activity monitoring module, etc.

[0046] Both devices have handshake modules that are used to calculate a digest value of the configuration data and write it into the PCIe mapping control area when the PCIe endpoint and the PCIe root complex handshake, and then write the handshake ready identifier; and to compare the local digest with the peer digest after either party polls for the identifier. If they match, the handshake is successful and the data plane transmission and reception state is entered. If they do not match, the connection is rejected.

[0047] Both devices have activity monitoring modules that are used to continuously monitor the activity of the other end through the heartbeat sub-region; the updater splits the heartbeat information into a count field and a generation field, first updates the count field, inserts a memory barrier, and then updates the generation field; the observer polls the count field and the generation field, and determines whether the other end is alive, restarted, or failed based on the changes in the two fields.

[0048] The PCIe endpoint client library and the PCIe root complex client library expose logical channel interfaces to user processes and hide PCIe registers, DMA registers, and physical address management, preventing user processes from directly managing physical addresses and PCIe registers. The first and second pull modules pull data to be transmitted through the same PCIe link; the first and second daemons independently perform callback release, backpressure handling, and error recovery.

[0049] According to the technical solution of the present invention, the first data transmission direction and the second data transmission direction generate backpressure states or credit values ​​respectively based on the available memory, queue depth, or number of unreleased buffers on the receiving side; the sending side performs one of blocking wait, returning an error, or discarding old data based on the backpressure state or credit value. Furthermore, the first data transmission direction and the second data transmission direction store their respective control blocks and backpressure / credit fields through different PCIe mapping control area offsets, avoiding the overlap of sequence numbers, acknowledgments, and failure states between the two directions. This achieves isolation of bidirectional data streams on the same PCIe link, ensuring that high-bandwidth first data transmission does not interfere with second data transmission, and that congestion in second data transmission does not disrupt first data transmission.

[0050] First embodiment: Figure 3 This is a schematic diagram of the bidirectional data transmission process according to the first embodiment of the present invention.

[0051] The following combination Figure 3 The principles of the technical solution of the present invention will be further explained in conjunction with the first embodiment. For clarity, the terms involved in the first embodiment will be explained as follows.

[0052] Client library: An API interface running in user process space, responsible for abstracting user-mode access to the data plane. It provides logical channels, virtual addresses, metadata, and error codes to user processes, while hiding the physical address of the BAR control area, PCIe configuration space registers, and DMA controller registers.

[0053] Daemon processes: These are control plane processes that reside permanently in the system and are responsible for the core logic of the control plane. They are used for device discovery, BAR configuration, handshake negotiation, control block management, DMA start / stop, backpressure, and error handling. They do not interact directly with the user but manage the underlying transport mechanisms.

[0054] User process: refers to the application process that runs in the user mode of the operating system and carries specific business logic. It accesses data transmission services through the API interface provided by the client library and does not directly manipulate PCIe registers or physical addresses.

[0055] Host: Used as a PCIe Endpoint, which contains at least one of an embedded computing module, a system-on-a-chip module, a CPU module, a GPU / AI computing module, or a programmable logic module that supports PCIe Endpoint mode.

[0056] Domain controller: Used as a PCIe Root Complex, its managed memory pool is typed memory or reserved memory area; PCIe mapping control area: is the BAR region; PCIe link: Implemented through an external PCIe high-speed connector or a board-level PCIe connector, such as an Oculink connector, MCIO connector, SFF-8644 connector, PCIe cable connector, or board-side high-speed connector.

[0057] refer to Figure 3 The core of the first embodiment is the reinjection process S1 and the data acquisition process S2.

[0058] Process S1, Recharge: First, the host places the backfeed data in its addressable zone by the domain controller and publishes a backfeed control block containing the backfeed data source address and transport description information. Specifically, the backfeed block includes at least one of the following: channel identifier, data type identifier, data length, metadata length, source PCIe address, sequence number, and transport mode.

[0059] Then, the domain controller reads the backfill control block and triggers the startup of its DMA engine to pull backfill data from the host side in PCIe read mode. The pulled backfill data is written to the domain controller's managed memory pool.

[0060] As can be seen from the above process, the data backflow is initiated by the domain controller (data receiving side), which is significantly different from the implementation method in the existing technology where the host actively performs write operations to the physical memory of the domain controller.

[0061] Process S2, Data Acquisition: First, the domain controller publishes a data acquisition control block containing the acquisition data source address and transmission description information located in its managed memory. Specifically, the data acquisition control block includes at least one of the following: number of data items, data physical address, data length, metadata address, metadata length, sequence number, and error status. For example, the managed memory pool on the domain controller side can be QNXtyped memory or a reserved memory area. Before publishing the data acquisition control block, its daemon process performs at least one of the following checks on the data physical address submitted by the user process: range check, length check, and alignment check, belonging to the managed memory pool. If the check fails, the data acquisition control block is not published and an error status is returned.

[0062] Then, the host reads the data acquisition control block, allocates space in its DMA receive ring matching the length of the acquired data according to the data acquisition control block, and starts its DMA engine to actively initiate a PCIe read to pull the acquired data from the managed memory pool on the domain controller side. The pulled acquired data is written to the DMA receive ring.

[0063] As can be seen from the above process, similar to the backfeed, the data acquisition direction also adopts the pull mode initiated by the data receiving side (i.e., the peer), so it can share the same set of managed memory verification, zero-copy delivery and backpressure isolation mechanisms with the backfeed direction, simplifying memory and queue management during bidirectional concurrency.

[0064] In the first embodiment, the data area of ​​the host-side DMA receive ring is mapped with a write-merge attribute to improve DMA write bandwidth. The data area is only used for sequential data writing by the DMA engine and sequential data reading by the CPU, and atomic read-write operations such as LDXR / STXR are not performed.

[0065] The consumer cursor of the receiving loop is located in a different memory region from the data: the consumer cursor is located in a POSIX shared memory region mapped with ordinary cacheable attributes.

[0066] After the user process completes data consumption in the callback function, the host-side client library advances the consumer cursor on the cacheable shared memory using regular atomic operations, marking the consumed byte range. The producer side (DMA engine or its interrupt handler) reads the consumer cursor and reclaims the consumed byte range in the data area, freeing up storage space for subsequent data writing and reuse.

[0067] By separating the data area and the cursor area in the above architecture, reliable space release and recycling of the variable-size receiver ring are achieved without accessing the write-merge region to perform atomic operations.

[0068] This embodiment also includes a micro-batch processing path in the data acquisition direction. When the acquired data is less than a preset threshold, the domain controller-side daemon writes multiple acquired data items and their descriptions into the temporary storage area in the PCIe mapping control area and publishes them through a data acquisition control block. After unpacking, the host-side daemon writes each data item sequentially into a unified host-side DMA receive ring and retains its respective metadata.

[0069] Figure 4 An exemplary diagram of the layout of the BAR control area, also known as the PCIe mapping control area, of the first embodiment is shown. The BAR control area is mapped to the PCIe link and includes a first control block and a second control block that are separate from each other. The first control block and the second control block respectively contain a production sequence number, an acknowledgment number, a failure status, length information, and address information. The BAR control area is also used to store handshakes, heartbeats, etc.

[0070] To execute processes S1 and S2, an initialization process S0 and a transmission control process S3 are also required.

[0071] In process S0, the host and domain controller each start a daemon process. The two daemons exchange version numbers, control area layout and capability bits, and heartbeat handshake. The host exposes a segment of local memory as the BAR control area to the domain controller; the domain controller maps the BAR control area to its address space; the host and domain controller complete the handshake through the BAR control area.

[0072] During the handshake process, each of the two daemons calculates a digest value (e.g., SHA-256 or its truncated form) for a set of configuration data that determines the data plane interpretation (such as control block offset table, channel description, alignment and byte order conventions, etc.), and writes the digest to a designated location in the PCIe mapping control area. After writing the digest, an identifier indicating that the handshake is ready is written. When the peer polls and finds the identifier is set, it compares its locally calculated digest with the digest published by the peer. If they match, data plane transmission and reception proceed; if they do not match (typically due to mismatched daemon versions / configurations or mixed deployments), the connection is rejected and a configuration inconsistency event is reported. This avoids data corruption caused by misalignment of control area layout or field interpretation. Because the digest is written before the identifier, the peer can be assured that the digest is readable once it observes the identifier, avoiding reading a partially initialized digest.

[0073] In process S3, the system enters an idle state after startup.

[0074] In idle state, the system waits to establish a connection with the peer; if the link is lost, the system remains in idle state and continuously monitors the link status.

[0075] Once the handshake is successful and the channel is created, the system switches from idle to running state. In running state, the system supports bidirectional concurrent data transmission, meaning that data acquisition and data feedback can occur simultaneously on the same PCIe link.

[0076] The operational state includes a backpressure self-looping action, meaning that when the receiver's buffer in either direction is insufficient, the transmitter executes one of the following processing methods according to a preset strategy: blocking and waiting, returning an error, or discarding old data. The system reports its status using the acknowledgment number and failure number in the BAR control area. Backpressure processing is an internal loop action within the operational state and does not change the system's operational state.

[0077] When a PCIe link interruption is detected during operation, the system exits the running state and enters the link recovery state. The daemon stops new DMA operations and publishes a link loss event to the client. In the link recovery state, the system performs a re-handshake and digest verification process. After the link is restored, the handshake is re-established and the channel is restored. There are two possible outcomes in the link recovery state: If the handshake and digest verification pass, the system switches from the link recovery state back to the running state and resumes normal bidirectional data transmission. If verification fails or the link remains lost, the system switches from the link recovery state to the stop state. Once in the stop state, the system ceases data transmission and requires manual reset to return to the idle state.

[0078] During system operation, heartbeat and liveness monitoring must be maintained. Following this embodiment, in the PCIe mapping control area, heartbeat can be achieved by performing two independent fixed-width writes to the count field and generation field, respectively, using a memory barrier to ensure the release order. That is, the heartbeat information is split into two fixed-width (e.g., 64-bit each) fields: the updater first writes to one field, inserts a data memory barrier (DMB) to ensure write order and visibility, and then writes to the other field; the observer determines whether the peer is alive, whether a restart has occurred, or whether a generation flip has occurred based on the two fields and their agreed-upon relationship. By combining two independent fixed-width writes with the release order of the memory barrier, equivalent heartbeat consistency is achieved on security constraint platforms lacking wide atomic write support, avoiding the observer reading fragmented intermediate states. Therefore, this embodiment can maintain heartbeat and liveness detection on platforms that do not support or disable wider-width single atomic writes across PCIe (e.g., 128-bit atomic writes).

[0079] Second embodiment: The following second embodiment further illustrates the specific implementation of the device involved in the technical solution of the present invention. The PCIe endpoint device in the second embodiment is a data acquisition and recharge host, and the PCIe root complex device is a vehicle-mounted domain controller.

[0080] The data acquisition and recharge host of the second embodiment includes: a chassis and a computing module, a high-speed storage module, a power management module and a PCIe endpoint controller disposed in the chassis; the PCIe endpoint controller is connected to the vehicle-mounted domain controller under test as a PCIe root complex via an external PCIe connector on the chassis.

[0081] The external PCIe connector is at least one of the following: Oculink connector, MCIO connector, SFF-8644 connector, PCIe cable connector, or board-side high-speed connector.

[0082] The high-speed storage module is an NVMe solid-state drive or a removable solid-state drive module. The high-speed storage module has a DMA receive ring, whose data buffer is mapped with a write-merge attribute. The consumer cursor of the receive ring is placed in a shared memory region mapped with a normal cacheable attribute. The consumer cursor advances atomically over the shared memory region to reclaim consumed byte ranges in the data buffer.

[0083] The chassis also includes an auxiliary control interface group arranged in partitions on the interface panel with the external PCIe connector. The auxiliary control interface group includes at least one of Ethernet, CAN / CAN-FD, time synchronization or power control interface, and is not located in the transmission path of the PCIe payload data plane.

[0084] The computing module is configured to run: The PCIe endpoint driver exposes a segment of local memory as a BAR to the domain controller and provides a restricted DMA startup interface and event notification interface, without exposing the address translation unit and the raw registers of the DMA engine to user space. Before starting DMA, the PCIe endpoint driver checks at least one of the following: PCIe link status, address natural alignment, and mapping window validity. If the conditions are not met, it refuses to start and returns an error status.

[0085] The host-side daemon publishes the source PCIe address and length of the data to be re-re ... The host-side client library provides data acquisition and backfeed interfaces to user processes in the form of logical channels, virtual addresses, metadata, and error codes, while hiding PCIe registers, BAR mappings, and DMA registers.

[0086] Other functions and structures that the host should have can be implemented through the technical solution of this invention or existing technology, and will not be elaborated here.

[0087] The process of bidirectional data transmission between the data acquisition and recharge host in the second embodiment is as follows: In the direction of recharge: Step EP11: The host application opens the backfeed channel and writes the data frame to be backfeed into the transmission buffer. Step EP12: The host-side daemon publishes information such as the source PCIe address, data length, metadata length, and sequence number in the feedback control block of the BAR control area. Step EP13: Respond to the domain controller's PCIe read data pull request and transmit the backfeed data to the domain controller; Step EP14: In response to the update notification from the domain controller side, update the confirmation number and back pressure status in the backfeed control block.

[0088] In the area of ​​data acquisition: Step EP21: In response to the data acquisition notification, publish information such as the physical address and length of the acquired data in the data acquisition control block of the BAR control area; Step EP22: The host-side daemon reads the data acquisition control block, starts the DMA engine, and pulls the acquired data from the managed memory on the domain controller side in PCIe read mode and writes it to the DMA receiver ring. In step EP23, the host-side daemon notifies the user process via callback, delivering the virtual address and metadata (including timestamps, calibration parameters, etc.) of the data. The user process then performs business processing such as writing to disk, tagging, slicing, or visualization based on this information. In step EP24, after the user callback returns, the host-side client library advances the consumer cursor in the cacheable shared memory to mark the position of the consumed data, and the corresponding space in the receive loop is released for subsequent data reuse.

[0089] The vehicle domain controller of the second embodiment includes: PCIe interface, used to connect PCIe endpoint devices via PCIe link; The address mapping unit is used to map the BAR control area of ​​the PCIe endpoint device to its address space; The second pull module is used to pull first direction transmission data from the PCIe endpoint device in PCIe read mode; The managed memory pool is used to store data to be transmitted during second-direction data transmission and data received during first-direction data transmission. The managed memory pool is subject to inbound access constraints by its memory management unit, and active writes by the PCIe endpoint device to the managed memory pool are rejected or discarded.

[0090] The second daemon process is used to write the physical address and length information of the second direction transmission data into the BAR control area of ​​the PCIe endpoint device, and to poll the BAR control area to start the second DMA controller to pull the first direction transmission data.

[0091] The bidirectional data transmission method flow on the vehicle domain controller side in the second embodiment is as follows: Recharge direction: Step RC11: The domain controller-side daemon reads the pushback control block to determine the location and length of the data to be pushed back. Step RC12: Start its DMA engine and pull back the data from the host side in PCIe read mode; write the pulled data into the pcie6Memory receive buffer in the managed memory on the domain controller side; Step RC13: After the data writing is complete, a callback is triggered on the domain controller side, and the application on the domain controller side consumes the data frame. Step RC14: After consumption is complete, release the receive buffer and update the acknowledgment number and backpressure status in the feedback control block to notify the host side that the resources are reclaimable.

[0092] In the area of ​​data acquisition: Step RC21: The domain controller application submits the collected data. The data is located in the managed memory and provides physical address and length information (small data can be submitted inline). In step RC22, the domain controller-side daemon performs verification on the data address, including range verification, length verification, and alignment verification, to confirm that it belongs to the managed memory pool. Step RC23: After successful verification, the domain controller-side daemon publishes information such as the physical address and length of the acquired data in the data acquisition control block of the BAR control area. Step RC24: Respond to the host-side PCIe read data retrieval request and transmit the acquired data to the host; Step RC25: In response to the update notification from the host side, update the acknowledgment number and backpressure status in the second control block.

[0093] Third embodiment: The third embodiment is an example of the application of the method and system of the present invention in the field of autonomous driving data testing. When the on-board domain controller under test runs a real-time operating system with safety constraints (such as QNX / DRIVE OS Safety), its memory management unit (SMMU / IOMMU) sets a whitelist for PCIe inbound access that is fixed at compile time: only a certain segment of managed type memory is open to external PCIe devices, and usually only read permissions are allowed; active writes (CPU MMIO writes or host-side DMA writes) to the domain controller's physical memory from the external host side (as a PCIe endpoint) are silently discarded by the SMMU. Therefore, the conventional practice of actively writing back data to the domain controller's memory from the host side is not available on such restricted platforms. Existing PCIe data acquisition / backload solutions generally do not consider the above-mentioned domain controller inbound write whitelist constraints.

[0094] The technical solution of this invention changes the backfeed direction to a system where the domain controller actively pulls data from the host side via PCIe read; the data acquisition direction also uniformly adopts a PCIe read initiated by the host side to pull data from the data receiving side, thus sharing the same managed memory verification, zero-copy delivery, and backpressure isolation mechanism with the backfeed direction, simplifying memory and queue management during bidirectional concurrency. The data acquisition and backfeed system in this embodiment includes: The data acquisition and recharge host (hereinafter referred to as the host) is configured as a PCIe endpoint device, running Linux / Ubuntu, and includes a PCIe EP driver, a host-side daemon and a host-side client library, as well as a host-side DMA memory pool.

[0095] The vehicle-mounted domain controller is configured as a PCIe root complex device, running QNX or other real-time operating systems, and includes domain controller-side daemons and client libraries, as well as a domain controller-side managed memory pool, such as typed memory or PCIe6Memory. The domain controller-side managed memory pool is constrained by the inbound whitelist of the memory management unit, causing the host's proactive writes to the domain controller's physical memory as a PCIe endpoint to be rejected or discarded by the memory management unit.

[0096] The PCIe link can be a PCIe link that supports external high-speed cable connections or board-level connections.

[0097] The BAR control area, or PCIe mapping control area, is mapped to the PCIe link. It stores handshakes, heartbeats, status acknowledgments, and separate feedback control blocks (first control block and data acquisition control block, second control block). The feedback control block and data acquisition control block each contain a production sequence number, acknowledgment number, failure status, length information, and address information.

[0098] Because the backfeedback direction and the data acquisition direction each have independent control blocks, independent queues, independent memory budgets, independent error codes, independent callback release processes, and independent backpressure states, the two data transmission directions can run simultaneously on the same physical PCIe link, but they are presented as mutually unaware logical data planes at the SDK layer.

[0099] In the feedback direction, the host-side application creates a feedback transmission channel through the client library and requests or fills feedback frames. The host-side daemon writes the data length, metadata length, channel identifier, type identifier, sequence number, and source PCIe address of the feedback frame into the feedback control block in the BAR control area. The domain controller-side daemon polls or receives this control block. Based on the source PCIe address and length, the domain controller-side eDMA actively pulls data from the host-side BAR or DMA-accessible area, writes it to the domain controller-side managed memory, and sends it back to the user process via a callback from the client library. The user process then processes and releases the corresponding buffer.

[0100] In data acquisition, domain controller-side applications submit acquired data through a client library. Submission methods include inline small data submission and physical buffer submission. The domain controller-side daemon verifies whether the submitted data originates from managed memory, generates a data acquisition control block, and writes it to the BAR control area. The host-side daemon reads the data acquisition control block, retrieves data from the domain controller-side physical address via host-side eDMA, and sends a callback notification to the host-side application. After processing by the host-side application, the system automatically or semi-automatically releases the ring space.

[0101] One specific operating flow of the system is as follows: When the host starts, it loads the PCIeEP driver, creates the EP function, configures the vendor / device identifier, and sets up the BAR control area and DMA memory pool. When the domain controller starts, it enumerates the host EP devices, maps the BAR control area, and enables managed typedmemory. The daemons on both sides complete versioning, layout, capability, and heartbeat handshakes through the BAR control area.

[0102] During data backfeedback, the host-side application opens a backfeedback channel named camera_front_30hz, configuring the data type, metadata length, maximum frame size, and backpressure strategy. The application writes the decoded YUV image or custom data into the transmit frame. The host-side daemon writes the frame description into the backfeedback control block, and the domain controller-side daemon retrieves the data from the source PCIe address via eDMA, writes it to the receive buffer in pcie6Memory, and triggers a domain controller-side callback. The domain controller-side AEx software consumes the frame and releases the buffer.

[0103] During data acquisition, the domain controller-side application submits the algorithm output data to the DAQ API. For small datasets, multiple data items are batch-written to the BAR staging area and published at once; for large datasets, the physical address and length are submitted. After reading the data acquisition control block, the host-side daemon reads the data from the domain controller side via eDMA. The host-side application obtains the virtual address and metadata in the callback, which is used for disk writing, tagging, slicing, or visualization. The ring space is released after the callback returns.

[0104] When an anomaly occurs, such as insufficient memory at the receiver, the sender may enter a blocking wait state, return a would-block error, or discard old data, depending on its configuration. The system reports the status using the acknowledgment and failure sequence numbers in the BAR control area. If the link is interrupted, the daemon stops new DMA operations and publishes a link loss event to the client; after the link is restored, a re-handshake is performed and the channel is restored.

[0105] This embodiment employs an independent eDMA channel, independent control block, and independent memory budget / backpressure, with read pull initiated by the peer end, enabling throughput in each direction to approach the available line speed of the PCIe link used. During concurrent data acquisition and backfeeding, the throughput reduction in each direction relative to its single-direction baseline is bounded and largely independent, maintaining frame-free operation under concurrency. The processor usage of the SDK daemon is essentially decoupled from throughput, resulting in low CPU overhead per unit throughput. This is not based on a specific link rate.

[0106] In a real-world data test of this embodiment, a PCIe Gen4 x4 link was used. The measured peak values ​​in one direction were approximately 3.9 GB / s for the backfeed direction and approximately 2.9 GB / s for the data acquisition direction, both close to the available bandwidth of the link in the corresponding direction. Multiple sets of tests have verified the concurrent operation of data acquisition and backfeed: in a 5-minute concurrent socket test, there were no frame drops in either direction, with the host-side SDK daemon consistently using approximately 0.23 processor cores and the domain controller-side approximately 0.13 processor cores; in a 4-hour bidirectional stress test, with both directions running simultaneously, the backfeed side maintained approximately 1.39 GiB / s and the data acquisition side reached approximately 2.48 GiB / s, with both daemons remaining active throughout without frame drops; longer (e.g., 6 hours) bidirectional socket tests also verified that both directions maintained the above throughput levels simultaneously.

[0107] The aforementioned Gen4 x4 and Oculink are merely one method of actual testing. The PCIe link of this invention is not limited to a specific generation (such as Gen3, Gen4, Gen5 or subsequent standards), number of channels (such as x1, x2, x4, x8, x16), or physical connector form (such as Oculink, MCIO, onboard gold fingers, cable connectors, etc.); the absolute measured values ​​depend on the PCIe generation, number of channels, and specific hardware configuration used. On links with higher generations or more channels, the architecture of this invention can achieve correspondingly higher absolute throughput, while the relative effects such as bidirectional concurrency without interference, low CPU usage, and zero-copy delivery remain unchanged.

[0108] As can be seen from this embodiment, the technical effect of the present invention is independent of the PCIe generation and the number of channels, and increases proportionally with the increase of the link generation or the number of channels.

[0109] It should be noted that the method flow steps and system modules of the embodiments of the present invention are implemented by computer software, or they can be implemented by combining computer software with necessary hardware.

[0110] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A bidirectional data transfer method based on PCIe, applied between a PCIe root complex configured with a managed memory pool and a PCIe endpoint, characterized in that, The method includes: The PCIe endpoint and the PCIe root complex complete the handshake through the PCIe mapping control area; Create a first data transmission logical channel and a second data transmission logical channel; In the first data transmission direction, the PCIe endpoint publishes a first control block containing the address and length of the first data source to be transmitted; the PCIe root complex, according to the first control block, pulls the first data to be transmitted from the PCIe endpoint in PCIe read mode and writes it into its managed memory pool. In the second data transmission direction, the PCIe root complex publishes a second control block containing the address and length of the second data source to be transmitted; the PCIe endpoint retrieves the second data to be transmitted from the managed memory pool of the PCIe root complex in PCIe read mode according to the second control block.

2. The method according to claim 1, characterized in that, The first and second data to be transmitted are retrieved through the same PCIe link, and the first and second data transmission directions are independently handled by callback release, backpressure processing and error recovery.

3. The method according to claim 1, characterized in that, The first control block and the second control block are arranged in offset regions that are separate from each other in the PCIe mapping control area.

4. The method according to claim 1, characterized in that, The method further includes: performing at least one of managed memory range verification, length verification, or alignment verification on the second data to be transmitted; after the verification is passed, the source address and length information of the data are written into the second control block.

5. The method according to claim 1, characterized in that, When the PCIe endpoint and the PCIe root complex handshake, each party calculates a digest value for the configuration data and writes it into the PCIe mapping control area, and then writes the handshake ready identifier. After either party polls the identifier, it compares the local digest with the peer's digest. If they match, the handshake is successful and the data plane transmit / receive state is entered. If they do not match, the connection is rejected.

6. The method according to claim 1, characterized in that, For the second data to be transmitted, if its length is less than a preset threshold, multiple data items are written in batches to the temporary storage area of ​​the PCIe mapping control area, and the multiple data items are published at once through the second control block; if its length is greater than or equal to the preset threshold, the physical address and length information of the second data to be transmitted are directly submitted.

7. The method according to claim 1, characterized in that, The method further includes an activity detection step: The PCIe endpoint and the PCIe root complex continuously monitor the activity of the other end through the heartbeat subregion of the PCIe mapping control region; The updater splits the heartbeat information into a count field and a generation field, first updates the count field, inserts a memory barrier, and then updates the generation field. The observer polls the count field and the generation field, and determines whether the peer is alive, restarted, or failed based on the changes in the two fields.

8. The method according to claim 1, characterized in that, The PCIe endpoint and the PCIe root complex each execute steps independently: Provide a logical channel interface to user processes, allowing them to submit descriptions of the data to be transmitted and configure the data type, maximum frame size, and backpressure strategy. Return data to the user process in the form of a virtual address, metadata, or error code, and notify the user process that the data is ready through a callback mechanism; The user process accesses data through the virtual address in the callback function, and the corresponding buffer space is automatically released after the callback returns.

9. The method according to claim 1, characterized in that, The PCIe endpoint allocates a variable-size DMA receive ring space based on the data length of the second data to be transmitted. If the remaining space at the end of the ring is insufficient to accommodate the second data to be transmitted, a skip flag is written and allocation continues from the beginning of the ring.

10. The method according to claim 9, characterized in that, The PCIe endpoint advances via a consumer cursor located in a cacheable shared memory region to release consumed space in the write-merge-mapped DMA data region; the cacheable shared memory region is a different memory region from the DMA receive ring space.

11. A bidirectional data transmission system based on PCIe, comprising a PCIe root complex configured with a managed memory pool, PCIe endpoints, a PCIe mapping control area, and a PCIe link connecting the PCIe endpoints and the PCIe root complex, characterized in that: The PCIe mapping control area includes a handshake sub-region, a first control block, and a second control block. The PCIe endpoint includes: The first client library is used to create the first data transmission logical channel; The first pull module is used to pull the second data to be transmitted from the PCIe root complex managed memory pool in PCIe read mode. The first daemon process is used to write the source address and length of the first data to be transmitted into the first control block; and to start the first fetch module according to the second control block. The PCIe root complex includes: The second client library is used to create a second data transmission logical channel; The second pull module is used to pull the first data to be transmitted from the PCIe endpoint in PCIe read mode and write it into its managed memory pool. The second daemon process is used to create a second data transmission logical channel, write the source address and length of the data to be transmitted in the second direction into the second control block, and start the second pull module according to the first control block.

12. A PCIe endpoint device based on bidirectional PCIe data transmission, characterized in that, include: A PCIe interface is used to connect the PCIe root complex device via a PCIe link. The first client library is used to create the first data transmission logical channel; The first pull module is used to pull the second data to be transmitted from the PCIe root complex managed memory pool in PCIe read mode. The first daemon process is used to write the source address and length of the first data to be transmitted into the first control block; And the first pull module is started according to the second control block.

13. A PCIe root complex device based on PCIe bidirectional data transmission, characterized in that, include: PCIe interface, used to connect PCIe endpoint devices via PCIe link; The second client library is used to create a second data transmission logical channel; The second pull module is used to pull the first data to be transmitted from the PCIe endpoint in PCIe read mode and write it into its managed memory pool. The second daemon process writes the source address and length of the data to be transmitted in the second direction into the second control block of the PCIe mapping control area; And the second pull module is started according to the first control block of the PCIe mapping control area; The managed memory pool is used to store the data to be transmitted during the second-direction data transmission and the data received during the first-direction data transmission.

Images