A network card communication method supporting dynamic switching of RDMA and TCP dual protocol stacks
Patent Information
- Application Number
- CN202610847163.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-12
- Publication Date
- 2026-09-04
AI Technical Summary
[0005]针对现有技术的不足,本发明提供了一种支持RDMA与TCP双协议栈动态切换的网卡通信方法,解决了现有网络通信中异构协议栈切换缺乏底层内存共享与状态连续性支撑,导致应对网络拥塞时需中断连接并执行多次跨态数据拷贝,异构传输协议之间的序列号与通信上下文未建立映射,存在切换开销大且易引发数据乱序与重传的问题
[0048] 1. This invention constructs a custom memory page pool that spans protocol stacks, making page frames visible to both the network card hardware driver and the operating system kernel network stack simultaneously. During RDMA and TCP protocol switching, the system directly completes data transfer by reloading the page frame address, avoiding the data copying process between user mode and kernel mode that occurs during traditional connection switching. This reduces CPU processing overhead and memory bus bandwidth consumption during protocol switching.
Smart Images

Figure CN122698680A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer network communication and data transmission technology, specifically to a network card communication method that supports dynamic switching between RDMA and TCP dual protocol stacks. Background Technology
[0002] In data center network communication, Remote Direct Memory Access (RDMA) technology, through kernel bypass and hardware-level zero-copy mechanisms, can provide high-bandwidth and low-latency data transmission. However, RDMA is sensitive to network packet loss, and when network congestion occurs, its reliance on link-layer flow control mechanisms can easily lead to head-of-line blocking and large-scale data retransmissions, resulting in a decrease in overall network throughput. TCP has a mature congestion control mechanism that can better handle complex network congestion states, but TCP data processing requires the participation of the operating system kernel network stack, resulting in high CPU overhead and high communication latency.
[0003] Existing network communication solutions suffer from a technical deficiency in handling network congestion, particularly in the low-overhead dynamic switching between heterogeneous protocol stacks like RDMA and TCP. During protocol stack switching, because the user-mode runtime environment of RDMA and the kernel-mode runtime environment of TCP are isolated, existing switching mechanisms typically require interrupting the current communication connection and then performing multiple data copies between user and kernel modes to complete the data handover. This cross-mode data copying method consumes significant memory bus bandwidth and CPU cycles, leading to increased communication latency. Furthermore, RDMA and TCP employ independent data segmentation and sequence number maintenance logic, lacking a low-level communication context and sequence number mapping mechanism between heterogeneous transmission protocols. When protocol degradation or rollback occurs, the system cannot accurately locate the transmission breakpoint of unacknowledged data, resulting in packet out-of-order delivery, state misalignment, and duplicate transmissions at the receiving end. Overall, existing technologies lack underlying memory sharing and state continuity support, failing to meet the system's requirement for low-overhead protocol switching based on congestion conditions in dynamic network environments.
[0004] Therefore, this invention proposes a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual protocol stacks to address the shortcomings of existing technologies. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual protocol stacks. This method solves the problems of lack of underlying memory sharing and state continuity support for heterogeneous protocol stack switching in existing network communication, which leads to the need to interrupt the connection and perform multiple cross-state data copies when dealing with network congestion. Furthermore, the lack of mapping between sequence numbers and communication contexts between heterogeneous transmission protocols results in high switching overhead and the potential for data out-of-order transmission and retransmission.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] The first aspect of this invention provides a network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual-protocol stacks, comprising:
[0008] The Transmission Control Adaptor layer binds hardware queue pairs to TCP sockets, maps page frames in the memory data buffer, and stores the memory region index into the security context mapping table of the network card hardware layer.
[0009] During RDMA transmission, the network card hardware layer writes the segment memory pointer, payload length, and packet sequence number into the asynchronous metadata shadow ring;
[0010] When congestion occurs, the Transmission Control Adaptor (TADA) layer empties the hardware queue pairs, calculates the target TCP wake-up sequence number and congestion window size based on the last acknowledgment sequence number and asynchronous metadata shadow ring, and writes them into the underlying Transmission Control Block; wakes up the TCP socket, mounts page frames according to the segmented memory pointer, inserts an adaptive protocol extension header carrying the memory region index and relative offset into the message, and sends it.
[0011] The peer extracts the adaptive protocol extension header, queries the security context mapping table to restore the target address, and performs direct data placement.
[0012] When congestion is relieved, the Transmission Control Adaptor (TADA) layer locates the target RDMA packet sequence number in the asynchronous metadata shadow ring based on the latest acknowledgment sequence number and updates it to the transmit register, resets the hardware queue pair, and puts the TCP socket to sleep.
[0013] Furthermore, the page frames that map the memory data buffer specifically include:
[0014] The transmission control adaptation layer requests contiguous locked pages of memory from the operating system kernel as a memory data buffer;
[0015] The Transmission Control Adaptor Layer registers a custom memory page pool across protocol stacks, and registers the list of page frame addresses corresponding to the locked memory in the custom memory page pool, so that the page frames are visible to both the network card hardware driver and the operating system kernel network stack, thus constructing an address space that is visible to both heterogeneous protocol stacks.
[0016] Furthermore, the segmented memory pointers, payload length, and packet sequence number are written into the asynchronous metadata shadow ring, specifically including:
[0017] The programmable processing pipeline inside the network card hardware layer extracts segmented memory pointers, payload lengths, and packet sequence numbers without interfering with the main data path, and packages them into fixed-length descriptors.
[0018] The descriptor is written to the asynchronous metadata shadow ring by incrementing the write pointer of the lock-free circular queue; the target storage address is obtained by taking the modulo operation of the current count value of the write pointer with the total number of slots, multiplying the result of the modulo operation with the fixed byte length of the descriptor, and adding the result of the multiplication to the starting base address.
[0019] When the write pointer is about to catch up with the virtual read pointer maintained based on the latest acknowledgment sequence number, a hardware-level flow control backpressure mechanism is triggered to suspend the scheduling of new send requests.
[0020] Furthermore, congestion monitoring methods during congestion include:
[0021] The media access control sublayer of the network interface card hardware layer maintains a timer register, which is used to accumulate the priority traffic received by the port to control the effective blocking duration of pause frames;
[0022] When the value of the timer register exceeds the preset time threshold, a hardware interrupt instruction is generated and reported to the transmission control adaptation layer through the message signal interrupt mechanism, triggering the congestion degradation mechanism.
[0023] Furthermore, the target TCP wake-up sequence number is calculated based on the last confirmed sequence number and the asynchronous metadata shadow ring, specifically including:
[0024] The initial packet sequence number of the hardware queue pair is used as the starting point, and the last acknowledgment sequence number is used as the ending point.
[0025] In the asynchronous metadata shadow ring, extract the payload lengths corresponding to all packet sequence numbers from the starting point to the ending point and sum them up.
[0026] The target TCP wake-up sequence number is obtained by adding the accumulated payload length to the initial byte sequence number negotiated when the TCP socket is established.
[0027] Further, the congestion window size is calculated, specifically including:
[0028] The transmission control adaptation layer reads the current transmit injection rate and round-trip time estimate from the network interface card hardware layer.
[0029] Congestion state inheritance calculation is performed based on the bandwidth-delay product model. The current transmission injection rate is multiplied by the estimated round-trip time, and the result of the multiplication is divided by the maximum segment length to calculate the theoretical congestion window size.
[0030] The theoretical congestion window size is truncated and protected by combining the receiving window of the peer notification, and the final congestion window size is obtained and overwritten into the congestion window variable and slow start threshold variable of the underlying transmission control block.
[0031] Furthermore, page frames are mounted based on segmented memory pointers, specifically including:
[0032] The transmission control adaptation layer uses the last confirmed sequence number as the starting point for searching and reads segmented memory pointers one by one from the asynchronous metadata shadow ring.
[0033] The page frame address and page offset pointed to by the segmented memory pointer are assigned to the scattered cluster array elements of the TCP protocol stack socket buffer to achieve zero-copy reassembly of discrete memory at runtime.
[0034] Furthermore, direct data placement is performed, specifically including:
[0035] The peer node extracts the relative offset and the data payload length of the current message and performs an out-of-bounds check: it determines whether the sum of the relative offset and the data payload length of the current message is less than or equal to the total authorized buffer length read from the security context mapping table;
[0036] When the boundary check passes, the target address is calculated by adding the receiver's memory base address retrieved from the security context mapping table to the relative offset.
[0037] The network interface card (NIC) hardware layer bypasses the operating system kernel's network receive queue and schedules its internal direct memory access engine to write the data payload to the memory space pointed to by the target address.
[0038] Furthermore, the criteria for determining congestion relief include:
[0039] The network interface card (NIC) hardware layer continuously collects link round-trip time samples during communication and calculates the variance of the sample data within a fixed time window.
[0040] When the variance value is less than the set anti-oscillation threshold, and the kernel layer's TCP protocol stack does not trigger the retransmission timer within multiple consecutive probe cycles, it is determined that the network congestion has eased and a back-cut interrupt signaling is sent to the transmission control adaptation layer.
[0041] Furthermore, locating the target RDMA packet sequence number within the asynchronous metadata shadow ring specifically includes:
[0042] The difference between the latest confirmed byte sequence number and the initial byte sequence number negotiated by the TCP socket is used as the difference between the latest confirmed byte sequence number and the initial byte sequence number; the payload length of each data packet starting from the initial packet sequence number is accumulated in the asynchronous metadata shadow ring.
[0043] The location conditions for the target RDMA packet sequence number are: the cumulative payload length before the packet is less than or equal to the difference between the latest confirmed byte sequence number and the initial byte sequence number, and the sum of the cumulative payload length before the packet and the payload length of the packet itself is strictly greater than the difference between the latest confirmed byte sequence number and the initial byte sequence number;
[0044] If the difference between the latest confirmed byte sequence number and the initial byte sequence number is strictly greater than the cumulative payload length before the target RDMA packet sequence number, the control TCP socket will send the remaining offset bytes as an independent TCP packet for single confirmation, and enable the boundary forced alignment flag in the adaptive protocol extension header to bypass the peer delay confirmation mechanism to align the packet boundary before executing the subsequent hardware queue reset logic.
[0045] A second aspect of the present invention provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the network card communication method supporting dynamic switching of RDMA and TCP dual protocol stacks as described in the first aspect and any possible implementation thereof.
[0046] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the network card communication method supporting dynamic switching of RDMA and TCP dual protocol stacks as described in the first aspect and any possible implementation thereof.
[0047] This invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual-protocol stacks. It has the following advantages:
[0048] 1. This invention constructs a custom memory page pool that spans protocol stacks, making page frames visible to both the network card hardware driver and the operating system kernel network stack simultaneously. During RDMA and TCP protocol switching, the system directly completes data transfer by reloading the page frame address, avoiding the data copying process between user mode and kernel mode that occurs during traditional connection switching. This reduces CPU processing overhead and memory bus bandwidth consumption during protocol switching.
[0049] 2. This invention sets up an asynchronous metadata shadow ring at the network interface card (NIC) hardware layer to record metadata information of RDMA-transmitted data in real time. When a protocol switch is triggered, the system uses the packet sequence number and payload length recorded in the shadow ring to perform sequence number mapping calculations, thereby deriving the TCP wake-up sequence number or the RDMA breakpoint sequence number. This mechanism ensures the continuity of the context state during bidirectional protocol stack switching and solves the problems of data out-of-order transmission and duplicate transmission that easily occur during heterogeneous transmission protocol switching.
[0050] 3. When the protocol is downgraded to TCP transmission, this invention inserts an adaptive protocol extension header containing a memory region index and a relative offset into the data packet. When processing this packet, the receiving end can directly calculate the target address by extracting the extension header information and querying the local security context mapping table. The network card hardware can then directly place the data, allowing the receiving end to bypass the kernel network stack and complete data storage even in TCP downgraded transmission mode, thus maintaining low communication processing latency. Attached Figure Description
[0051] Figure 1 This is a system architecture diagram of the present invention supporting dynamic switching between RDMA and TCP dual protocol stacks;
[0052] Figure 2 This is a flowchart of the network card communication method of the present invention;
[0053] Figure 3 This is a schematic diagram illustrating the initialization principle of the cross-stack addressing space and security context of the present invention.
[0054] Figure 4 This is a diagram of the hardware data path and metadata capture architecture of the present invention;
[0055] Figure 5 This is a timing diagram for congestion state monitoring according to the present invention;
[0056] Figure 6 This is a flowchart illustrating the cross-protocol stack state calculation and synchronization process of the present invention;
[0057] Figure 7 This is a header structure diagram of the discrete memory reorganization and protocol extension of the present invention;
[0058] Figure 8 This is a flowchart of the network status detection and protocol switchback process of the present invention;
[0059] Figure 9 This is a comparison chart of the convergence of the end-to-end average delay at each time point in the network communication process of this invention.
[0060] Figure 10 This diagram illustrates the CPU utilization and dynamic adjustment of the congestion window in the high-concurrency transmission node during the congestion adjustment process of this invention. Detailed Implementation
[0061] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0062] See attached document Figure 1 This invention provides a system that supports dynamic switching between RDMA and TCP dual protocol stacks. The system includes a host node, and an application layer, an operating system kernel layer, and a network card hardware layer deployed within the host node.
[0063] The network interface card (NIC) hardware layer employs a network processing device with programmable data paths. A transmission control adaptation layer is deployed between the network protocol stack at the operating system kernel level and the driver interface at the NIC hardware layer. This transmission control adaptation layer is configured to perform state scheduling and underlying data path redirection between heterogeneous protocol stacks.
[0064] The transport control adaptation layer exposes virtual queue pairs to the upper application layer. The application layer initiates data send and receive requests by manipulating the virtual queue pairs.
[0065] A virtual queue pair is statically bound to a hardware RDMA queue pair and a dormant TCP socket at the underlying level. During protocol stack switching, the virtual queue pair masks the differences in the underlying protocols to maintain the application layer communication context.
[0066] The network interface card (NIC) hardware layer internally constructs an asynchronous metadata shadow ring. This asynchronous metadata shadow ring is independent of the actual payload data transmission channel and is used to bypass the recording of data segment descriptors, payload lengths, and packet sequence numbers when the hardware performs direct memory access operations.
[0067] The network interface card (NIC) hardware layer maintains a security context mapping table. This table records the correspondence between logical memory region indices, hardware access keys, and memory base addresses.
[0068] The operating system kernel allocates contiguous locked pages of memory data buffers. These memory data buffers provide a shared address space for heterogeneous protocol stacks through a cross-protocol stack memory mapping mechanism.
[0069] See attached document Figure 2 , Figure 2 This is a flowchart of a network interface card (NIC) communication method according to an embodiment of the present invention. The present invention provides a NIC communication method that supports dynamic switching between RDMA and TCP dual protocol stacks. The method operates based on the above-described system architecture and may include the following workflow steps:
[0070] S10, the Transmission Control Adaptor (TDA) layer intercepts the application layer's communication establishment request and initializes the virtual queue pair; the TDA layer binds the virtual queue pair to the hardware RDMA queue pair and the TCP socket;
[0071] S20, the transmission control adaptation layer allocates a memory data buffer and completes cross-stack page frame mapping; the two communicating nodes interact to generate corresponding memory region indexes and write the memory region indexes and local memory base addresses into the security context mapping table;
[0072] In a steady-state network environment, the network interface card (NIC) hardware layer transmits data via the RDMA protocol. The NIC hardware layer's direct memory access engine reads data from the memory data buffer and encapsulates it for transmission. Simultaneously, the NIC hardware layer synchronously writes the segmented memory pointer, payload length, and sequence number corresponding to the current data block into the asynchronous metadata shadow ring.
[0073] S40, the network interface card (NIC) hardware layer polls the link layer for priority flow control pause frames in real time; when the duration of the pause frame received by the NIC hardware layer reaches a preset time threshold, the NIC hardware layer generates a hardware interrupt command and reports it to the transmission control adaptation layer.
[0074] S50: After receiving the hardware interrupt command, the transmission control adaptation layer sends a state modification command to the network card hardware layer. The network card hardware layer switches the hardware RDMA queue pair to the empty transmission state to freeze the underlying state machine. The transmission control adaptation layer reads the last acknowledgment sequence number of the hardware RDMA queue pair, extracts the payload length recorded in the asynchronous metadata shadow ring, and calculates the target TCP wake-up sequence number through the homogeneous sequence number mapping algorithm.
[0075] S60, the transmission control adaptation layer reads the current transmit injection rate and round-trip time of the network card hardware layer; the transmission control adaptation layer calculates the congestion window size using the bandwidth-delay product and overwrites the congestion window size into the underlying transmission control block of the TCP socket;
[0076] S70, the Transmission Control Adapter (TCAT) wakes up the TCP socket; the TCAT uses the sequence number of the next packet after the last acknowledgment sequence number as an index to extract the segmented memory pointer in the asynchronous metadata shadow ring and mounts the page frame of the memory data buffer to the sending segment structure of the TCP socket;
[0077] S80, the Transmission Control Adaptive Layer inserts an Adaptive Protocol Extension header before the TCP data stream payload and then sends the message; the peer node intercepts the message at the bottom layer of the TCP protocol stack and extracts the memory region index in the Adaptive Protocol Extension header, restores the target address by querying the local security context mapping table, and performs direct data placement by hardware.
[0078] S90: During the TCP socket's degraded transmission, the network interface card (NIC) hardware layer monitors the network water level status. When the sample variance of the link round-trip time is lower than a set threshold and no retransmission is triggered, the transmission control adaptation layer intercepts the TCP sending state machine and reads the latest acknowledgment sequence number. The transmission control adaptation layer executes the reverse isomorphic mapping algorithm to locate the target RDMA packet sequence number, updates the target RDMA packet sequence number to the NIC hardware layer's sending register, resets the hardware RDMA queue to the ready-to-send state, and puts the TCP socket back into the sleep state.
[0079] The following section will provide a detailed explanation of the execution steps of the above system architecture and network card communication method, taking into account specific technical details.
[0080] See attached document Figure 3 , Figure 3 This is a schematic diagram illustrating the cross-stack addressing space and security context initialization principle according to an embodiment of the present invention. The present invention provides a network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual-protocol stacks. The specific steps included in establishing a communication connection are as follows:
[0081] S10, the adaptation layer intercepts the application layer's communication establishment request and initializes the virtual queue pair, binding the virtual queue pair to the hardware queue pair and the TCP socket. Specifically, this process includes the following sub-steps:
[0082] S101, the application layer initiates a connection request by calling the standard application programming interface. The adaptation layer captures this connection request at the kernel layer by attaching network hook functions and instantiates the control structure of the virtual queue pair in its local memory space. The virtual queue pair exposes an operational semantic interface consistent with the standard RDMA protocol.
[0083] In step S102, the adaptation layer calls the network card driver interface to allocate transmission resources at the hardware layer to instantiate a hardware queue pair. Simultaneously, the adaptation layer creates a TCP socket in the kernel's network protocol stack. Understandably, the TCP socket is configured to a suspended state after initialization, without allocating send queue resources. The adaptation layer records the identifier of the hardware queue pair and the file descriptor of the TCP socket in the context structure of the virtual queue pair, thereby establishing a static mapping association.
[0084] The adaptation layer allocates a data buffer and completes cross-stack page frame mapping. The communicating parties interactively generate memory region indexes and associate them with local memory base addresses, writing them into the mapping table. Specifically, this process includes the following sub-steps:
[0085] In step S201, the adaptation layer requests a contiguous block of locked page memory from the operating system kernel as a data buffer. The adaptation layer registers a custom memory page pool (PagePool) across the protocol stack via a kernel extension module, and registers the list of page frame addresses corresponding to the locked page memory in this custom memory page pool, making it visible to both the network card hardware driver and the operating system kernel network stack. This establishes a common address space visible to heterogeneous protocol stacks during the initialization phase. In the subsequent protocol downgrade switching phase (corresponding to step S703), the adaptation layer dynamically attaches pointers to specific page frames from the custom memory page pool directly to the distributed aggregation array of the currently created TCP socket buffer structure based on the segmented memory pointers extracted from the shadow ring. This avoids fictitious attachments during the initialization phase and achieves zero-copy reassembly at runtime.
[0086] In S202, during the connection handshake phase, the two communicating nodes exchange security authentication information and generate a globally unique memory region index. The communicating node obtains the memory base address of its locally allocated data buffer and the access key generated by the hardware layer. It then constructs a mapping entry by combining the memory region index with the memory base address and access key, and stores it permanently in a mapping table within the hardware layer. For the handshake connection and access key generation process of the RDMA protocol standard, those skilled in the art can use existing out-of-band management protocols or connection management services, which will not be elaborated upon here.
[0087] See attached document Figure 4 , Figure 4 This is a hardware data path and metadata capture architecture diagram according to an embodiment of the present invention. After the communication link is established, the system enters a steady-state operation phase, which specifically includes the following steps:
[0088] In S30, under a steady-state network environment, the hardware layer transmits data via the RDMA protocol. The direct memory access engine reads the data buffer, encapsulates the data, and simultaneously writes the segmented memory pointer, payload length, and sequence number to the shadow ring. Specifically, this process includes the following sub-steps:
[0089] S301, the application layer sends a work request downwards. The direct memory access engine of the hardware layer, based on the address information in the work request, directly pulls the data payload from the page frame of the data buffer through the peripheral component interconnect bus, encapsulates it into an RDMA network message, and injects it into the link.
[0090] In step S302, within the same clock cycle of a direct memory access operation, the programmable processing pipeline inside the hardware layer extracts the segmented memory pointer, the actual read payload length, and the hardware-allocated packet sequence number contained in the current work request. The hardware layer packages this metadata into a fixed-length descriptor and writes it to the corresponding storage slot in the shadow ring through an auto-increment operation of the write pointer in a lockless circular queue. Preferably, to prevent shadow ring overflow, the network interface card (NIC) hardware layer maintains a virtual read pointer by comparing it with the latest acknowledgment sequence number of the underlying connection. When the write pointer is about to wrap around and catch up with the virtual read pointer, the NIC hardware layer triggers the underlying hardware-level flow control backpressure mechanism, suspending the scheduling of new transmission requests and preventing the writing of new descriptors until acknowledgment is received from the peer, allowing the virtual read pointer to advance. This backpressure mechanism operates independently of the actual data transmission channel, preventing unacknowledged packet metadata within the shadow ring from being illegally overwritten by newly generated descriptors, ensuring the integrity of the backtracking state during protocol degradation.
[0091] See attached document Figure 5 , Figure 5 This is a congestion state monitoring timing diagram according to an embodiment of the present invention. During steady-state transmission, the system performs real-time monitoring of the underlying network state, specifically including the following steps:
[0092] S40, the hardware layer polls priority flow control pause frames. When the duration of the received pause frame reaches a preset time threshold, a hardware interrupt instruction is generated and reported to the adaptation layer. Specifically, this process includes the following sub-steps:
[0093] In S401, the media access control sublayer of the hardware layer maintains a timer register. This timer register is used to accumulate the effective blocking duration of priority flow control pause frames received by the port.
[0094] In step S402, when the value in the timer register exceeds a preset time threshold, the firmware at the hardware layer triggers exception handling logic, constructs an interrupt vector, and sends a hardware interrupt command to the host node via a message signal interrupt mechanism. The interrupt service routine registered at the adaptation layer captures this signaling and initiates the protocol degradation process. In this embodiment, the aforementioned preset time threshold is statically configured based on the product of the switch buffer emptying time and the basic round-trip time of the link, and its value is typically set to a range of 10 to 50 milliseconds to filter out instantaneous congestion at the μs level.
[0095] See attached document Figure 6 After the degradation process is triggered, the system performs cross-protocol stack state calculation and synchronization, which includes the following steps:
[0096] S50, the adaptation layer issues a state modification command to switch the hardware queue pair to an empty send state, reads the last acknowledgment sequence number and extracts the payload length from the shadow ring, and calculates the target TCP wake-up sequence number using a homogeneous sequence number mapping algorithm. Specifically, this process includes the following sub-steps:
[0097] In S501, the adaptation layer sends a queue status control word to the hardware layer through the driver interface, controlling the state machine of the hardware queue to transition from ready to send to empty send. The hardware layer stops scheduling new send tasks and records the sequence number of the packet corresponding to the currently completed transmission and received confirmation from the other end in the status register. The adaptation layer reads this status register to obtain the last confirmation sequence number.
[0098] S502, the adaptation layer executes the homogeneous sequence number mapping algorithm. Since RDMA uses discrete packet sequence numbers while TCP uses continuous byte stream sequence numbers, the adaptation layer needs to traverse the shadow ring with the last acknowledgment sequence number as the upper limit, accumulating the payload length of all sent but unacknowledged packets to achieve packet-to-byte offset mapping. The calculation formula for the homogeneous sequence number mapping algorithm is as follows:
[0099] ;
[0100] In the formula, The target TCP wake-up sequence number. The initial byte sequence number negotiated when establishing a TCP socket. The initial packet sequence number for the hardware queue pair. For final confirmation of the serial number, The first recorded in the shadow ring The payload length corresponding to each packet sequence number. The adaptation layer calculates this according to the formula. The value is used as the starting sequence number for subsequent TCP transmissions.
[0101] S60, the adaptation layer reads the current transmission injection rate and round-trip time from the hardware layer, calculates the congestion window size using the bandwidth-delay product, and overwrites it into the TCP socket's transmission control block. Specifically, this process includes the following sub-steps:
[0102] S601, the adaptation layer extracts the injection rate value that limits the sending rate of the hardware queue and the estimated round-trip time of the network link at the current moment by accessing the exposed interface of the congestion control algorithm engine inside the hardware layer.
[0103] S602, the adaptation layer performs congestion state inheritance calculation based on the bandwidth-delay product model, and the calculation formula is as follows:
[0104] ;
[0105] In the formula, let To calculate the target congestion window size, To obtain the injection rate value, This is an estimated round-trip time. This is the maximum segment length negotiated by the TCP protocol. The formula above uses the product of sending bandwidth and link delay to derive the maximum number of concurrent packets that the network link can accommodate under current conditions.
[0106] See attached document Figure 7 , Figure 7 This is a discrete memory reorganization and protocol extension header structure diagram according to an embodiment of the present invention. After state synchronization is completed, the system performs protocol switching and data degradation transmission, specifically including the following steps:
[0107] S70, the adaptation layer wakes up the TCP socket, retrieves the segment memory pointer from the shadow ring using the last acknowledgment sequence number as an index, and mounts the page frames of the data buffer to the send segment structure of the TCP socket. Specifically, this process includes the following sub-steps:
[0108] S701, the adaptation layer modifies the scheduling status word of the TCP socket to restore its running privileges, and overwrites the send sequence number in the transmission control block with the target TCP wake-up sequence number calculated in step S50. .
[0109] S702, adapter layer with Starting from the shadow ring, segmented memory pointers are read one by one. The adaptation layer directly assigns the page frame address and page offset of the data buffer pointed to by the segmented memory pointer to the segmented array element of the TCP protocol stack socket buffer, completing the zero-copy reassembly of discrete memory.
[0110] In S80, the adaptation layer inserts an adaptive protocol extension header before sending the TCP packet payload. The peer node intercepts the packet, extracts the memory region index, queries the mapping table to restore the target address, and performs direct data placement by hardware. Specifically, this process includes the following sub-steps:
[0111] In step S801, the adaptation layer constructs a custom packet header structure at the kernel layer as an adaptive protocol extension header. This header records the memory region index negotiated in step S20 and the relative offset of the current data block in the data buffer. The adaptation layer appends this header to the front end of the reassembled TCP packet payload and calls the underlying network card driver to send the packet to the peer node.
[0112] In S802, the peer node's kernel network stack triggers a hook function to intercept received TCP packets, parses and removes the adaptive protocol extension header, and extracts the memory region index and relative offset. The peer node then uses the aforementioned memory region index as a search keyword to query its local mapping table and obtain the corresponding memory base address.
[0113] S803, the formula for calculating the target address is as follows:
[0114] ;
[0115] In the formula, let For the calculated target address, To retrieve the obtained memory base address, The relative offset extracted from the adaptive protocol extension header will be calculated by the peer node. The data is then sent to the receiving network interface card (NIC) hardware. The NIC hardware bypasses the standard receive queue processing branch of the kernel protocol stack and performs a direct memory access write operation, precisely storing the TCP packet payload at the target address. The memory space it points to simulates the direct placement characteristics of RDMA on the TCP stack.
[0116] See attached document Figure 8 , Figure 8 This is a flowchart of network state detection and protocol rollback according to an embodiment of the present invention. During TCP degradation transmission maintenance, the system evaluates the network state in real time to trigger the rollback mechanism, specifically including the following steps:
[0117] S90: During degraded transmission, the hardware layer monitors the network water level status. When the sample variance of the link round-trip time is lower than a set threshold and no retransmission is triggered, the adaptation layer intercepts the sending state machine and executes the reverse isomorphic mapping algorithm to locate the target RDMA packet sequence number, resets the hardware queue pair, and puts the TCP socket into a sleep state. Specifically, this process includes the following sub-steps:
[0118] In S901, the hardware layer continuously collects link round-trip time samples during TCP communication and calculates the variance of the sample data within a fixed time window in the firmware. When this variance is less than a set anti-surge threshold (this threshold is based on the idle link baseline variance calculated when the system establishes a connection, typically 1.2 to 1.5 times the baseline variance), and the kernel layer's TCP protocol stack does not trigger the retransmission timer within multiple consecutive probe cycles, it indicates that network congestion has been alleviated, and the hardware layer sends a switchback interrupt signaling to the adaptation layer.
[0119] In S902, the adaptation layer freezes the TCP socket's send control mechanism and reads the latest acknowledged byte sequence number from the TCP protocol stack at the kernel layer. Since TCP uses byte stream acknowledgments, there may be misaligned RDMA packet boundaries. The adaptation layer executes a reverse isomorphic mapping algorithm to search for the target RDMA packet sequence number in the shadow ring. The search and location logic satisfies the following inequality:
[0120] ;
[0121] In the formula, The latest confirmed byte sequence number is the argument of the summation operator. The range of values is from to ; This is the sequence number of the target RDMA packet being searched and located. If Not strictly aligned to the packet boundary (i.e.) (If the sum is greater than the aforementioned sum), the system controls the TCP socket to send the remaining unacknowledged payload of the data packet as an independent TCP segment, and waits for the other end to return a boundary-aligned acknowledgment segment before executing the subsequent back-switch logic.
[0122] S903, the adaptation layer sends control commands to the hardware layer, specifying the target RDMA packet sequence number. The transmit sequence number register at the hardware layer is updated. Subsequently, the adaptation layer issues a state modification instruction to restore the hardware queue pair to the ready-to-transmit state and strips the underlying scheduling resources of the TCP socket, causing it to re-enter the sleep state. The communication data stream is then smoothly switched back to the RDMA hardware acceleration path.
[0123] The above description elaborates on the specific execution logic of the method of the present invention under different network conditions and the collaborative working mode of each system level.
[0124] See attached document Figure 3 , Figure 3 This is a schematic diagram illustrating the principle of cross-stack addressing space and security context initialization according to an embodiment of the present invention. The present invention provides a network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual-protocol stacks. During the initialization phase, a cross-stack addressing space and security context establishment mechanism is executed, which may include:
[0125] The application layer initiates a communication connection request. The adaptation layer intercepts this communication establishment request at the kernel layer. The adaptation layer instantiates the control structure of the virtual queue pair in its local memory address space. The virtual queue pair shields the application layer from the differences in underlying heterogeneous network protocols, providing a unified operational semantic interface.
[0126] The adaptation layer calls the underlying driver interface to allocate transmission resources at the hardware layer to instantiate hardware queue pairs. Simultaneously, the adaptation layer creates TCP sockets in the kernel's network protocol stack. The adaptation layer then configures the TCP sockets to a suspended state.
[0127] The adaptation layer obtains the resource identifier of the hardware queue pair and the file descriptor of the TCP socket. The adaptation layer then writes these resource identifiers and file descriptors into the control structure of the virtual queue pair. This operation completes the establishment of the static binding relationship between the virtual queue pair, the hardware queue pair, and the TCP socket.
[0128] After binding is complete, the adapter layer requests a contiguous block of locked memory from the operating system at the kernel level as a data buffer. The memory pages of the data buffer do not participate in the page swapping mechanism during the operating system's lifetime, ensuring that they are continuously persisted in random access memory.
[0129] The adaptation layer obtains the starting address of the page frame corresponding to the data buffer. It then invokes a memory mapping mechanism to directly map the page frame starting address to a segmented linked list structure of the TCP protocol stack socket buffer in the kernel layer. This mapping operation enables the hardware layer's direct memory access engine and the kernel layer's TCP protocol stack to have addressing paths pointing to the same storage medium, thereby constructing a unified cross-stack addressing space in the system.
[0130] During the underlying connection handshake phase, the two communicating nodes negotiate and exchange security verification parameters. The communication process between nodes generates a globally unique memory region index according to preset rules.
[0131] The communication node obtains the memory base address of the locally allocated data buffer. The hardware layer's memory management unit generates the corresponding hardware access key for the data buffer.
[0132] The communication node associates the generated memory region index, memory base address, and hardware access key to construct a security mapping entry. The communication node then writes this security mapping entry into a mapping table within the hardware layer. In subsequent downgrades to TCP protocol transmission and when the receiving end needs to perform direct memory placement operations, the mapping table is used to verify the memory access permissions of data packets and provide the actual addressing calculation parameters.
[0133] After the cross-stack addressing space and security context are initialized, the network link is in a non-congested state, and the system executes steady-state data transmission and metadata shadow synchronization mechanism, which includes the following steps:
[0134] In step S301, the hardware-layer Direct Memory Access (DMI) engine receives a work request from the application layer. Specifically, the DMI engine parses the address information in the work request and, in conjunction with a pre-established memory mapping relationship, locates the specific page frame in the data buffer. Subsequently, the DMI engine directly retrieves the data payload from the data buffer through the peripheral bus interface, encapsulates the data payload into an RDMA network packet, and injects it into the network link. It is understood that those skilled in the art can refer to existing protocol specifications (such as RoCEv2 or iWARP specifications) to implement the specific encapsulation format of the RDMA network packet and the underlying read timing of the DMI engine. The basic data processing flow is well-known in the field and will not be elaborated upon here.
[0135] In step S302, during the parallel processing cycle where the direct memory access engine reads the data buffer and encapsulates the message, the processing pipeline within the hardware layer triggers bypass metadata capture logic. Preferably, the processing pipeline, without interfering with the main data path or initiating additional main memory reads, extracts the segment memory pointer and the corresponding payload length from the currently scheduled work request by monitoring the internal data bus. The segment memory pointer records the starting address of the current data block in the data buffer. Simultaneously, the hardware layer's send scheduler obtains the packet sequence number allocated to the current RDMA network message. The extracted segment memory pointer, payload length, and packet sequence number together constitute a metadata set describing the characteristics of the current data segment. This bypass capture mechanism minimizes the latency impact of the metadata extraction process on main data transmission.
[0136] S303, the hardware layer constructs the metadata set into a fixed-byte descriptor and synchronously writes it into the shadow ring. In this embodiment, the shadow ring adopts a lock-free circular queue contiguous memory data structure. The hardware layer internally maintains a monotonically increasing write pointer to indicate the currently writable slots in the shadow ring. The formula for calculating the target storage address is as follows:
[0137] ;
[0138] In the formula, The calculated target storage address; This is the current count value of the write pointer; The total number of slots configured for the shadow ring; This is the starting base address of the shadow ring in the host memory or the onboard cache of the network card; Fixed byte length for a single descriptor (typically configured as 16 bytes or 32 bytes to align with the bus width).
[0139] The addressing logic based on the modulo operation described above is significant in that it constructs a reusable sliding window, avoiding unlimited memory consumption. The hardware layer performs this modulo operation to determine the target memory address to which the current descriptor is written. After storing the metadata descriptor in the memory slot, the write pointer is incremented.
[0140] Furthermore, to ensure the completeness of the algorithm logic and avoid the loss of valid states due to looping around the circular queue, the total number of slots... The configured value must be strictly greater than the maximum number of packets in transit allowed by the network link bandwidth-delay product (BDP). Simultaneously, the hardware layer maintains a virtual read pointer by comparing the latest acknowledgment sequence number of the underlying TCP or RDMA connection; when the write pointer... When the read pointer is about to catch up, the hardware layer triggers the underlying backpressure mechanism to pause transmission, effectively preventing newly generated descriptors from illegally overwriting metadata of packets within the shadow ring that have not yet been acknowledged by the peer. This storage structure update mechanism operates independently of the actual data transmission channel, maintaining steady-state network throughput performance at the hardware layer while fully recording the underlying state to cope with potential protocol degradation.
[0141] This invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual protocol stacks. During steady-state transmission, it implements congestion monitoring and degradation interrupt triggering mechanisms, which may include:
[0142] S401, the hardware-layer Media Access Control (MAC) sublayer monitors the network link's receive port in real time. Internally, the MAC sublayer maintains a dedicated timer register. This timer register is configured to accumulate the effective blocking duration represented by received priority flow control pause frames. Whenever the network link receives a priority flow control pause frame, the MAC sublayer's parsing module extracts the pause time quantum carried in the frame, converts the pause time quantum into a time value, and adds it to the timer register.
[0143] In step S402, the hardware layer firmware periodically reads the current accumulated value of the timer register and compares it with a preset time threshold. The preset time threshold represents the maximum congestion limit allowed by the network link. When the value in the timer register reaches the preset time threshold, the hardware layer determines that the current link faces a risk of network congestion and triggers internal exception handling logic.
[0144] S403, the hardware layer's exception handling logic constructs a specific interrupt vector. The hardware layer utilizes a message signal interrupt mechanism to send hardware interrupt commands to the host node via the peripheral component interconnect bus. The adaptation layer, deployed in the kernel layer, has corresponding interrupt service routines pre-registered. The adaptation layer captures the hardware interrupt command, triggers the protocol degradation callback procedure, and initiates the underlying state freeze and protocol stack switching process. For the layer parsing mechanism of priority flow control pause frames and the interrupt distribution principle of the peripheral component interconnect bus, those skilled in the art can refer to current Ethernet and bus standards for implementation; the underlying basic communication mechanisms are well-known technologies in the field and will not be elaborated upon here.
[0145] This invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual protocol stacks. After receiving a hardware interrupt command that triggers protocol degradation, the system executes a state freeze and cross-stack congestion inheritance mechanism based on homogeneous mapping, which may include the following steps:
[0146] S50, the adaptation layer issues a state modification command to switch the hardware queue pair to an empty send state, reads the last acknowledgment sequence number and extracts the payload length from the shadow ring, and calculates the target TCP wake-up sequence number using a homogeneous sequence number mapping algorithm. Specifically, this process includes the following sub-steps:
[0147] In S501, the adaptation layer responds to the hardware interrupt instruction and sends a status control instruction to the hardware layer. Upon receiving this instruction, the hardware layer modifies the underlying state machine control word, smoothly transitioning the hardware queue from a ready-to-send state to a emptied-for-send state. Understandably, in the emptied-for-send state, the hardware queue stops scheduling new transmission requests and only processes acknowledgment packets for in-flight packets in the network link. Once all transmitted packets have received acknowledgments or triggered a timeout retransmission timer, the adaptation layer reads the hardware layer's status register to obtain the hardware queue's current last acknowledgment sequence number. This last acknowledgment sequence number indicates the logical boundary of the data packet when the RDMA protocol stack stops transmission.
[0148] In S502, the adaptation layer uses the acquired last acknowledgment sequence number to perform a homogeneous sequence number mapping algorithm. Since the TCP protocol uses a byte-stream-based sequence number space, while the RDMA protocol uses a discrete-message-based sequence number space, the adaptation layer needs to perform cross-protocol stack addressing unit conversion based on the metadata recorded in the shadow ring. The adaptation layer starts with the base packet sequence number from the hardware queue when establishing a connection and ends with the last acknowledgment sequence number, traversing and reading the historical records stored in the shadow ring.
[0149] S503, the adaptation layer calculates the target TCP wake-up sequence number according to the following formula:
[0150] ;
[0151] In the formula, let To calculate the generated target TCP wake-up sequence number, The initial byte sequence number negotiated by the TCP socket during the connection establishment phase; summation symbol. independent variable The range of values is from to ;in The base packet sequence number of the hardware queue pair. The final confirmation serial number read. The index in the shadow ring is The formula above defines the payload length corresponding to the packet sequence number. The significance of this formula is that the adaptation layer derives the equivalent byte offset for the TCP protocol stack by summing the payload lengths of all successfully transmitted RDMA data packets acknowledged by the underlying hardware. This offset is used to provide state inheritance during heterogeneous protocol stack switching, ensuring the continuity of the data flow during the switching process.
[0152] S60, the adaptation layer reads the current transmission injection rate and round-trip time from the hardware layer, calculates the congestion window size using the bandwidth-delay product, and overwrites it into the TCP socket's transmission control block. Specifically, this process includes the following sub-steps:
[0153] In S601, the adaptation layer accesses the underlying congestion control engine by calling the control interface exposed by the hardware layer. The adaptation layer reads the status parameters of the hardware queue limiting the transmission rate from the congestion control engine, specifically obtaining the current transmission injection rate and the estimated round-trip time of the network link. These status parameters accurately reflect the available bandwidth and latency status in the network just before the protocol degradation is triggered.
[0154] S602, the adaptation layer performs cross-stack inheritance calculation of congestion state based on the bandwidth-delay product model to avoid the reduction in transmission throughput caused by the default slow start mechanism of the protocol when the TCP socket is woken up. The adaptation layer calculates the theoretical congestion window size using the following formula:
[0155] ;
[0156] In the formula, let The calculated theoretical congestion window size (in segments). To obtain the current injection rate, This is an estimated round-trip time. This represents the maximum segment length negotiated by the TCP socket. The above formula uses the product of sending bandwidth and link delay to derive the maximum number of concurrent packets that the network link can accommodate under current conditions. To ensure the validity of the protocol and avoid algorithmic dead zones, the adaptation layer further incorporates the TCP receiver's advertised window to truncate the theoretical congestion window size, resulting in the final written congestion window size. satisfy:
[0157] ;
[0158] In the formula, where This operation calculates the number of segments based on the latest received window announced by the peer. This prevents deadlock caused by a calculated window of 0, or large-scale packet loss caused by exceeding the peer's receiving capacity.
[0159] In S603, the adaptation layer locates the Transmission Control Block (TCB) memory structure corresponding to the TCP socket in the kernel layer. The adaptation layer then calculates and truncates the protected... The numerical values are used to update and replace the congestion window variable and slow start threshold variable in the transmission control block. In this embodiment, through this update operation, the TCP socket directly obtains the network probe state equivalent to that before the RDMA hardware shutdown, achieving smooth inheritance of the underlying congestion state. For the data structure and variable definitions of the TCP transmission control block in the kernel, those skilled in the art can refer to the network protocol stack design specifications of current operating systems; its basic storage form is well-known in the field and will not be elaborated upon here.
[0160] This invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual protocol stacks. After completing cross-stack synchronization of congestion state parameters, the system executes a discrete memory reassembly, protocol extension header encapsulation, and data transmission mechanism, which may include the following steps:
[0161] S70, the adaptation layer wakes up the TCP socket, retrieves the segment memory pointer from the shadow ring using the last acknowledgment sequence number as an index, and mounts the page frames of the data buffer to the send segment structure of the TCP socket. Specifically, this process includes the following sub-steps:
[0162] S701, the adaptation layer modifies the scheduling status word of the TCP socket, enabling it to obtain low-level scheduling privileges from the operating system and transition from the hibernation / suspended state to the running state. The adaptation layer directly overwrites the send sequence number variable inside the TCP socket transmission control block with the target TCP wake-up sequence number calculated in step S50.
[0163] S702, the adaptation layer uses the last acknowledgment sequence number obtained as the index base address for the search starting point. The adaptation layer reads the segment memory pointers corresponding to the data blocks to be sent from the shadow ring one by one according to the sending order. The segment memory pointers indicate the actual page frame base address and the relative offset within the page for that data payload in the data buffer.
[0164] In S703, the adaptation layer extracts the page frame base address and relative offset within the page pointed to by the obtained segmented memory pointers, and assigns these values to the segmented array elements of the TCP protocol stack socket buffer in the kernel layer. The operating system kernel no longer allocates new memory space for data copying; instead, it utilizes the underlying segmented array pointer redirection mechanism to achieve zero-copy reassembly of discrete memory across heterogeneous protocol stacks. For the basic data structure and segmented array organization of the TCP protocol stack socket buffer, those skilled in the art can refer to the network kernel source code of standard operating systems for configuration. Its basic memory organization is well-known in the field and will not be elaborated upon here.
[0165] In S80, the adaptation layer inserts an adaptive protocol extension header before sending the TCP packet payload. The peer node intercepts the packet, extracts the memory region index, queries the mapping table to restore the target address, and performs direct data placement by hardware. Specifically, this process includes the following sub-steps:
[0166] S801, the adaptation layer constructs a custom adaptive protocol extension header at the kernel layer. In a preferred embodiment, this adaptive protocol extension header is configured as a fixed-length data structure of 16 bytes, with internal fields divided into: a 4-byte memory region index field for storing a unique identifier for global negotiation; an 8-byte relative offset field for accurately pointing to the address offset of the current data block in the local data buffer; a 2-byte payload length field for recording the number of TCP payload bytes immediately following the extension header; and a 2-byte control flag field, which defines flags for triggering specific hardware behaviors (such as the aforementioned boundary forced alignment flag Flush_Align). The data field of the adaptive protocol extension header is written with the global memory region index generated in the handshake negotiation in step S20, and also with the relative offset mapped in the data buffer to the current reassembled packet payload. The adaptation layer appends this adaptive protocol extension header to the front end of the TCP packet payload that has completed zero-copy reassembly, and calls the hardware layer to inject the encapsulated downgraded packet into the network transmission.
[0167] In S802, before a downgraded TCP packet arrives at the network interface card (NIC) of the peer node and before a direct memory access (DMA) to the host memory is initiated, the programmable data path hardware parsing engine within the NIC hardware layer directly intercepts the packet. Based on the fixed-length data structure of the header, the hardware parsing engine directly extracts and strips the adaptive protocol extension header encapsulated in the standard TCP payload front-end, accurately parsing out the memory region index, relative offset, current packet data payload length, and related control flags by byte offset.
[0168] In S803, the network interface card (NIC) hardware layer uses the extracted memory region index as a search keyword to directly look up the security context mapping table maintained internally by the local hardware. The NIC hardware layer compares the validity of the hardware access key at the hardware level, and after the permission verification is passed, reads the receiver memory base address statically bound to the memory region index and the authorized total length of the buffer.
[0169] S804, the conversion of the target address satisfies the following calculation formula:
[0170] ;
[0171] In the formula, let For the calculated target address, To retrieve the memory base address from the mapping table, To accommodate the relative offset carried in the adaptive protocol extension header, S805, after the boundary check passes, the network card hardware layer directly schedules the internal direct memory access engine to start the write timing, completely bypassing the host operating system kernel network receive queue and soft interrupt handling branch, and directs the actual data payload carried by the TCP packet to the calculated destination address. The data is directly delivered and written to the corresponding page frame in the peer's data buffer, achieving zero-copy direct data placement in TCP degradation state through a closed loop at the network card hardware level.
[0172] This invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual-protocol stacks. During TCP protocol degradation transmission, the system evaluates the network status in real time to trigger a switchback mechanism, specifically including the following steps:
[0173] S90, during degraded transmission, the hardware layer monitors the network water level status. When the sample variance of the link round-trip time is lower than a set threshold and no retransmission is triggered, the adaptation layer intercepts the sending state machine and executes the reverse isomorphic mapping algorithm to locate the target RDMA packet sequence number, resets the hardware queue pair, and puts the TCP socket into a sleep state. The network water level status is used to characterize the proportion of space occupied by buffered data in the network interface card's (NIC) underlying send or receive queues and the severity of current network link congestion. The specific monitoring method is as follows: the hardware layer reads the occupancy rate of the NIC's internal buffer queue in real time and performs a comprehensive evaluation based on the fluctuation of the round-trip time. Since the sample variance of the link round-trip time can intuitively reflect the jitter of network latency, the smaller the variance, the more stable the network queue backlog. Therefore, monitoring the sample variance can accurately quantify the changing trend of the network water level. Specifically, this process includes the following sub-steps:
[0174] In S901, the hardware layer continuously collects link round-trip time samples during TCP communication and calculates the variance of the sample data within a fixed time window in the firmware. When the calculated variance is less than a set anti-surge threshold, indicating that the current network congestion is in a stable low-level zone, and the TCP protocol stack in the kernel layer has not triggered a retransmission timer for several consecutive probe cycles, the hardware layer determines that the network link congestion has been alleviated and stable transmission conditions are available. The hardware layer sends a back-switch interrupt signaling to the adaptation layer via the peripheral bus. In this embodiment, the time window is preferably configured to be 10 to 50 milliseconds, and the number of probe cycles is configured to be 3 to 5. The anti-surge threshold is set based on the idle link baseline variance calculated when the system establishes a connection, typically taking 1.2 to 1.5 times the baseline variance. For the TCP protocol stack's retransmission timer maintenance and round-trip time sampling mechanism, those skilled in the art can refer to the standard transmission control protocol specification documents, which will not be elaborated here.
[0175] In step S902, after receiving the interrupt signaling, the adaptation layer freezes the TCP socket's transmission control mechanism and suspends data enqueueing operations in the TCP protocol stack within the kernel layer. The adaptation layer reads the latest confirmed byte sequence number from the TCP protocol stack's transmission control block. Based on this byte sequence number, the adaptation layer executes a reverse isomorphic mapping algorithm to search for the corresponding target RDMA packet sequence number in the shadow ring. The specific implementation steps are as follows: Step 1: The adaptation layer reads the latest confirmed byte sequence number from the TCP protocol stack and subtracts it from the initial byte sequence number to calculate the total effective byte offset of currently confirmed transmissions. Step 2: Starting from the initial packet sequence number of the hardware queue pair, the adaptation layer sequentially traverses and reads the fixed-length descriptors in the shadow ring. Step 3: During the traversal, the effective payload length of the current data packet is accumulated one by one. Step 4: The accumulated sum is compared with the total offset. When the accumulated sum just exceeds the total offset, the traversal stops, and the data packet corresponding to this accumulation transition point is the desired target RDMA packet sequence number.
[0176] S903, it should be noted that because the TCP protocol uses a byte stream acknowledgment mechanism, the latest acknowledged byte sequence number may not be exactly aligned with the payload boundary of the RDMA data packet. To avoid search dead zones, the reverse search process must satisfy the following inequality interval positioning logic:
[0177] ;
[0178] In the formula, let This is the latest confirmed byte sequence number. This is the initial byte sequence number negotiated by the TCP socket during the connection establishment phase. The initial packet sequence number for the hardware queue pair. The target RDMA packet sequence number for the search location, and the independent variable for the summation symbol. The range of values is from to ; and These are the payload lengths corresponding to the packet sequence numbers recorded in the shadow ring. The adaptation layer is located in the shadow ring using... Starting from the beginning, the payload length of each data packet is accumulated. By finding the interval that satisfies the above inequality, the target RDMA packet sequence number is derived. The significance of this algorithm lies in its ability to accurately locate the target data packet containing the latest TCP acknowledgment byte within a discrete message sequence space. Furthermore, if unaligned residual bytes exist (i.e.... (Strictly greater than the sum of the preceding intervals on the left), the adaptation layer controls the TCP socket to encapsulate the remaining unacknowledged payload of this data packet. When sending this downgraded TCP segment, the adaptation layer enables the flush_align flag in the adaptive protocol extension header it carries. After the peer node intercepts and parses the enabled flag, it forcibly bypasses the operating system's delayed ACK mechanism and immediately returns a standard TCP ACK acknowledgment packet for the tail of the remaining bytes, thereby ensuring that the sending end obtains the latest acknowledgment byte sequence number. It can instantly and accurately advance to the boundary alignment position of the data packet and complete the protocol back-switch control logic in a closed loop.
[0179] S904, the adaptation layer sends control commands to the hardware layer to calculate the target RDMA packet sequence number. The sequence number is written into the hardware layer's transmit sequence number register. Subsequently, the adaptation layer issues a state modification instruction, modifying the underlying state control word to restore the hardware queue from an empty transmit state to a ready transmit state. Simultaneously, the adaptation layer revoks the underlying scheduling resources of the TCP socket, causing the TCP socket to re-enter a sleep state. After completing the register update and state machine reset, the communication data stream smoothly switches back to the RDMA hardware acceleration path, and the system returns to the initial steady-state mode of dual-stack communication.
[0180] This invention provides a network interface card (NIC) communication system that supports dynamic switching between RDMA and TCP dual protocol stacks, which may include: a cross-stack initialization module, a steady-state transmission and shadow synchronization module, a congestion monitoring and interrupt triggering module, a state freezing and congestion inheritance module, a discrete reassembly and degraded transmission module, and a state detection and protocol rollback module.
[0181] The cross-stack initialization module is used to execute the cross-stack addressing space establishment and security context initialization operations described in steps S10 to S20 of the above embodiments. The steady-state transmission and shadow synchronization module is used to execute the data direct access packet and metadata shadow ring synchronization operations described in step S30 of the above embodiments in a steady-state network environment. The congestion monitoring and interrupt triggering module is used to execute the priority flow control pause frame polling and abnormal interrupt triggering operations described in step S40 of the above embodiments. The state freezing and congestion inheritance module is used to execute the target TCP wake-up sequence number calculation based on homogeneous mapping and the congestion state cross-stack inheritance operation based on the bandwidth-delay product model described in steps S50 to S60 of the above embodiments. The discrete reassembly and degraded transmission module is used to execute the cross-heterogeneous protocol stack discrete memory zero-copy reassembly and the secure direct data placement operation based on the adaptive protocol extension header described in steps S70 to S80 of the above embodiments. The state detection and protocol back-switch module is used to execute the network waterline state monitoring and reverse homogeneous mapping protocol back-switch operation described in step S90 of the above embodiments during degraded transmission.
[0182] This invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual-protocol stacks. During the communication establishment phase, a cross-stack addressing space and security context initialization mechanism is executed, which may include the following steps:
[0183] S10, the adaptation layer intercepts the application layer's communication establishment request and initializes virtual queue pairs, binding the virtual queue pairs to hardware queue pairs and TCP sockets. Specifically, this process includes the following sub-steps: S101, the application layer initiates a network communication establishment request. The adaptation layer captures this communication establishment request in the system. To shield the application layer from the implementation differences of the underlying heterogeneous protocol stack, the adaptation layer instantiates virtual queue pairs in its local memory space. The virtual queue pairs expose a unified work request submission interface to the outside world, and their internal logical structure includes a virtual send queue and a virtual receive queue, serving as a unified proxy interface for data interaction by the application layer.
[0184] S102, the adaptation layer calls the underlying device driver to instantiate a hardware queue pair dedicated to the RDMA protocol stack within the network interface card (NIC) hardware at the hardware layer. Simultaneously, the adaptation layer creates a TCP socket in the operating system network protocol stack at the kernel layer. In this embodiment, after completing the connection establishment and initial sequence number negotiation on the control plane, the adaptation layer places its send queue in a suspended state. This suspended state indicates that the TCP socket only maintains a connection heartbeat during the initial steady-state phase of dual-stack communication, without actively participating in service data transmission or preempting the main processor's scheduling resources. S103, the adaptation layer extracts the unique identifier of the virtual queue pair and establishes a static mapping association between this unique identifier and the queue number of the hardware queue pair and the file descriptor of the TCP socket. The adaptation layer records the established static mapping association in the locally maintained connection context, completing the underlying dual-protocol stack resource allocation and static binding operation.
[0185] S20, the adaptation layer allocates a data buffer and completes cross-stack page frame mapping. The communicating parties interactively generate memory region indexes and associate them with local memory base addresses, writing them into the mapping table. Specifically, this process includes the following sub-steps:
[0186] In step S201, the adaptation layer requests a contiguous block of virtual memory pages from the operating system as a data buffer. Subsequently, the adaptation layer calls the operating system's memory locking interface to perform memory pinning on the data buffer, preventing the operating system from swapping out the corresponding page frames to the disk device when system memory resources are scarce. After the page pinning process is complete, the adaptation layer obtains the list of page frame addresses actually occupied by the data buffer by traversing the page table.
[0187] S202, the adaptation layer locates the Transmission Control Block (TCB) buffer structure of the TCP socket in the kernel layer. The adaptation layer utilizes the kernel's zero-copy mechanism to map and associate the acquired page frame address list with the shared segmented linked list of the TCP socket buffer. This operation allows the kernel-level TCP protocol stack and the underlying hardware RDMA protocol stack to share the same memory region, achieving unified addressing of heterogeneous protocol stack spaces from the perspective of underlying memory management. For the specific data structure of the TCP socket buffer and the underlying page table mounting principle of the operating system, those skilled in the art can refer to the memory management specifications of standard operating systems, and will not be elaborated further here.
[0188] S203, During the connection handshake negotiation between the two communicating ends, the adaptation layer interacts with the peer node to generate a globally unique memory region index (MemoryRegionIndex). This memory region index serves as a secure credential for direct cross-node access to the data buffer. The adaptation layer binds the negotiated memory region index, the local memory base address of the data buffer, and the hardware access key allocated by the hardware layer to construct a secure mapping entry. S204, To prevent memory access out-of-bounds errors that could lead to hardware malfunctions, the adaptation layer must strictly define address boundaries, calculated using the following formula:
[0189] ;
[0190] In the formula, The total number of page frames calculated; The total length in bytes of the data buffer requested by the application layer; This represents the standard memory page size of the operating system (typically configured as 4KB or 2MB); the symbol ⌈⌉ represents the rounding up operation. The significance of the above formula is that, considering the requested buffer length may not be an integer multiple of the system page size, the rounding up operation ensures that even unaligned bytes at the end of the buffer receive a complete page mapping, effectively avoiding out-of-bounds access dead zones that the hardware engine might trigger at the end of the data. The above calculation establishes the legal address space boundaries of the data buffer. Finally, the adaptation layer writes the constructed secure mapping entries and address space boundary data into a mapping table maintained internally by the hardware layer. The mapping table uses the memory region index as the primary key index, providing the underlying address translation basis and secure out-of-bounds verification benchmark for subsequent direct memory access packet execution by the hardware engine and secure direct data placement in degraded states.
[0191] Furthermore, to ensure the stability of the mapping table during long-term operation and the efficient utilization of limited storage space, the hardware layer configures the security context mapping table with dynamic lifecycle management and entry eviction policies. When the application layer actively destroys the communication connection, or when the TCP / RDMA underlying link triggers a timeout exception, the adaptation layer intercepts the disconnection signaling and triggers a resource reclamation mechanism, synchronously clearing the mapping entries associated with it in the mapping table through the hardware control interface. At the same time, for zombie entries left over due to improper handover, the hardware layer internally maintains an eviction mechanism based on the Least Recently Used (LRU) algorithm. When the mapping table capacity reaches the warning threshold, old entries that have not been accessed or matched for a long time are prioritized for overwriting, thereby preventing hardware-level addressing table overflow failure.
[0192] The present invention provides an electronic device, which may include: a processor, a memory, a communication interface, and a communication bus.
[0193] The processor, memory, and communication interface communicate and exchange signals and data via a communication bus. The communication interface integrates a network interface controller hardware that supports both RDMA and TCP dual-protocol stack processing logic, used to perform data transmission and reception operations across the network link. A computer program is pre-written and stored in the memory. When the processor executes this program, it implements the aforementioned network interface card communication method that supports dynamic switching between RDMA and TCP dual-protocol stacks.
[0194] This invention also provides a non-volatile computer-readable storage medium. This computer-readable storage medium stores computer-executable instructions. When executed by the processor of a computer system, the computer-executable instructions cause the computer system to perform all the processing steps of the network interface card communication method supporting dynamic switching between RDMA and TCP dual-protocol stacks described above. Regarding the specific storage form of the computer-readable storage medium, those skilled in the art can implement it using forms including, but not limited to, flash memory, read-only memory, random access memory, or solid-state drives. The basic storage and retrieval mechanisms are well-known technologies in the field and will not be elaborated upon here.
[0195] This invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual protocol stacks, and performs data transmission and metadata shadow synchronization mechanisms in a steady-state network environment. The method may include the following steps:
[0196] In S30, under a steady-state network environment, the hardware layer transmits data via the RDMA protocol. The direct memory access engine reads the data buffer, encapsulates the data, and simultaneously writes the segmented memory pointer, payload length, and sequence number to the shadow ring. Specifically, this process includes the following sub-steps:
[0197] In S301, under a stable network environment where there is no network congestion, the system operates in hardware-accelerated transmission mode. Data transmission requests submitted by the application layer are converted into underlying work requests. The direct memory access engine within the hardware layer obtains this work request and, based on the memory description information in the work request, directly reads the data payload to be sent from the data buffer. The direct memory access engine encapsulates the read data payload into a standard RDMA network packet and injects it into the network link to perform the transmission operation. For the working mechanism of the direct memory access engine and the basic encapsulation format of RDMA packets, those skilled in the art can refer to relevant network communication architecture specifications; its basic hardware transmission process is well-known technology in the field and will not be elaborated upon here.
[0198] In S302, during the parallel processing phase where the Direct Memory Access Engine (DME) fetches data and executes packet encapsulation, the hardware layer's internal processing pipeline bypasses and intercepts metadata. This hardware bypass listening logic extracts key metadata of the current network packet from the DME's internal registers in real time without initiating additional main memory reads or interfering with the main data transmission link. Specifically, the extracted metadata includes a segmented memory pointer indicating the storage location of the current data payload in the data buffer, the payload length carried by the current packet, and the packet sequence number assigned to the packet.
[0199] S303: After extracting the aforementioned metadata, the hardware layer constructs a fixed-length descriptor in a specific format in the underlying storage space. The hardware layer uses lock-free write logic to store this fixed-length descriptor in the storage slot corresponding to the shadow ring. The shadow ring is a pre-allocated contiguous circular storage queue in memory, specifically used to record recent historical transmission status information to provide underlying addressing reference during heterogeneous protocol stack switching.
[0200] S304, the calculation formula is as follows:
[0201] In the formula, let the starting memory base address of the shadow ring be... The number of bytes occupied by a single fixed-length descriptor is The maximum total number of descriptors that the shadow ring can hold is The current value of the monotonically increasing write pointer maintained by the hardware is... The hardware layer calculates the slot address mapped in the shadow ring for the current fixed-length descriptor. The hardware layer uses the calculated slot address The fixed-length descriptor is stored in the memory space pointed to by the address, and the write pointer is triggered after the write is completed. Incrementing. The addressing logic based on modulo operation described above is significant in constructing a reusable sliding window. To prevent dead zones caused by overwriting valid states, the maximum total number... It needs to be configured to be greater than the maximum number of packets in transit allowed by the network link bandwidth-delay product (BDP); and when the write pointer is about to wrap around and overwrite the metadata of a packet that has not yet received confirmation from the other end, the hardware layer triggers the underlying backpressure mechanism to suspend the writing of new descriptors, thereby ensuring the integrity of the state record while reasonably avoiding performance loss.
[0202] During steady-state transmission, the system assesses the link congestion status in real time to determine whether to trigger protocol degradation, which may include the following steps:
[0203] S40, the hardware layer polls priority flow control pause frames. When the continuous blocking time triggered by receiving a pause frame reaches a preset time threshold, a hardware interrupt instruction is generated and reported to the adaptation layer. Specifically, this process includes the following sub-steps:
[0204] In S401, during steady-state data transmission at the hardware layer, when a downstream device in the network link faces the risk of running out of its receive buffer, it sends a Priority Flow Control Pause Frame (PFCPauseFrame) to the upstream node. The media access control sublayer of the hardware layer is responsible for receiving and parsing the network data frames entering the link, and continuously extracting and identifying the parsed Priority Flow Control Pause Frames using internal logic circuitry.
[0205] In step S402, when the Media Access Control (MAC) sublayer identifies a priority flow control pause frame, it extracts the pause time quantum carried in the frame header. The hardware layer has a dedicated cumulative congestion register to record the effective congestion duration of the network link in a paused transmission state. The hardware layer's clock circuit increments the value of this cumulative congestion register according to the extracted pause time quantum, based on the system clock cycle. Furthermore, when the pause time ends and the link resumes normal transmission, the hardware layer automatically clears the cumulative congestion register to ensure that the statistical duration is the continuous congestion time, avoiding false triggering due to the accumulation of errors caused by intermittent minor congestion.
[0206] S403, the hardware-level comparison logic circuit compares the above values in real time during a preset clock cycle. The conditional logic satisfies the following relationship:
[0207] ;
[0208] In the formula, let the current continuous effective blocking duration recorded in the cumulative blocking register be . The system's preset time threshold for triggering protocol stack downgrade switching is... When the above conditions are met, the hardware layer determines that the current RDMA protocol based on the lossless network assumption is no longer suitable for the current link congestion environment, and the network transmission exceeds the congestion tolerance limit, triggering a backward-compatible protocol degradation operation. In this embodiment, the preset time threshold... The setting is based on the link end-to-end round-trip time (RTT), and is usually set to 3 to 5 times the base RTT, in order to effectively filter out transient network jitter.
[0209] S404: When the condition is met, the hardware layer initiates a cross-layer interrupt signaling transmission operation. The peripheral bus controller of the hardware layer constructs an exception interrupt vector and generates a hardware interrupt instruction on the host system's peripheral interconnect bus through a message signal interrupt mechanism (such as MSI-X). This hardware interrupt instruction carries an exception signature indicating severe network congestion and is routed from the underlying hardware to the adaptation layer running in the operating system kernel layer. After receiving the hardware interrupt instruction, the adaptation layer triggers a preset exception callback handler, suspends the current hardware accelerated transmission pipeline, and formally initiates the dual-stack protocol switching process.
[0210] This invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual protocol stacks. After triggering a degradation interrupt, it executes a state freeze based on homogeneous mapping and a cross-stack congestion inheritance mechanism, which may include the following steps:
[0211] S50, the adaptation layer issues a state modification command to switch the hardware queue pair to an empty send state, reads the last acknowledgment sequence number and extracts the payload length from the shadow ring, and calculates the target TCP wake-up sequence number using a homogeneous sequence number mapping algorithm. Specifically, this process includes the following sub-steps:
[0212] In S501, after the adaptation layer captures the hardware interrupt instruction, it sends a state modification control word to the hardware layer. The control unit of the hardware layer parses this control word and forcibly switches the operating state of the hardware queue pair from the normal sending state to the empty sending state. In the empty sending state, the hardware queue pair stops pulling new data payloads from the data buffer for packet encapsulation, and only waits for the network packets injected into the link to complete transmission or trigger a timeout. For the basic switching control instruction structure of the underlying hardware state machine, those skilled in the art can refer to the register programming manual of the standard network card device. Its basic register read / write mechanism is a well-known technology in the field and will not be described in detail here.
[0213] S502, the adaptation layer accesses the hardware layer's status register to read the latest and most valid last acknowledgment sequence number. The last acknowledgment sequence number indicates the sequence boundary position of consecutive network packets that the peer node has successfully received.
[0214] S503, the adaptation layer uses the obtained last acknowledgment sequence number as a reference point, and traverses and extracts the payload length of each network packet from the initial state to the last acknowledgment state in the shadow ring according to the recording order.
[0215] S504, the adaptation layer performs an isomorphic sequence number mapping algorithm based on the extracted payload length. The specific calculation formula is as follows:
[0216] In the formula, let the initial TCP byte sequence number negotiated by the TCP socket during the connection establishment phase be . The initial packet sequence number of the hardware queue pair is The final confirmation serial number read is The first recorded in the shadow ring The effective payload length of each package is The adaptation layer uses this algebraic equation to perform sequence number space transformation across heterogeneous protocol stacks and calculates the target TCP wake-up sequence number. This achieves strict alignment of the dual protocol stacks in terms of data byte stream transmission progress.
[0217] S60, the adaptation layer reads the current transmission injection rate and round-trip time from the hardware layer, calculates the congestion window size using the bandwidth-delay product, and overwrites it into the TCP socket's transmission control block. Specifically, this process includes the following sub-steps:
[0218] S601, the adaptation layer calls the status query interface exposed by the underlying hardware layer to obtain the network status parameters maintained in real time by the hardware congestion control algorithm; the specific extracted network status parameters include the current transmission injection rate of the hardware layer and the estimated round-trip time of the network link.
[0219] In S602, the adaptation layer uses the acquired network state parameters to perform cross-stack congestion state conversion and calculates the target congestion window size based on the bandwidth-delay product model. The specific calculation formula is as follows:
[0220] ;
[0221] In the formula, let the obtained injection rate be... The estimated round-trip time is The standard maximum segment length is ,
[0222] S603, the adaptation layer locates the transmission control block memory structure of the TCP socket in the kernel layer. The adaptation layer calculates the target congestion window size. This directly overwrites the congestion window state variable in the transmission control block. This direct overwrite operation enables the awakened TCP stack to obtain the true network congestion awareness state of the hardware layer before degradation, forcing the TCP stack to skip the slow start probing phase and directly resume data transmission at a rate adapted to the currently available network bandwidth.
[0223] This invention provides a network interface card (NIC) communication method that supports dynamic switching between RDMA and TCP dual-protocol stacks. During the dual-protocol stack switching, a discrete memory reorganization and secure direct data placement mechanism is executed, which may include the following steps:
[0224] S70, the adaptation layer wakes up the TCP socket, retrieves the segment memory pointer and corresponding length from the shadow ring using the last acknowledgment sequence number as an index, and mounts the page frames of the data buffer to the send segment structure of the TCP socket. Specifically, this process includes the following sub-steps:
[0225] In S701, the adaptation layer modifies the task scheduling status word of the kernel layer operating system to control the TCP socket stream in the suspended state to transition to the ready-to-run state. After the TCP socket obtains processor execution rights, the adaptation layer synchronously writes the target TCP wake-up sequence number calculated in the previous step into the TCP socket's transmission control block, triggering the TCP protocol stack to take over the subsequent data transmission tasks from the byte stream position corresponding to that sequence number. It is understood that those skilled in the art can refer to the standard operating system's network kernel stack implementation specifications for the basic scheduling and control logic of the TCP socket state machine, and will not be elaborated upon here.
[0226] In S702, the adaptation layer uses the sequence number of the next packet after the last acknowledgment sequence number returned by the hardware layer as an index to locate the storage slot corresponding to the starting network packet that has not yet received an acknowledgment frame in the shadow ring. The adaptation layer parses the fixed-length descriptor in this storage slot and extracts the recorded segmented memory pointer and the corresponding payload length. The segmented memory pointer explicitly points to the base address of the underlying page frame in the data buffer that actually stores the communication data to be sent.
[0227] In S703, the adaptation layer performs zero-copy reassembly of cross-stack discrete page frames. Leveraging the scatter-gather feature of the operating system's underlying network stack, the adaptation layer assigns the extracted segmented memory pointers and payload length to the scatter-gather array element in the TCP socket send buffer list. This operation maps the page frames in the data buffer as standard TCP data segments to the TCP send data structure, avoiding a secondary data copy process from the user-space address space to the kernel-space TCP protocol stack buffer. This effectively maintains near-zero-copy processing efficiency in degraded transmission mode.
[0228] In S80, the adaptation layer constructs and sends a downgraded message containing the adaptive protocol extension header. The peer node intercepts the message, extracts the addressing metadata, performs a security boundary check using a mapping table, and then the underlying hardware performs direct data placement. Specifically, this process includes the following sub-steps:
[0229] In S801, the adaptation layer inserts a custom adaptive protocol extension header as a pre-encapsulation at the beginning of the TCP data payload when constructing TCP network packets. This adaptive protocol extension header is used to synchronize critical addressing metadata necessary to maintain RDMA memory semantics with the peer node during TCP transmission mode downgraded to unstructured byte stream-based mode.
[0230] In S802, the structure fields of the adaptive protocol extension header at least include the globally negotiated memory region index and the relative offset of the current data segment relative to the start of the data buffer. The adaptation layer reads the address of the current segment memory pointer, calculates the difference between this address and the local memory base address of the data buffer, and generates the relative offset. The adaptation layer injects the TCP packet, with the extension header encapsulated and the data payload attached, into the kernel layer's network stack driver and sends it to the network link via the network card.
[0231] S803: After receiving the TCP downgrade message, the peer node triggers a low-level network interrupt and sends it up to the peer node's kernel network stack. Preferably, the adaptation layer running on the peer node has pre-registered a custom mount hook function in the low-level packet processing path of the kernel network protocol stack (such as eBPF or Netfilter mount point). This hook function intercepts the message before the network protocol stack performs regular TCP payload stripping to the upper-layer protocol, extracts the adaptive protocol extension header, and parses out the memory region index, relative offset, and the data payload length of the current message.
[0232] In step S804, the peer node's adaptation layer sends the extracted memory region index to the mapping table maintained by the peer's underlying hardware, triggering hardware-level security permission and base address lookup operations. If the mapping table lookup is successful and the memory access key verification passes, the peer node obtains the receiving end's memory base address statically bound to the memory region index, as well as the total authorized length of the buffer.
[0233] To prevent memory corruption caused by unauthorized tampering, the S805 requires the peer node to perform strict security boundary out-of-bounds checks. The check criteria must be met:
[0234] ;
[0235] In the formula, let the relative offset extracted by the peer node be... The current message's data payload length is The total length of the buffer authorization recorded in the mapping table is If the above inequality does not hold, the system determines it as illegal access or a message going out of bounds, discards the message, and generates an error log. If the above inequality holds, the system calculates the target address directly written by the hardware using the following formula. :
[0236] ;
[0237] In the formula, This is the receiver's memory base address obtained through a mapping table lookup. S806, after the boundary check passes, the adapter layer construction of the peer node includes the target address. The system sends a direct data placement instruction along with the payload data pointer to the peer node's underlying hardware. The peer node's hardware parses this instruction, schedules the direct memory access engine, bypasses the standard TCP kernel receive socket buffer pool, and places the actual data payload carried in the packet according to the destination address. It directly writes to the page frame corresponding to the peer's data buffer, thus effectively implementing a zero-copy reception mechanism for direct memory access at the remote end in the TCP degradation communication link.
[0238] Specific application examples:
[0239] Considering the high-concurrency data read / write operations between storage nodes and compute nodes in a specific distributed high-performance computing cluster, the constants and initial parameters set for the method are as follows:
[0240] The basic congestion window size is set to 64KB; the round-trip time smoothing factor is set to 0.125; the priority flow control pause frame duration threshold is set to 5.0μs; the send injection rate safety limit is set to 0.6; the network high watermark threshold (triggering degradation) is set to 85%; the network low watermark threshold (triggering backswitching) is set to 40%; and the minimum value determination threshold (time deviation) is set to 0.5μs.
[0241] During the closed-loop adaptive network state adjustment process performed by the computer (host node), the computer polls and calculates the network interface card (NIC) status register within the transmission control adaptation layer. At a certain iterative execution moment of sudden network congestion, the computer extracts the duration of the current priority flow control pause frame as 5.2 μs, exceeding the preset time threshold of 5.0 μs, triggering a hardware interrupt. The computer extracts the last acknowledgment discrete packet sequence number of the current RDMA hardware queue pair as 1050, the single packet payload length as 4096 bytes, the initial discrete packet sequence number of the initial connection as 1000, and the TCP initial byte sequence number as 20000. The computer substitutes these sequences into the isomorphic sequence number mapping formula to calculate the target TCP wake-up sequence number:
[0242] ;
[0243] The computer directly overwrites the calculated target TCP wake-up sequence number 224800 into the TCP socket context. Simultaneously, the computer extracts the current measured network round-trip time as 120μs, the baseline round-trip time as 50μs, and the current injection rate as 0.8. The computer then uses the congestion window size calculation formula to calculate the updated dynamic congestion window size:
[0244] ;
[0245] The computer re-introduces the calculated 16KB as a new window input data into the TCP socket send buffer model, regenerates a degraded TCP packet with an adaptive protocol extension header, eliminates buffer overflow or network avalanche caused by blind retransmission, and prevents a precipitous drop in throughput.
[0246] As downgraded TCP packets are continuously sent and Direct Data Placement (DDP) is received by the peer node, the system completes multiple rounds of network congestion release detection. After the latest round of state detection, the computer calculates the current network interface card (NIC) queue water level as 35%. Since the calculated water level of 35% is less than the preset low water level threshold of 40%, the judgment condition is triggered, and the computer performs round-trip time (RTT) deviation calculation. The computer extracts the measured RRT times from the last ten probes and sums the arithmetic mean difference with the baseline RRT time to obtain a time deviation of 0.4 μs. The time deviation of 0.4 μs is less than the preset minimum judgment threshold of 0.5 μs, so the computer sends a protocol switchback end signal, terminates the current TCP downgrade transmission loop, locates the target RDMA packet sequence number, locks the current lossless network state, and switches back to hardware RDMA direct transmission mode.
[0247] Actual operation tests were conducted and data were compared. Specific verification results are detailed in the appendix. Figure 9 and attached Figure 10 The experiment compared network communication data from a traditional standard TCP / IP protocol stack communication method with a cross-stack homogeneous network card communication and dynamic protocol switching method under the same 200G / 400G Ethernet card and the same high-concurrency I / O load conditions. The experimental data is recorded in Table 1.
[0248] Table 1. Comparison of operational data between the traditional standard TCP / IP protocol stack communication method and a cross-stack homogeneous network interface card communication and dynamic protocol switching method.
[0249] Traditional standard TCP / IP protocol stack communication methods 48.5 65.2 75.4 2.5 A method for cross-stack homogeneous network interface card communication and dynamic protocol switching 5.2 8.5 12.6 0.0
[0250] See attached document Figure 9 And Table 1, Appendix Figure 9 This graph compares the convergence of end-to-end average latency at various time points in the network communication process. The horizontal axis represents the cumulative test time in milliseconds (ms), ranging from 0 to 100. The vertical axis represents the end-to-end average latency in microseconds (μs), ranging from 0 to 60. The graph includes latency curves for a traditional control method and a latency convergence curve for a cross-stack homogeneous network interface card (NIC) communication and dynamic protocol back-switch method.
[0251] In the initial stage of network congestion, from 20ms to 40ms during testing, the traditional standard TCP / IP protocol stack communication method did not decouple the kernel protocol stack from the hardware queue. This caused it to degenerate directly into interrupt-driven kernel processing when congestion occurred, resulting in slow latency reduction and stagnation in the later stages. Ultimately, the average latency remained at 48.5μs and could not be reduced further. A cross-stack homogeneous NIC communication and dynamic protocol back-off method expands the hardware state to the TCP socket through an asynchronous metadata shadow ring, thus breaking free from the deep involvement limitation of the kernel protocol stack. The latency convergence curve of this cross-stack homogeneous NIC communication and dynamic protocol back-off method shows a stable and rapid control trend. It can quickly penetrate the minimum value judgment threshold of 10.0μs at 45ms, and the final latency is strictly controlled within 5.2μs, reducing the latency tolerance accumulation phenomenon caused by frequent context switching between kernel mode and user mode.
[0252] See attached document Figure 10 And Table 1, Appendix Figure 10 This diagram illustrates the CPU utilization and dynamic adjustment of the congestion window during congestion management for high-concurrency transmission nodes. The horizontal axis represents measurement time in milliseconds (ms), ranging from 0 to 100. The left vertical axis represents CPU utilization in percentage (%), also ranging from 0 to 100; the right vertical axis represents the ratio of the dynamic congestion window to the base window size, dimensionless, ranging from 0.1 to 1.0. The diagram includes a line graph showing CPU consumption using a traditional method, a line graph showing CPU consumption using a cross-stack homogeneous NIC communication and dynamic protocol back-switch method, and a curve showing the dynamic adjustment of the congestion window.
[0253] Around 30ms, due to sudden changes in cluster computing tasks, a micro-burst occurred in the network link, causing congestion. Traditional standard TCP / IP protocol stack communication methods lack a memory page frame cross-stack mapping mechanism, causing the measurement operation to directly exceed the data copy limit of the kernel buffer, resulting in packet loss and retransmission. The peak CPU utilization shown in the traditional method's CPU consumption curve rapidly increased to 65.2%.
[0254] A cross-stack homogeneous NIC communication and dynamic protocol backoff method dynamically updates the congestion window size by extracting the round-trip time. The dynamic window adjustment curve shows that when congestion approaches the packet loss threshold, the computer automatically reduces the congestion window size, strictly limiting the ratio of the dynamic congestion window to the basic window size to below 0.5, thus maintaining the NIC's transmission state within the safe throughput envelope. Simultaneously, combined with Direct Data Placement (DDP) of the secure context map, the CPU consumption curve of this cross-stack homogeneous NIC communication and dynamic protocol backoff method remains below 8.5% throughout the entire adjustment period, ensuring high efficiency in node communication and accuracy in zero-copy data transmission.
Claims
1. A network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks, characterized in that, Includes the following steps: The Transmission Control Adaptor layer binds hardware queue pairs to TCP sockets, maps page frames in the memory data buffer, and stores the memory region index into the security context mapping table of the network card hardware layer. During RDMA transmission, the network card hardware layer writes the segment memory pointer, payload length, and packet sequence number into the asynchronous metadata shadow ring; When congestion occurs, the Transmission Control Adaptor layer empties the hardware queue pair, calculates the target TCP wake-up sequence number and congestion window size based on the last acknowledgment sequence number and the asynchronous metadata shadow ring, and writes them into the underlying Transmission Control Block. The TCP socket is awakened, page frames are mounted according to the segmented memory pointer, and an adaptive protocol extension header containing the memory region index and relative offset is inserted before the message is sent. The peer extracts the adaptive protocol extension header, queries the security context mapping table to restore the target address, and performs direct data placement. When congestion is relieved, the Transmission Control Adaptor (TADA) updates the target RDMA packet sequence number in the asynchronous metadata shadow ring to the send register based on the latest confirmed sequence number, resets the hardware queue pair, and puts the TCP socket to sleep.
2. The network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks according to claim 1, characterized in that, The page frame of the mapped memory data buffer is specifically implemented as follows: The transmission control adaptation layer requests contiguous locked pages of memory from the operating system kernel as a memory data buffer; The transmission control adaptation layer registers a custom memory page pool across protocol stacks and registers the list of page frame addresses corresponding to the locked memory in the custom memory page pool, so that the page frame is visible to both the network card hardware driver and the operating system kernel network stack at the same time, in order to build an address space that is visible to both heterogeneous protocol stacks.
3. A network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks according to claim 1, characterized in that, The step of writing the segmented memory pointer, payload length, and packet sequence number into the asynchronous metadata shadow ring specifically includes: The programmable processing pipeline inside the network card hardware layer extracts the segmented memory pointer, the payload length, and the packet sequence number without interfering with the main data path, and packages them into a fixed-length descriptor. The descriptor is written into the asynchronous metadata shadow ring by incrementing the write pointer of the lock-free circular queue. The calculation logic of the target storage address is as follows: the current count value of the write pointer is moduloed by the total number of slots, the result of the modulo operation is multiplied by the number of bytes occupied by the descriptor, and the result of the multiplication is added to the starting base address to obtain the target storage address. When the write pointer is about to catch up with the virtual read pointer maintained based on the latest confirmation sequence number, a hardware-level flow control backpressure mechanism is triggered to suspend the scheduling of new sending requests.
4. A network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks according to claim 1, characterized in that, The congestion monitoring method during congestion specifically includes: The media access control sublayer of the network interface card hardware layer maintains a timer register, and uses the timer register to accumulate the effective blocking duration of the priority traffic control pause frame received by the port. When the value of the timer register exceeds a preset time threshold, a hardware interrupt instruction is generated and reported to the transmission control adaptation layer through a message signal interrupt mechanism, triggering the congestion degradation mechanism.
5. A network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks according to claim 1, characterized in that, The calculation of the target TCP wake-up sequence number is performed using a homogeneous sequence number mapping algorithm, and the specific steps include: The initial packet sequence number of the hardware queue pair is used as the starting point, and the last confirmation sequence number is used as the ending point; In the asynchronous metadata shadow ring, extract the payload lengths corresponding to all packet sequence numbers from the starting point to the ending point and sum them up; The target TCP wake-up sequence number is obtained by adding the accumulated payload length to the initial byte sequence number negotiated when the TCP socket is established.
6. A network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks according to claim 1, characterized in that, The calculation of the congestion window size specifically includes: The transmission control adaptation layer reads the current transmit injection rate and round-trip time estimate of the network interface card hardware layer; Congestion state cross-stack inheritance calculation is performed based on the bandwidth-delay product model. The current sending injection rate is multiplied by the estimated round-trip time, and the result of the multiplication is divided by the maximum segment length to calculate the theoretical congestion window size. The theoretical congestion window size is truncated and protected by combining the receiving window of the peer notification, and the final congestion window size is obtained and overwritten into the congestion window variable and slow start threshold variable of the underlying transmission control block.
7. A network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks according to claim 1, characterized in that, Mounting the page frame according to the segmented memory pointer specifically includes: The transmission control adaptation layer uses the last confirmed sequence number as the search starting point to read the segmented memory pointers one by one from the asynchronous metadata shadow ring; The page frame address and page offset pointed to by the segmented memory pointer are assigned to the scattered cluster array element in the TCP socket send buffer list to achieve zero-copy reassembly of discrete memory at runtime.
8. A network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks according to claim 1, characterized in that, The execution of direct data placement specifically includes: The peer node extracts the relative offset and the data payload length of the current message, and performs an out-of-bounds check: it determines whether the sum of the relative offset and the data payload length of the current message is less than or equal to the total authorized length of the buffer read from the security context mapping table; When the boundary check passes, the target address is calculated by adding the receiver memory base address retrieved from the security context mapping table to the relative offset. The network interface card (NIC) hardware layer bypasses the operating system kernel's network receive queue and schedules its internal direct memory access engine to write the data payload into the memory space pointed to by the target address.
9. A network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks according to claim 1, characterized in that, The congestion relief determination criteria include: The network card hardware layer continuously collects link round-trip time samples during communication and calculates the variance of the sample data within a fixed time window. When the variance value is less than the preset anti-oscillation threshold, and the kernel layer's TCP protocol stack does not trigger the retransmission timer within multiple consecutive probe cycles, it is determined that the network congestion has eased and a back-cut interrupt signaling is sent to the transmission control adaptation layer.
10. A network interface card (NIC) communication method supporting dynamic switching between RDMA and TCP dual protocol stacks according to claim 1, characterized in that, The process of locating the target RDMA packet sequence number in the asynchronous metadata shadow ring employs a reverse isomorphic mapping algorithm, and the search and location logic includes: The difference between the latest confirmed byte sequence number and the initial byte sequence number negotiated by the TCP socket is used as the difference between the latest confirmed byte sequence number and the initial byte sequence number; the payload length of the data packets starting from the initial packet sequence number is accumulated one by one in the asynchronous metadata shadow ring; The target RDMA packet sequence number must satisfy the interval positioning logic of the target RDMA packet sequence number: the cumulative payload length before the data packet is less than or equal to the difference between the latest confirmed byte sequence number and the initial byte sequence number, and the sum of the cumulative payload length before the data packet and the payload length of the data packet itself is strictly greater than the difference between the latest confirmed byte sequence number and the initial byte sequence number; If the difference between the latest confirmed byte sequence number and the initial byte sequence number is strictly greater than the cumulative payload length before the target RDMA packet sequence number, the system controls the TCP socket to send the remaining bytes as an independent TCP packet for single confirmation, and enables the boundary forced alignment flag in the adaptive protocol extension header to forcibly bypass the peer delay confirmation mechanism to align the packet boundary before executing the subsequent back-switch logic and state machine reset.