Network acceleration system and method based on DPDK kernel bypass

The network acceleration system, which bypasses the DPDK kernel, achieves collaboration between the user-space passthrough module, resource isolation module, and compilation optimization proxy module, thus solving the CPU resource contention problem in high-frequency trading scenarios and achieving a low-latency, high-throughput network acceleration effect.

CN123114722APending Publication Date: 2026-11-17SHANGHAI YISAILONG TECHNOLOGY CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202610897598.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-06-22
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In high-frequency trading scenarios, existing technologies suffer from performance jitter due to the disorderly contention for CPU resources between user-space network pass-through acceleration mechanisms and complex application computing logic. This fails to achieve the organic synergy between smart NIC hardware bypass acceleration, operating system-level fine-grained resource isolation, and extreme optimization of the compiler execution layer for high-level language applications.

Method used

By using a network acceleration system with DPDK kernel bypass, a user-space pass-through module, a resource isolation module, and a compilation optimization proxy module are established to achieve direct mapping from the physical receive ring buffer of the smart network card to the user-space virtual address space. Local interrupts are disabled, busy polling is performed, and vectorized instruction sequence replacement and cache locking are carried out to eliminate computational logic latency jitter.

Benefits of technology

It achieves network acceleration with both high throughput and low latency, stably achieves microsecond-level end-to-end low latency response, eliminates hardware interrupt preemption and computational logic latency jitter, and ensures the orderly coordination of CPU resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114722A_ABST
    Figure CN123114722A_ABST
Patent Text Reader

Abstract

This invention belongs to the field of server optimization, and particularly relates to a network acceleration system and method based on DPDK kernel bypass. This system maps the physical receive ring buffer of the smart network card to the user-space virtual address space through the DPDK driver, establishing a pass-through data path to the bypass operating system kernel protocol stack. It reserves a dedicated physical core by reserving kernel boot parameters and disabling local interrupts, binding the packet receiving thread to this dedicated physical core and performing busy polling in an interrupt-free state. When a new packet arrives, it is handled by the Java Virtual Machine, which intercepts the just-in-time compiler compilation event, performs vectorized instruction sequence replacement on the target methods marked with annotations, and performs last-level cache locking and synchronization block lock elimination on heap memory objects. This invention solves the microsecond-level performance jitter problem caused by the disordered contention for CPU resources between user-space network pass-through and complex computational logic, achieving a network acceleration effect that combines high throughput and low latency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of server optimization, and in particular relates to a network acceleration system and method based on DPDK kernel bypass. Background Technology

[0002] In fields such as high-speed securities and futures trading and high-performance computing, network latency and stability are key indicators determining system competitiveness. As trading instruction processing enters the microsecond-level competition, traditional network transmission and reception modes based on the operating system kernel protocol stack have become major performance bottlenecks. General-purpose kernel network stacks have inherent defects such as frequent system calls and context switching, multiple data copies, and uncontrollable interrupt handling, which easily lead to excessively high CPU processing latency and application performance fluctuations. Although current technologies attempt to improve performance from single dimensions such as kernel bypassing, hardware offloading, and thread isolation, the following technical challenges remain in the pursuit of extremely low latency and high determinism.

[0003] Patent CN110460540B discloses a low-latency, high-frequency trading system and method. It uses an FPGA to preprocess transaction data packets and then establishes a high-speed user-space channel with a PC via a DPDK high-speed network card's data processing unit, achieving hardware-software co-acceleration. However, this solution focuses on accelerating the hardware signal link between the FPGA and the DPDK network card, lacking co-optimization for application-layer memory access, garbage collection, and other computational logic jitter. It also does not address resource scheduling and instruction set-level optimization under complex workloads. Therefore, when facing complex applications such as transaction gateways, it is difficult to eliminate microsecond-level performance spikes caused by application-layer computation.

[0004] Patent CN117978694A discloses a method for dynamic CPU resource control of real-time DPDK traffic. It predicts future traffic using an LSTM model and dynamically adjusts the number of packet-receiving threads and CPU core distribution when packet-receiving capacity is insufficient. While this solution achieves dynamic resource scheduling based on traffic prediction, resource adjustment primarily focuses on meeting network packet-receiving capacity requirements. It does not accelerate execution from the compilation and execution levels, such as instruction set optimization and memory prefetching, nor does it design a CPU affinity scheduling mechanism tailored to business characteristics. Furthermore, it lacks end-to-end optimization methods for the deterministic execution of application-layer computational logic.

[0005] Patent CN117614901A discloses a high-efficiency parallel lock-free flow table state processing method and system based on DPDK. By establishing flow tables and timers corresponding to the number of cores, it achieves parallel lock-free classification processing of service flows. While this method improves parallel processing efficiency through lock-free design, its optimization mainly focuses on flow table state management. It does not address the deep integration of smart network interface card (NIC) hardware offloading capabilities and operating system-level tuning, nor does it consider the impact of load fluctuations on processor stability under overclocking conditions, making it difficult to achieve end-to-end deterministic low latency.

[0006] The existing technologies described above suffer from the following problems: While they have improved network latency through FPGA hardware offloading, flow-driven dynamic resource scheduling, and parallel lock-free flow table processing, none have achieved the organic synergy of intelligent network interface card hardware bypass acceleration, operating system-level fine-grained resource isolation, and extreme optimization of the compiler execution layer for high-level language applications. Therefore, how to eliminate the disorderly contention for CPU resources between user-space network pass-through acceleration mechanisms and complex application computing logic in scenarios such as high-frequency, intensive transactions, and how to resolve the performance jitter caused by the disconnect between their resource scheduling and execution optimization, constitutes a critical technical problem urgently needing to be solved in this field. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention proposes a network acceleration system and method based on DPDK kernel bypass. The system includes a user-space passthrough module, a resource isolation module, and a compilation optimization proxy module. Through the DPDK driver, the physical receive ring buffer of the smart network interface card is mapped to the user-space virtual address space, establishing a passthrough data path to the bypassed operating system kernel protocol stack. A dedicated physical core is reserved in the kernel boot parameters, and local interrupts are disabled. The packet receiving thread is bound to this dedicated physical core, performing busy polling in an interrupt-free state. When a new packet arrives, it is handled by the Java Virtual Machine, which intercepts the just-in-time compiler compilation event, performs vectorized instruction sequence replacement on the marked annotation target methods, and performs last-level cache locking and synchronization block lock elimination on heap memory objects. This invention solves the microsecond-level performance jitter problem caused by the disordered contention for CPU resources between user-space network passthrough and complex computational logic, achieving a network acceleration effect that combines high throughput and low latency.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A network acceleration system based on DPDK kernel bypass is applied to edge computing nodes. These edge computing nodes are deployed on a direct data channel between the physical receiving port of the smart network interface card and the service processing port of the user-mode application process. The system includes:

[0010] The user-mode passthrough module is configured to: obtain the passthrough data path between the user-mode application process and the smart network card. The passthrough data path is established by mapping the physical receive ring buffer of the smart network card to the virtual address space of the user-mode application process through the DPDK driver. It is used to enable the user-mode application process to bypass the operating system kernel protocol stack and directly access the packet payload data in the packet buffer data structure.

[0011] The resource isolation module is configured to: acquire a dedicated physical core bound to the packet receiving processing thread in the user-mode application process, wherein the dedicated physical core is reserved by kernel startup parameters and local interrupts are disabled, and redirect the smart network card hardware queue interrupt to the general physical core set, so that the packet receiving processing thread performs busy polling on the write pointer register value of the physical receive ring buffer in an uninterrupted state.

[0012] Specifically, the network acceleration system also includes:

[0013] The compilation optimization agent module is configured to: when the busy polling determines that a new message has arrived, hand over the message payload data to the Java Virtual Machine process running on the dedicated physical core for processing, and intercept the code compilation event triggered by the just-in-time compiler during the processing of the Java Virtual Machine process;

[0014] Based on the code compilation event, for the target method marked with vectorized optimization annotation in the Java Virtual Machine process, an instruction sequence replacement operation is performed to replace the loop body operation instruction sequence of the target method with the vectorized single instruction multiple data stream instruction sequence supported by the dedicated physical core;

[0015] Based on the heap memory objects marked with cache resident annotations in the Java Virtual Machine process, after global garbage collection is completed, a last-level cache locking operation is performed to retain the data of the heap memory objects in the last-level cache; at the same time, for synchronization blocks marked with lock removal annotations, their corresponding bytecode synchronization instructions are removed.

[0016] Specifically, the user-mode passthrough module includes a buffer aggregation unit and a page table mapping execution unit, wherein the buffer aggregation unit is configured as follows:

[0017] Obtain the number N of hardware receive queues reported by the smart network card, where N is an integer greater than 1;

[0018] Traverse the N physical receive ring buffers corresponding to the N hardware receive queues, and extract the physical base address and buffer length of each physical receive ring buffer;

[0019] In the virtual address space of the user-mode application process, a continuous virtual address range with a total length equal to the sum of the buffer lengths of the N physical receive ring buffers is calculated and reserved, and the starting virtual address of the continuous virtual address range is used as the aggregation base address.

[0020] Specifically, the page table mapping execution unit is configured to: use the aggregated base address as the starting address of the mapping target, use the list of physical base addresses of the N physical receive ring buffers as the set of mapping source addresses, and use the list of buffer lengths of the N physical receive ring buffers as the set of mapping size parameters, call the single-batch page table mapping interface provided by the DPDK driver to establish a complete page table mapping relationship between the continuous virtual address range and the N physical receive ring buffers in one go, so that the physical base address of the i-th physical receive ring buffer corresponds to the virtual address offset position obtained by adding the aggregated base address to the cumulative value of the buffer lengths of the previous (i-1) physical receive ring buffers, where i is an integer from 1 to N.

[0021] Specifically, the user-mode passthrough module further includes a direct reference establishment unit and a zero-copy access encapsulation unit: The direct reference establishment unit is configured to: obtain the address of the memory pointer field used to store message payload data in the message buffer data structure; update the pointer value stored in the memory pointer field address to the virtual address obtained by adding the mapping base address of the physical receive ring buffer in the user-mode virtual address space to the offset of the current message data in the physical receive ring buffer, so that the message payload data pointer in the message buffer data structure directly points to the original storage location of the message data in the physical receive ring buffer; The zero-copy access encapsulation unit is configured to: provide a message data access interface to the upper-layer business processing logic; when the upper-layer business processing logic requests to read the message payload data through the message data access interface, return a copy of the message payload data pointer, so that the upper-layer business processing logic directly performs a memory read operation on the original storage location of the message data in the physical receive ring buffer through the copy of the message payload data pointer, bypassing the heap memory allocator of the user-mode application process.

[0022] Specifically, the user-mode passthrough module further includes a status marking unit and a recycling synchronization unit: The status marking unit is configured to: after the application process completes processing the packet payload data in the packet buffer data structure, obtain the address of the status marking field used to record the occupancy status of buffer bits in the packet buffer data structure; update the status value stored at the status marking field address from a first status value indicating an occupancy status to a second status value indicating a release status, so that the buffer bits in the physical receive ring buffer corresponding to the packet buffer data structure are marked as pending recycling status; The recycling synchronization unit is configured to: traverse the packet buffer data structure in the packet buffer data structure where the status marking field value is the second status value, extract the offset of the buffer bits marked as pending recycling status in the physical receive ring buffer, use the offset as a recycling parameter to call the buffer release interface of the DPDK driver, so that the smart network card returns the corresponding buffer bits to the allocatable space pool of the physical receive ring buffer according to the offset, and updates the free bit field of the descriptor table entry in the physical receive ring buffer to an allocatable status.

[0023] Network acceleration methods based on DPDK kernel bypass include:

[0024] Obtain the pass-through data path between the user-mode application process and the smart network card. The pass-through data path is established by mapping the physical receive ring buffer of the smart network card to the virtual address space of the user-mode application process through the DPDK driver. It is used to enable the user-mode application process to bypass the operating system kernel protocol stack and directly access the packet payload data in the packet buffer data structure.

[0025] Obtain the dedicated physical core bound to the packet receiving processing thread in the user-mode application process. The dedicated physical core is reserved by the kernel startup parameters and local interrupts are disabled. The smart network card hardware queue interrupt is redirected to the general physical core set so that the packet receiving processing thread can perform busy polling on the write pointer register value of the physical receive ring buffer in an uninterrupted state.

[0026] When the busy polling determines that a new message has arrived, the message payload data is handed over to the Java Virtual Machine process running on the dedicated physical core for processing, and during the processing of the Java Virtual Machine process, the code compilation event triggered by the just-in-time compiler is intercepted;

[0027] Based on the code compilation event, for the target method marked with vectorized optimization annotation in the Java Virtual Machine process, an instruction sequence replacement operation is performed to replace the loop body operation instruction sequence of the target method with the vectorized single instruction multiple data stream instruction sequence supported by the dedicated physical core;

[0028] Based on the heap memory objects marked with cache resident annotations in the Java Virtual Machine process, after global garbage collection is completed, a last-level cache locking operation is performed to retain the data of the heap memory objects in the last-level cache; at the same time, for synchronization blocks marked with lock removal annotations, their corresponding bytecode synchronization instructions are removed.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] This invention establishes a direct mapping from the DPDK-driven network interface card (NIC) physical receive ring buffer to the user-space virtual address space through a user-space passthrough module. This allows application processes to bypass the kernel protocol stack and directly access packet data, fundamentally eliminating data copying and context switching overhead between kernel and user modes, thereby reducing single-packet processing latency. Furthermore, a resource isolation module binds the packet receiving thread to a dedicated physical core reserved by the kernel startup parameters via a CPU affinity mask, and redirects the NIC hardware queue interrupt to a general-purpose core set. This allows the packet receiving thread to perform busy polling of the physical receive ring buffer write pointer register while local interrupts are disabled, eliminating hardware... Interrupt preemption triggers TLB entry eviction and cache line invalidation events, ensuring latency determinism under high load. Furthermore, by intercepting just-in-time compiler compilation events through the compiler optimization proxy module, vectorized instruction sequence replacement is performed on the target methods marked with annotations, and last-level cache locking and synchronization block lock elimination are performed on heap memory objects. This eliminates the latency jitter of the computation logic caused by just-in-time compilation re-optimization, garbage collection pauses, and cache misses in the Java Virtual Machine. Ultimately, it achieves orderly coordination between the user-space network passthrough acceleration mechanism and complex application computing logic in CPU resource utilization, and stably achieves microsecond-level end-to-end low-latency response in high-frequency and intensive scenarios such as high-speed transactions. Attached Figure Description

[0031] Figure 1 This is a block diagram of a network acceleration system based on DPDK kernel bypass provided in Embodiment 1 of the present invention;

[0032] Figure 2 This is a flowchart illustrating the implementation logic of the user-mode pass-through module provided in Embodiment 2 of the present invention.

[0033] Figure 3 The implementation logic flowchart of the resource isolation module provided in Embodiment 3 of the present invention;

[0034] Figure 4 This is a flowchart illustrating the implementation logic of the compiler optimization proxy module provided in Embodiment 4 of the present invention.

[0035] Figure 5 The flowchart of the network acceleration method based on DPDK kernel bypass provided in Embodiment 5 of the present invention is shown. Detailed Implementation

[0036] Example 1

[0037] Please see Figure 1 This invention provides an embodiment of a network acceleration system based on DPDK kernel bypass, applied to an edge computing node. The edge computing node is deployed on a direct data channel between the physical receiving port of the smart network interface card and the service processing port of the user-mode application process, comprising:

[0038] The user-mode passthrough module is configured to: obtain the passthrough data path between the user-mode application process and the smart network card. The passthrough data path is established by mapping the physical receive ring buffer of the smart network card to the virtual address space of the user-mode application process through the DPDK driver. It is used to enable the user-mode application process to bypass the operating system kernel protocol stack and directly access the packet payload data in the packet buffer data structure.

[0039] The resource isolation module is configured to: acquire a dedicated physical core bound to the packet receiving thread in the user-mode application process, wherein the dedicated physical core is reserved by the kernel startup parameters and local interrupts are disabled, and redirect the smart network card hardware queue interrupt to the general physical core set, so that the packet receiving thread performs busy polling on the write pointer register value of the physical receive ring buffer in an uninterrupted state.

[0040] The compilation optimization agent module is configured to: when the busy polling determines that a new message has arrived, hand over the message payload data to the Java Virtual Machine process running on the dedicated physical core for processing, and intercept the code compilation event triggered by the just-in-time compiler during the processing of the Java Virtual Machine process;

[0041] Based on the code compilation event, for the target method marked with vectorized optimization annotation in the Java Virtual Machine process, an instruction sequence replacement operation is performed to replace the loop body operation instruction sequence of the target method with the vectorized single instruction multiple data stream instruction sequence supported by the dedicated physical core;

[0042] Based on the heap memory objects marked with cache resident annotations in the Java Virtual Machine process, after global garbage collection is completed, a last-level cache locking operation is performed to retain the data of the heap memory objects in the last-level cache; at the same time, for synchronization blocks marked with lock removal annotations, their corresponding bytecode synchronization instructions are removed.

[0043] As a concrete example, in an edge computing node of a high-speed securities and futures trading scenario, this node is deployed at the near-end network access layer of the exchange's data center. The smart NIC continuously receives UDP multicast market data messages from the trading switch via a physical receiving port. The user-space passthrough module described in this application uses the DPDK driver to batch map the physical receiving ring buffers corresponding to the four hardware receiving queues of the smart NIC to the user-space virtual address space of the Java trading gateway process, establishing a zero-copy passthrough data path from the physical receiving port directly to the Java off-heap memory. The resource isolation module binds the packet receiving processing thread to a dedicated physical core (such as core 4) and redirects all hardware interrupts to a general-purpose physical core set (cores 0-3), enabling the packet receiving thread to perform busy polling of the ring buffer write pointer without interruption. When a new message arrives, the compilation optimization proxy module intercepts the just-in-time compiler compilation event in the Java Virtual Machine process on core 4, performs AVX-512 vectorization instruction replacement on the market data decoding method annotated with @Vectorize, and performs last-level cache locking on the market data snapshot cache annotated with @CacheResident. Thus, this edge computing node, through a three-pronged approach of user-space pass-through, resource isolation, and compilation optimization, compresses end-to-end message processing latency jitter from the microsecond level (approximately ±50 microseconds) to a controllable nanosecond level (within approximately ±5 microseconds) on the direct data channel between the physical receiving port and the service processing port, achieving stable coexistence of low latency and high throughput at a peak throughput of 100,000 messages per second.

[0044] In this embodiment, the user-space pass-through module obtains the physical base address and buffer length of the physical receive ring buffer by passing the port identifier and hardware receive queue index number of the smart network card through the rte_eth_rx_queue_setup interface provided by the DPDK driver; it then calls the mmap system call, passing the physical base address and the buffer length, and the operating system kernel establishes a page table mapping relationship between the physical memory page frames of the physical receive ring buffer and a continuous virtual address range in the virtual address space of the user-space application process; after the mmap call returns the starting virtual address of the continuous virtual address range, the user-space application process forcibly casts the starting virtual address to a pointer of type struct rte_mbuf, and directly executes memory read or memory write instructions on the packet payload data in the physical receive ring buffer through this pointer. When the memory read or memory write instructions are executed, the processor memory management unit directly converts them into accesses to the physical memory page frames of the physical receive ring buffer according to the established page table mapping relationship, without going through any data copy processing functions in the operating system kernel protocol stack.

[0045] As a specific example, in this embodiment, the smart network card port identifier is 0, the hardware receive queue index number is 0, and the rte_eth_rx_queue_setup interface returns the physical base address PhysBase=0x10000000 and the buffer length BufLen=67108864 bytes of the physical receive ring buffer.

[0046] The first parameter, NULL, indicates that the operating system kernel automatically selects a contiguous unused virtual address range as the starting address for mapping; the second parameter, BufLen, is 67108864 bytes, or 64MB, representing the length of the physical memory region to be mapped; the third parameter, PROT_READ|PROT_WRITE, specifies that the mapped region has read and write access permissions, allowing user-mode processes to execute load and store instructions on that region; the fourth parameter, MAP_SHARED|MAP_LOCKED|MAP_POPULATE, is a set of mapping flags, where MAP_SHARED enables write operations on the mapped virtual address range to be directly synchronized to the smart network card's physical buffer, MAP_LOCKED locks the mapped physical page frames in memory to prevent them from being swapped to disk, and MAP_POPULATE requires the kernel to pre-fill all page table entries of the process page table before the system call returns to avoid triggering page faults during subsequent accesses; the fifth parameter, fd, is the open device file descriptor used to identify the system file corresponding to the smart network card; the sixth parameter, PhysBase, is 0x10000000, which is the physical base address of the smart network card's physical receive ring buffer. The entire mapping operation means: through a single system call, a continuous virtual address range of 64MB is allocated in the virtual address space of the user-mode application process. This range is then mapped to the smart network card's physical receive ring buffer starting from physical address 0x10000000. Simultaneously, physical page frames are locked, page tables are pre-filled, and write operations directly reach the hardware. This allows subsequent user-mode processes to directly access the packet data received by the network card using memory read / write instructions via the returned virtual address, completely bypassing the operating system kernel protocol stack and data copying process. The starting virtual address VirtBase=0x7f0000000000 returned by mmap is forcibly cast to struct rte_mbuf *mbuf_pool. When it is necessary to read the packet payload data at offset 0x1000, it is done through mbuf_pool[0x1000 / sizeof(struct rte_mbuf)] or directly through (volatile). The instruction `uint32_t)(VirtBase+0x1000)` executes a memory read instruction. The `volatile` keyword ensures that the compiler will not optimize away this memory read operation, guaranteeing that each read directly accesses physical memory. This instruction is used by the processor's MMU to directly access the packet data at physical address 0x10000000+0x1000=0x10001000 based on the page table mapping. The entire process, from executing the memory read instruction to loading the data into the processor register, does not call `copy_from_user`, `skb_copy`, or any kernel protocol stack data copy functions.

[0047] In this embodiment, the resource isolation module obtains a list of reserved dedicated physical core identifiers by parsing the kernel startup parameter isolcpus in the / proc / cmdline file; it calls the pthread_setaffinity_np function, passing in the thread identifier of the packet receiving processing thread and the CPU affinity mask. In the CPU affinity mask, only the bit corresponding to the first physical core identifier in the physical core identifier list is 1, and the remaining bits are 0, thus binding the packet receiving processing thread to the dedicated physical core. Kernel startup parameters nohz_full=4,5,6,7 disable periodic timer interrupts on the dedicated physical core, migrating all timer events to the general-purpose core; rcu_nocbs=4,5,6,7 removes the specified dedicated physical core from the RCU callback processing list, causing RCU callbacks to be migrated to the general-purpose core for execution, thereby preventing RCU callbacks from executing on the dedicated physical core and preempting the packet receiving thread; irqaffinity=0-3 sets the default affinity of all hardware interrupts to the general-purpose core set 0-3. These three parameters together achieve a zero-interrupt running environment for the dedicated physical core. The specific operation process of the busy polling is as follows: The packet receiving processing thread executes the following steps in an infinite loop: read the write pointer register value of the physical receive circular buffer, compare the current read value with the historical write pointer value stored in the previous loop; if the current read value is not equal to the historical write pointer value, it is determined that a new packet has arrived in the physical receive circular buffer, the packet payload data is handed over to the Java Virtual Machine process for processing, and the historical write pointer value is updated to the current read value; if the current read value is equal to the historical write pointer value, it is determined that no new packet has arrived, the packet processing logic is not executed, and the thread directly enters the next loop to read the write pointer register value. In this embodiment, among the kernel boot parameters, isolcpus is used to isolate the specified physical core from the kernel general scheduler, preventing it from being preempted by ordinary processes; nohz_full is used to disable the periodic clock tick interrupt on the specified physical core and migrate timer events to other cores to reduce unnecessary wake-ups of dedicated cores; rcu_nocbs is used to offload the RCU callback processing of the specified core to the RCU callback thread of other cores, avoiding background maintenance tasks from being executed on dedicated cores; irqaffinity is used to set the default route target core for all hardware interrupts, centralizing interrupt handling to the general core set, so that dedicated physical cores are not affected by any hardware interrupts.

[0048] As a specific example, in this embodiment, ` / proc / cmdline` parses to obtain `isolcpus=4,5,6,7`, the dedicated physical core identifier list is `{4,5,6,7}`, and the thread identifier of the packet receiving processing thread is `tid_rx`. It calls `pthread_setaffinity_np(tid_rx,sizeof(cpu_set_t),&cpuset)`, where bit 4 of `cpuset` is 1 and the remaining bits are 0. After the packet receiving processing thread is bound to physical core 4, it enters a busy polling loop: defining the variable `volatile uint32_t curr_wptr`, `prev_wptr=0`; in the first loop, it uses `curr_wptr=*(volatile...)` to... The `uint32_t*)(BAR0_base+WPTR_REG_OFFSET)` function reads the write pointer register value and obtains `curr_wptr=0x0000`. It compares this value with `prev_wptr=0`, and `0x0000==0` is true, indicating no new packet. `prev_wptr` is then kept at 0, and the second loop begins. In the second loop, the smart network card has written a packet to offset 0x1000 via DMA and updated the write pointer register to 0x1000. `curr_wptr=0x1000` is read and compared with `prev_wptr=0`. `0x1000!=0` is true, indicating a new packet has arrived, triggering the packet processing flow. After processing, `prev_wptr=0x1000` is set back, and the third loop continues polling. The busy polling loop runs on dedicated physical core 4 with local interrupts disabled, and is not affected by hardware interrupt preemption. Each loop reads the value of the write pointer register directly from the PCIe BAR space mapped address through a single MOV instruction, which takes about 3-5 processor clock cycles.

[0049] In this embodiment, the compiler optimization agent module is attached to the Java Virtual Machine process using the parameters -javaagent: / opt / esolo / compiler-agent.jar -XX:+EnableJVMCI -XX:+UseJVMCICompiler specified when the Java Virtual Machine process starts. After attachment, the compiler optimization agent module registers the implementation class of the ClassFileTransformer interface in the premain method, and traverses all methods of the loaded classes in the transform callback method. It performs bytecode analysis on the target methods marked with the vectorized optimization annotation, and replaces the scalar operation bytecode instruction sequence of the loop body with the vectorized bytecode instruction sequence corresponding to the AVX-512 instruction set supported by the dedicated physical core. The specific operation process for intercepting the code compilation event triggered by the just-in-time (JIT) compiler is as follows: The compilation optimization proxy module registers a compilation event callback through the CompilationRequestListener of the OpenJDK JVMCI interface. When the JIT compiler of the Java Virtual Machine process is about to compile a target method marked with vectorized optimization annotations, the compilation optimization proxy module intercepts the compilation request and queries the pre-compiled cache mapping table to see if a replacement vectorized instruction sequence for the target method already exists. If it exists, the HotSpotJVMCIRuntime.getInstance().getCompiler().installCode() method is called to directly inject the pre-compiled machine code into the JIT compiler's code cache, skipping the intermediate representation generation and instruction selection stages of the JIT compiler.In this embodiment, among the Java Virtual Machine startup parameters and the JVMCI interface, `-javaagent` specifies the path to the Java agent JAR file, enabling the agent to be loaded and its `premain` method to run before the main method is executed; `-XX:+EnableJVMCI` enables the Java Virtual Machine Compiler interface, making JVMCI-related compilation functions available; `-XX:+UseJVMCICompiler` sets the JVMCI compiler as the Java Virtual Machine's just-in-time (JIT) compiler backend, allowing compilation requests to be intercepted and controlled by the JVMCI interface; `premain` is the entry point method of the Java agent, called after class loading and before the main method execution, used to perform agent initialization and bytecode converter registration; `ClassFileTransformer` is a class file converter interface defined in the Java standard library, used to modify class bytecode during class loading; `transform` is the callback method of this interface, called when the class is loaded or redefined to perform bytecode replacement operations; `AVX-512` is the High-Level Vector Extensions instruction set for Intel processors, supporting single-instruction multiple-data operations on 512-bit wide data at a time; OpenJDK... JVMCI is the Java Virtual Machine Compiler Interface provided by OpenJDK, allowing external programs to intervene in the just-in-time (JIT) compilation process. CompilationRequestListener is a compilation request listener defined in JVMCI, used to notify registered listeners when the JIT compiler triggers a compilation event. HotSpotJVMCIRuntime is the runtime environment instance of JVMCI in the HotSpot virtual machine, providing methods for accessing the compiler instance. InstallCode is a method defined in the compiler interface, used to install compiled machine code into the JIT compiler's code cache, making it directly callable in subsequent executions.

[0050] As a concrete example, in this embodiment, the Java Virtual Machine process startup command line adds the parameters `-javaagent: / opt / esolo / compiler-agent.jar -XX:+EnableJVMCI -XX:+UseJVMCICompiler` to load the compilation optimization agent and enable the JVMCI compilation interface. The compilation optimization agent registers a bytecode converter by calling `instrumentation.addTransformer(newVectorizeTransformer(),true)` in the `premain` method. The `VectorizeTransformer.transform` method performs bytecode analysis on the `processOrder` method annotated with `@Vectorize` in the class `com / esolo / trade / OrderValidate`. It detects that the loop body contains element-wise addition operations on the `int[]prices` array. Based on the ASM 9.0 bytecode generation framework, it replaces the original `iaload` and `iadd` scalar bytecode instruction sequences with vectorized bytecode instruction sequences corresponding to the AVX-512 `vpaddd` instructions. When the `processOrder` method is called to the JIT compiler's compilation threshold (default 10,000 times), the JIT compiler triggers a compilation event. The compilation optimization agent intercepts this event through the registered `CompilationRequestListener`, searches the pre-compiled cache mapping table using the fully qualified name and method descriptor of the `processOrder` method as keys, and returns the generated vectorized machine code byte array after a match. The `installCode` method is then called to inject the generated vectorized machine code into the code cache. The JIT compiler directly returns the entry address of this machine code without performing the control flow graph construction and instruction selection process, reducing the compilation time from approximately 350 microseconds before interception to approximately 8 microseconds. The last-level cache locking operation is implemented by calling the mlock system call. For objects marked with cache resident annotations, off-heap memory is allocated using sun.misc.Unsafe.allocateMemory to store their data. The off-heap memory address is stored in the object's nativeAddress field. When locking, the off-heap memory address and data length are passed in, and the operating system kernel marks the corresponding physical page frame as non-swapable, so that its physical page frame has the highest retention priority in the last-level cache replacement strategy, avoiding the Java garbage collector moving heap objects and causing the locked physical page frame to become invalid.

[0051] Example 2

[0052] Please see Figure 2It should be further explained that the user-mode pass-through module in this embodiment includes a buffer aggregation unit, a page table mapping execution unit, a direct reference establishment unit, a zero-copy access encapsulation unit, a status marking unit, a recycling synchronization unit, a pre-allocation management unit, a buffer pool acquisition unit, a fragment protection recycling management unit, and an in-pool return execution unit.

[0053] The buffer aggregation unit is configured to: obtain the number N of hardware receive queues reported by the smart network card, where N is an integer greater than 1; traverse the N physical receive ring buffers corresponding to the N hardware receive queues, extract the physical base address and buffer length of each physical receive ring buffer; calculate and reserve a continuous virtual address range in the virtual address space of the user-mode application process, with a total length equal to the sum of the buffer lengths of the N physical receive ring buffers, and use the starting virtual address of the continuous virtual address range as the aggregation base address;

[0054] In this embodiment, when the smart network interface card (NIC) dynamically activates a new hardware receive queue during operation, it obtains the physical base address PhysBase_N+1 and the buffer length Len_N+1 of the new physical receive ring buffer corresponding to the new hardware receive queue; it reads the current total length VirtTotalLen of the continuous virtual address range and calculates the total length of the expanded virtual address range. The system call extends the contiguous virtual address range to obtain the starting position NewAggrBase of the extended virtual address range; NewAggrBase+VirtTotalLen is used as the mapping base address of the newly added physical receive ring buffer in the contiguous virtual address range; the total length of the contiguous virtual address range is updated to NewVirtTotalLen, and the number of hardware receive queues N is updated to N+1, so that the aggregated layout of the contiguous virtual address range supports the dynamic hot addition of hardware receive queues without interrupting the established page table mapping relationship.

[0055] The page table mapping execution unit is configured to: use the aggregated base address as the starting address of the mapping target, use the list of physical base addresses of the N physical receive ring buffers as the set of mapping source addresses, and use the list of buffer lengths of the N physical receive ring buffers as the set of mapping size parameters, call the single-batch page table mapping interface provided by the DPDK driver to establish a complete page table mapping relationship between the continuous virtual address range and the N physical receive ring buffers in one go, so that the physical base address of the i-th physical receive ring buffer corresponds to the virtual address offset position obtained by adding the aggregated base address to the cumulative value of the buffer lengths of the previous (i-1) physical receive ring buffers, where i is an integer from 1 to N.

[0056] In this embodiment, before calling the single-batch page table mapping interface, the total length of the continuous virtual address range VirtTotalLen = ΣLen_i (i = 1 to N) is calculated, and the aggregate base address AggrBase is read. VirtTotalLen is compared with the sum of the buffer lengths of the N physical receive ring buffers, ΣLen_i. If VirtTotalLen == ΣLen_i, the mapping parameter verification is deemed successful, and the single-batch page table mapping operation is executed. If VirtTotalLen != ΣLen_i, the difference between the two, Diff = VirtTotalLen - ΣLen_i, is calculated, an error code containing the Diff value indicating mapping parameter verification failure is generated, and the mapping operation is aborted to avoid page table mapping out-of-bounds access due to accumulated buffer length calculation deviation.

[0057] The direct reference establishment unit is configured to: obtain the address of the memory pointer field used to store message payload data in the message buffer data structure; update the pointer value stored at the memory pointer field address to the virtual address obtained by adding the mapping base address of the physical receive ring buffer in the user-mode virtual address space to the offset of the current message data in the physical receive ring buffer, so that the message payload data pointer in the message buffer data structure directly points to the original storage location of the message data in the physical receive ring buffer;

[0058] In this embodiment, before updating the pointer value stored at the address of the memory pointer field, the memory protection flag ProtFlag is extracted from the ol_flags field of the struct rte_mbuf structure of the DPDK driver. If ProtFlag & PKT_RX_IP_CKSUM_BAD is not zero, the attribute flag of the packet payload data pointer is set to a read-only reference pointer PtrAttr = READ_ONLY_REF, so that the upper-layer service processing logic can only perform memory read operations and not memory write operations through the read-only reference pointer. When a memory write request to the read-only reference pointer is detected, a memory access violation error code is returned to protect the integrity of the original packet data in the physical receive ring buffer.

[0059] The zero-copy access encapsulation unit is configured to: provide a message data access interface to the upper-layer business processing logic; when the upper-layer business processing logic requests to read the message payload data through the message data access interface, return a copy of the message payload data pointer, so that the upper-layer business processing logic can directly perform a memory read operation on the original storage location of the message data in the physical receive ring buffer through the copy of the message payload data pointer, bypassing the heap memory allocator of the user-mode application process.

[0060] In this embodiment, before returning the copy of the packet payload data pointer, the mapped base address MapBase and the mapped end address MapEnd=MapBase+BufLen corresponding to the physical receive ring buffer in the user-mode virtual address space are read, where BufLen is the buffer length of the physical receive ring buffer; it is determined whether the value PtrVal of the packet payload data pointer satisfies MapBase PtrVal<MapEnd; if PtrVal<MapBase or PtrVal MapEnd, it is determined that the value of the packet payload data pointer exceeds the address range, the return of the copy of the packet payload data pointer is rejected, the return value is set to a null pointer NULL, and a pointer out-of-bounds error log record is generated at the same time, so as to prevent the upper-layer service processing logic from accessing an unmapped or reclaimed physical memory area through an invalid pointer.

[0061] The pre-allocation management unit is configured to: when the pass-through data path is established, acquire a preset pre-allocation quantity M of packet buffer data structures, where M is an integer greater than 1; in the virtual address space of the user-mode application process, pre-allocate the memory space for M packet buffer data structures through one continuous memory allocation operation, take the start address of each allocated packet buffer data structure as a node pointer, construct a preset buffer pool linked list containing M nodes, and store the head pointer of the preset buffer pool linked list in the global buffer pool management structure;

[0062] In this embodiment, during the running process of the application process, the current available node count AvailCount of the preset buffer pool linked list is polled every preset monitoring period T_monitor; when AvailCount<LowWaterMark, the quantity K to be supplemented is calculated as K=min(M-AvailCount, MaxExpandBatch), where LowWaterMark is a preset low water level threshold, and MaxExpandBatch is a preset maximum single expansion batch value; the memory space for K packet buffer data structures is pre-allocated through one continuous memory allocation operation, the newly allocated K node pointers are added in batches to the tail of the available node queue of the preset buffer pool linked list, and AvailCount is updated to AvailCount+K, so that the preset buffer pool has dynamic expansion capability during operation to cope with burst traffic peaks.

[0063] The buffer pool acquisition unit is configured to: when the application process needs to access new packet payload data, check the number of currently available nodes in the preset buffer pool linked list; if the number of currently available nodes is greater than zero, retrieve an available node from the head of the preset buffer pool linked list, return the first address of the packet buffer data structure corresponding to the available node, and update the head pointer of the preset buffer pool linked list to the next node pointer value of the available node. The retrieval and update operations are completed in a lock-free state through a single atomic comparison and exchange instruction, so as not to trigger the application process to perform dynamic memory allocation in the heap memory during runtime.

[0064] In this embodiment, before retrieving an available node from the head of the pre-set buffer pool linked list, the current value of the head pointer of the pre-set buffer pool linked list, Head_old, and the next node pointer value, Head_old->next, are used as the comparison value CompareVal=Head_old and the swap value SwapVal=Head_old->next for the atomic comparison swap instruction. In a single atomic operation CAS(&Head,CompareVal,SwapVal), the old value of the head pointer is read, the available node is retrieved, and the head pointer is updated simultaneously. If the atomic comparison swap instruction returns false, indicating that the head pointer was concurrently modified by another thread during execution, causing the comparison to fail, the current head pointer value, Head_new, is read again, and the atomic comparison swap operation is performed again with CompareVal=Head_new and SwapVal=Head_new->next. This retry process is repeated until the atomic comparison swap instruction returns true.

[0065] The status marking unit is configured to: after the application process completes the processing of the message payload data in the message buffer data structure, obtain the address of the status marking field in the message buffer data structure used to record the occupancy status of the buffer bit; update the status value stored at the status marking field address from a first status value representing the occupancy status to a second status value representing the release status, so that the buffer bit in the physical receive ring buffer corresponding to the message buffer data structure is marked as pending reclamation.

[0066] In this embodiment, after updating the status value to the second status value, the current system timestamp Ts_current is read and written into the preset release timestamp field Ts_release in the message buffer data structure. When the recycling synchronization unit performs traversal recycling, for each message buffer data structure whose status flag field value is the second status value, the difference between the current system timestamp and the release timestamp field value DeltaT=Ts_now-Ts_release is calculated. DeltaT is compared with the preset delayed recycling threshold DelayThreshold. Recycling operation is only performed on message buffer data structures that satisfy DeltaT>DelayThreshold to prevent premature recycling caused by application processes repeatedly accessing the same buffer bit in a very short time.

[0067] The recycling synchronization unit is configured to: traverse the message buffer data structure in which the status flag field value is the second status value, extract the offset of the buffer bit marked as to be recycled in the physical receive ring buffer, call the buffer release interface of the DPDK driver with the offset as the recycling parameter, so that the smart network card returns the corresponding buffer bit to the allocable space pool of the physical receive ring buffer according to the offset, and updates the free bit field of the descriptor table entry in the physical receive ring buffer to the allocable state;

[0068] In this embodiment, the recycling synchronization unit is further configured to: count the number of buffer bits successfully recycled this time R within one traversal recycling cycle; read the total number of buffer bits SlotTotalCap of the physical receive ring buffer, and calculate the recycling ratio Ratio=R / SlotTotalCap; compare Ratio with a preset batch recycling ratio threshold BulkReclaimThreshold; if Ratio>BulkReclaimThreshold, then after the end of this traversal recycling cycle, continuously scan the free bit fields of the descriptor table entries in the physical receive ring buffer, and count the number of consecutive free segments FragCount in all descriptor table entries marked as free; if FragCount>1, then perform a descriptor table entry rearrangement operation, merging discrete free buffer bits into a continuous free region with a starting offset of 0 by moving the descriptor table entry pointer, so as to improve the allocation hit rate of continuous buffers when writing subsequent packets. In this embodiment, the batch recovery ratio threshold is preset based on the statistical relationship between the total number of descriptor entries in the physical receive ring buffer and the number of buffer bits released in a single recovery cycle. Combined with the bursty characteristics of message arrivals in high-frequency trading scenarios and the dynamic changes in the fragmentation of the ring buffer's free bit fields, this threshold is set as a percentage value between 0 and 1. The preset basis is that when the single recovery ratio exceeds this threshold, it indicates that a large number of buffer bits are released in a concentrated manner within the same recovery cycle. This situation typically corresponds to the critical window where transaction traffic transitions from peak to trough periods. At this time, the free bit fields in the descriptor entries of the physical receive ring buffer are relatively discrete. If descriptor entry rearrangement is not performed in time, the hit rate of continuous buffer bit allocation will significantly decrease when subsequent burst traffic arrives, thereby affecting message writing efficiency and introducing additional allocation delays. Taking into account the trade-off between the execution overhead of a single reordering operation and the cumulative allocation latency caused by not performing reordering, the batch reclamation ratio threshold is set to 20%. That is, when the number of reclaimed buffer bits exceeds one-fifth of the total number of buffer bits, descriptor table entry reordering is triggered. This ratio avoids the processor cycle overhead caused by frequent reordering and ensures the continuity of free areas and the determinism of subsequent allocation in high-ratio release scenarios.

[0069] The fragment-protection type recycling management unit is configured to: after the application process completes processing the packet payload data in the current packet buffer data structure, intercept the release operation request initiated by the application process for the current packet buffer data structure; determine whether the target memory address of the release operation request falls within the address range of the continuous virtual address space block managed by the preset buffer pool linked list; if it falls within the address range, intercept the return operation performed to the general heap memory manager of the operating system in the release operation request, and mark the current packet buffer data structure as a state to be returned to the preset buffer pool;

[0070] In this embodiment, the start address PoolVirtStart and the end address PoolVirtEnd of the continuous virtual address space block are read; the target memory address TargetAddr of the release operation request is compared with PoolVirtStart and PoolVirtEnd; if TargetAddr>=PoolVirtStart and TargetAddr<PoolVirtEnd, it is determined that TargetAddr falls within the address range; if TargetAddr<PoolVirtStart or TargetAddr>=PoolVirtEnd, it is determined that TargetAddr does not fall within the address range, the parameters of the release operation request are transparently transmitted to the general heap memory manager of the operating system as they are, and the general heap memory manager of the operating system performs a standard heap memory release operation on TargetAddr, so as to ensure that the normal processing path of the memory release request outside the management range of the preset buffer pool is not blocked.

[0071] The pool return execution unit is configured to: insert the starting address of the current message buffer data structure marked as to be returned to the preset buffer pool state as a new node pointer into the tail of the available node queue of the preset buffer pool linked list, and update the current available node count of the preset buffer pool linked list to the original value plus 1, so that the current message buffer data structure is returned to the preset buffer pool for cyclic reuse, so as not to release the sub-ranges in the continuous virtual address space block back to the operating system general heap memory manager. In this embodiment, before inserting the new node pointer into the tail of the available node queue, the starting address PayloadStart and the total length PayloadLen of the message load data field of the message buffer data structure corresponding to the new node pointer, as well as the starting address StatusFieldAddr and the length StatusFieldLen of the status flag field, are obtained. The preset memory erasure function secure_zero_memory is called to perform a byte-by-byte zeroing write operation on the memory region with a length of PayloadLen starting from PayloadStart and the memory region with a length of StatusFieldLen starting from StatusFieldAddr, so that the message buffer data structure returned to the preset buffer pool does not contain the residual data from the previous use when it is retrieved and reused next time, so as to avoid data crosstalk between different transaction weeks. In this embodiment, the `secure_zero_memory` function is called by the pool-based return execution unit to perform a byte-by-byte zeroing operation on the packet payload data field and status flag field of the packet buffer data structure before it is returned to the preset buffer pool. This ensures that the returned packet buffer data structure does not contain any residual data from the previous use when it is retrieved and reused next time. The core feature that distinguishes this function from a regular `memset` is that it uses the `volatile` type modifier or compiler memory barriers to ensure that the memory zeroing operation is not judged as "useless code" and eliminated during the compilation optimization stage, thereby guaranteeing that the zero-value write will definitely be executed in the final generated machine code. In high-frequency trading scenarios, message buffer data structures carried in the same physical memory area are reused cyclically during different trading weeks. The message payload data of the previous transaction may contain sensitive information such as user identity, trading instructions, and fund amounts. If the logical release is completed only by atomically updating the status flag field without physically zeroing the payload data field, the next transaction may read residual data from the previous transaction when reusing the same buffer bit due to program abnormalities or pointer misuse, resulting in cross-transaction data interference or leakage of sensitive information.Therefore, the basis for using secure_zero_memory instead of ordinary memset is that the former guarantees the inevitable execution of the zeroing operation from the underlying level, fundamentally eliminating the security risk of residual data not being cleared due to compiler optimization, and ensuring the integrity and confidentiality of transaction message data during the cyclic reuse process of buffer bits.

[0072] As a specific example, in this embodiment, the initialization and operation process of the user-mode pass-through module includes: after the system powers on, the buffer aggregation unit sends a hardware information query request to the smart network card to obtain the number of currently active hardware receive queues N=4, as reported by the smart network card firmware. The basis for taking the number of hardware receive queues N as 4 in this embodiment is that: the selected smart network card has 2 physical ports, and each port supports a maximum of 2 hardware receive queues when RSS (Receiver Scaling) is enabled; therefore, the maximum value of N is 2×2=4. To ensure that all hardware receive queues are included in the aggregation mapping, N is taken as the actual number of active queues reported by the network card firmware. The four physical receive ring buffers corresponding to queue numbers qid=0, 1, 2, 3 are traversed sequentially. The physical base address register value of each physical receive ring buffer is read from the base address offset of BAR0 in the PCIe configuration space of the smart network card at 0x1000+qid×0x100, resulting in PhysBase_0=0x10000000, PhysBase_1=0x20000000, PhysBase_2=0x30000000, and PhysBase_3=0x40000000. The descriptor table entry depth register value DescNum_i=32768 and the buffer segment length register value SegLen_i=2048 bytes corresponding to a single descriptor are read from each buffer. The buffer length Len_i of each physical receive ring buffer is calculated as DescNum_i×SegLen_i=32768×2048=67108864 bytes=64MB. In this embodiment, the length of a single buffer segment is configured to 2048 bytes because: the standard Ethernet MTU is 1500 bytes, plus the 14-byte Ethernet frame header, 4-byte VLAN tag, 4-byte frame check sequence, and DPDK encapsulation header information, the maximum length of a single frame is 1522 bytes. Rounding the segment length up to a power of 2, 2048 bytes, satisfies the requirement for complete storage of a single frame and optimizes the storage efficiency of the CPU cache by aligning it to half the boundary of the memory page size (4KB). In the virtual address space of the user-mode application process, the mmap system call is used with addr=NULL and length=VirtTotalLen=256MB. The kernel allocates a continuous virtual address range with a starting address of AgggrBase=0x7f0000000000 and a length of 256MB in the user-mode virtual address space. mmap returns MAP_SUCCESS.In this embodiment, the total length of the virtual address range of 256MB is determined based on the following: each of the four physical receive ring buffers is 64MB, totaling 256MB. The consideration for using mmap for single allocation instead of segmented allocation is that single allocation ensures the contiguous virtual addresses of the four buffers, avoiding potential gaps in virtual addresses introduced by segmented allocation, and providing a continuous virtual address space foundation for subsequent page table batch mapping. Before calling the rte_extmem_batch_map batch page table mapping interface of the DPDK driver, the page table mapping execution unit calculates the total length of the continuous virtual address range VirtTotalLen = ΣLen_i = 256MB and reads the aggregate base address AggrBase = 0x7f00000000000. It then compares VirtTotalLen with the sum of the buffer lengths of the four physical receive ring buffers, ΣLen_i = 256MB, byte-by-byte with precision. If the comparison result is equal, the mapping parameter verification is considered successful.

[0073] The reason for using byte-by-byte precision comparison instead of coarse-grained comparison in this embodiment is that if the actual length of a buffer deviates from the calculated value by a single byte due to firmware version differences, byte-by-byte precision comparison can detect the error in time before mapping, avoiding out-of-bounds access across physical page boundaries during page table mapping, thereby protecting the integrity of the kernel page table structure. A batch mapping request structure containing four mapping entries is constructed: entry 0 is {phys=0x10000000,len=67108864,virt=0x7f0000000000}, entry 1 is {phys=0x20000000,len=67108864,virt=0x7f0004000000}, and entry 2 is {phys=0x30000000,len=67108864,virt=0 Entry 3 is {phys=0x40000000,len=67108864,virt=0x7f000C000000}. The `rte_extmem_batch_map` interface is called to submit all four entries to the kernel page table management module at once. The kernel iterates through the four entries to establish the mapping relationship between virtual page frame numbers and physical page frame numbers in the process page table, returning the batch mapping success flag `BATCH_MAP_OK`. This embodiment uses the batch page table mapping interface instead of line-by-line mapping because line-by-line mapping requires four kernel-to-user mode switches and four TLB refresh operations for page table entries, while batch mapping only requires one system call and one full TLB refresh, reducing the time spent in the mapping initialization phase from approximately 40 microseconds to approximately 12 microseconds.

[0074] The pre-allocation management unit reads the preset message buffer data structure pre-allocation quantity M=1024 from the configuration file. In this embodiment, the value of M is 1024 because: according to benchmark test data, at a peak transaction throughput of 100,000 transactions / second, the average lifespan of a single message from receipt to processing completion is approximately 15 microseconds. Based on the total physical receive ring buffer capacity of 32,768 buffer bits, 1024 pre-allocated message buffer data structures can cover approximately 15.6% of the buffer capacity, satisfying concurrent message access requirements while avoiding physical memory waste due to excessive pre-allocation; furthermore, 1024 is an integer power of 2, which matches the 64-byte alignment of the CPU cache line and the last-level cache capacity as an integer multiple. By calling the `posix_memalign` function, a contiguous memory block of size M × sizeof(mbuf_struct) = 1024 × 256 bytes = 262144 bytes is allocated in the virtual address space of the user-space application process. The memory alignment boundary is 64-byte cache line alignment, which aims to eliminate the false sharing problem caused by cross-cache line access. For each 256-byte sub-interval in this contiguous memory block, the first address of the sub-interval is taken as the first address of the message buffer data structure and assigned to the corresponding node pointer `NodePtr_j` (j = 0 to 1023). Traversing j = 0 to 1022, `NodePtr_j->next = NodePtr_{j+1}` is set in sequence, and the last node `NodePtr_1023->next = NULL`, a pre-buffered linked list structure containing 1024 nodes is constructed. The head node pointer `Head = NodePtr_0` and the current number of available nodes `AvailCount = 1024` are written into the global buffer pool management structure `GlobalPoolCtx`. In this embodiment, `mbuf_struct` is a custom message buffer data structure type defined in the user-space application process. It encapsulates all control information and data references involved in a message in the pass-through data path. As the basic node unit of the pre-set buffer pool linked list, it carries the complete lifecycle management of a message from reception to processing and recycling. This structure includes a message payload data pointer field, which stores a virtual address pointing to the original storage location of the message payload data in the physical receive ring buffer, allowing the upper-layer business processing logic to directly perform zero-copy reading of the message data written by the smart network card DMA through this pointer; a status flag field, which marks the current buffer bit as occupied or released with a binary mutex value, supporting lock-free state detection and atomic updates; a release timestamp field, which records the system time when the buffer bit is marked as released, for use in delayed recycling determination; and a next node pointer field, which links multiple message buffer data structures into a single linked list structure of the pre-set buffer pool linked list.The structure is designed to be 256 bytes in size, aligned with an integer multiple of the 64-byte cache line boundary. This alignment strategy is designed to ensure that each message buffer data structure completely covers all control information and pointer references required for message processing, while matching the structure size with the size of the processor's final cache line. This avoids the false sharing problem caused by storing a single structure across cache lines, thus ensuring that when the buffer pool acquisition unit retrieves a node in a lock-free state through a single atomic comparison and exchange instruction, the entire structure can be fully loaded within a cache line. This eliminates the additional memory transaction overhead and cache coherency protocol latency caused by cross-cache line access.

[0075] When the application process triggers the execution path for the first new message access, the buffer pool acquisition unit reads the current value of GlobalPoolCtx.AvailCount as 1024, determines that 1024 > 0, and enters the lock-free dequeue operation: it loops to read the current value of the head pointer Head_old = GlobalPoolCtx.Head, determines whether Head_old is NULL, if Head_old == NULL, it returns a null pointer and exits; if Head_old != NULL, it reads the value of Head_old->next as NextPtr; constructs an atomic comparison and swap instruction CAS(&GlobalPoolCtx.Head, CompareVal = Head_old, SwapVal = NextPtr), and executes a single atomic operation. The reason this embodiment uses the CAS lock-free algorithm instead of a mutex lock is that, in a high-frequency trading scenario of 100,000 transactions per second, the acquisition and release operations of a mutex lock incur an overhead of approximately 50 nanoseconds each time, and under high contention conditions, it will trigger the operating system kernel's thread scheduling mechanism, introducing microsecond-level uncontrollable blocking. The CAS instruction, on the other hand, is a CPU atomic instruction, taking approximately 10 nanoseconds per execution, and does not involve kernel mode switching, thus controlling the latency jitter of the buffer pool acquisition operation within the nanosecond range. If the CAS returns true, the node pointer AcquiredNode = Head_old is retrieved, GlobalPoolCtx.AvailCount is atomically decremented to 1023, and the starting address of the message buffer data structure corresponding to AcquiredNode is returned. If the CAS returns false, it indicates that GlobalPoolCtx.Head was concurrently modified by another thread during execution. Head_old = GlobalPoolCtx.Head is reread as the new comparison value, and the CAS loop is repeated. The second CAS operation returns true, successfully retrieving an available node.

[0076] The direct reference establishment unit queries the attribute field of the descriptor table entry corresponding to the buffer bit where the current packet data is located from the smart network card, and reads the memory protection flag ProtFlag=READ_ONLY (value 0x01); if ProtFlag==READ_ONLY is true, the attribute flag of the packet payload data pointer is set to PtrAttr=READ_ONLY_REF (value 0x01), which means that the pointer only allows the execution of load instructions (mov, vmovdqa, etc.) and prohibits the execution of store instructions (mov[ptr], reg, etc.), thereby ensuring the data integrity of the original packet data in the physical receive ring buffer in the parallel reading scenario at the hardware memory protection unit level; calculates the value of the packet payload data pointer PtrVal=MapBase+Offset=0x7f0000000000+0x1000=0x7f0000001000, and writes PtrVal into the packet payload data pointer field mbuf_struct.payload_ptr of the packet buffer data structure. Before returning a copy of the packet payload data pointer to the upper-layer service processing logic, the zero-copy access encapsulation unit reads the physical receive ring buffer mapping base address MapBase=0x7f0000000000 and the buffer length BufLen=67108864, calculates the mapping end address MapEnd=0x7f0000000000+67108864=0x7f0003FFFFFF; it determines that PtrVal=0x7f0000001000 satisfies 0x7f0000000000<=PtrVal<0x7f0004000000, and the pointer validity check passes. The unit then returns a copy of PtrVal to the upper-layer service processing logic, which directly executes the mov instruction to read the packet payload data from the physical receive ring buffer through this copy. The entire process does not trigger malloc calls or memcpy operations.

[0077] After the application process completes the service processing of the packet payload data in the current packet buffer data structure, the status flag unit reads the offset Offset=0x1000 of the packet buffer data structure, and obtains the status flag field address StatusFieldAddr=AcquiredNode+128-byte offset; the status value stored in StatusFieldAddr is atomically updated from OCCUPIED (value 0x55) to RELEASED (value 0xAA). In this embodiment, the reason why OCCUPIED is set to 0x55 and RELEASED is set to 0xAA is that: the binary representation of 0x55 is 01010101, and the binary representation of 0xAA is 10101010. The two values are mutually exclusive at every bit with no intermediate transition state, so that the completeness of status transition can be quickly verified through a single XOR operation in a concurrent environment. Meanwhile, these two values are different from common wild pointer values (such as 0x00000000, 0xFFFFFFFF) and memory fill patterns (such as 0xDEADBEEF) in the memory, which facilitates quick identification of the current status of the packet buffer data structure through a memory dump tool; after the update is completed, the current system timestamp Ts_current=1704067200000000 nanoseconds returned by clock_gettime is read, and Ts_current is written into the release timestamp field Ts_release at the 8-byte offset of StatusFieldAddr. The reclamation synchronization unit performs traversal reclamation at intervals of a preset reclamation period T_reclaim=100 microseconds. In this embodiment, the reason why the reclamation period is set to 100 microseconds is that: at a transaction throughput of 100,000 transactions per second, the average arrival interval between two adjacent packets is 10 microseconds. Setting the reclamation period to 10 times the average arrival interval can reclaim about 10 released buffer bits in batches within one reclamation period, thereby sharing the fixed overhead of reclamation traversal. At the same time, the 100-microsecond period will not cause the released buffer bits to stay in the allocatable space pool for a long time and affect the allocation efficiency of buffer bits for new packets. When traversing to the packet buffer data structure corresponding to AcquiredNode, StatusVal=0xAA is read, and it is determined that the condition StatusVal==RELEASED is satisfied; Ts_release=1704067200000000 nanoseconds is read, the current system timestamp Ts_now=1704067201500000 nanoseconds is obtained, and the difference DeltaT=1500000 nanoseconds is calculated; the preset delayed reclamation threshold DelayThreshold=1000000 nanoseconds is read, and it is determined that DeltaT=<1500000>1000000 is true.

[0078] The basis for setting the delay recovery threshold to 1,000,000 nanoseconds (i.e., 1 millisecond) in this embodiment is as follows: In the actual operation log analysis of the order optimization system, there are scenarios where the same trading terminal sends multiple cancellation and subsequent order replenishment orders in a very short period of time (usually within the range of 500-800 microseconds). If the buffer space is immediately recovered at the moment the message processing is completed, it may be that the original buffer space has been allocated to the new message when the replenishment order arrives, resulting in the risk of data overwriting. Setting the delay recovery threshold to 1 millisecond provides sufficient grace time for the above cancellation and replenishment order scenarios. At the same time, the 1-millisecond delay accounts for a very small proportion of the total end-to-end delay (42-74 microseconds for the Shanghai Stock Exchange link and 26-54 microseconds for the Shenzhen Stock Exchange link) and does not affect the overall response time of the system. The offset of the buffer bit in the physical receive ring buffer is Offset=0x1000. The rte_pktmbuf_free_bulk interface is called to return the buffer bit to the allocatable space pool. The smart NIC firmware locates the corresponding entry in the descriptor table according to the Offset value, and writes the free bit field DescEntry.free_flag of the entry from 0x0 to 0x1, completing a complete packet access and recycling cycle.

[0079] This embodiment aggregates multiple physical receive ring buffers into a continuous virtual address range through a buffer aggregation unit. The page table mapping execution unit then establishes a complete page table mapping relationship in one go through a single-batch page table mapping interface, merging multiple kernel-mode and user-mode switches into a single operation and eliminating the overhead of multiple page table refreshes introduced by line-by-line mapping. The pre-allocation management unit pre-allocates a fixed number of packet buffer data structures and constructs a pre-set buffer pool linked list when the pass-through data path is established, so that packet access does not require triggering runtime dynamic memory allocation, avoiding the impact of lock contention and memory fragmentation on determinism. The buffer pool acquisition unit uses a lock-free dequeue operation with a single atomic comparison and exchange instruction, replacing mutexes with processor atomic instructions, eliminating the uncontrollable blocking introduced by kernel thread scheduling triggered by mutexes under high contention conditions, and controlling the latency jitter of the buffer pool acquisition operation within the nanosecond range. The direct reference establishment unit directly points the packet payload data pointer to the original storage location of the packet data in the physical receive ring buffer, and the zero-copy access encapsulation unit returns the pointer after the pointer validity check is passed. The pointer copy allows upper-layer business processing logic to directly read memory from the physical receive circular buffer, bypassing the user-space heap memory allocator and eliminating the data copy overhead from the physical receive circular buffer to the user-space heap memory. A status marking unit uses mutually exclusive binary values ​​to mark the occupancy and release status of buffer bits. In a concurrent environment, a single XOR operation can verify the integrity of state transitions, preventing misreading of intermediate transition states. A reclamation synchronization unit performs traversal reclamation at a preset period and uses a delayed reclamation threshold to determine the delayed reclamation of released buffer bits, reserving a grace window for repeated access scenarios such as order cancellation and order replenishment within a very short time, avoiding the risk of data overwriting caused by immediate reclamation. A fragmentation-protected reclamation management unit intercepts release requests within the scope of the preset buffer pool, and the pool's return execution unit performs byte-by-byte zeroing of the returned message buffer data structure before inserting it into the available node queue for cyclic reuse. This prevents external fragmentation caused by releasing sub-ranges within consecutive virtual address space blocks back to the operating system's general-purpose heap memory manager and avoids crosstalk from residual data during different transaction weeks.

[0080] Example 3

[0081] Please see Figure 3 It should be further explained that the resource isolation module in this embodiment includes a NUMA topology awareness unit, an affinity binding execution unit, an interruption interference monitoring unit, a residual interrupt correction unit, an idle monitoring unit, a sleep control unit, an abnormal wraparound determination unit, a buffer reset correction unit, a thread expansion unit, and a queue binding unit.

[0082] The thread extension unit is configured to: obtain the number of hardware receive queues of the smart network card, and record it as the total number of queues; start a number of packet receiving processing threads equal to the total number of queues, so that each packet receiving processing thread corresponds to a hardware receive queue.

[0083] In this embodiment, the thread extension unit obtains the total number of queues by reading the receive queue count register at offset 0x60 in the PCIe configuration space of the smart network card. The receive queue count register has a bit width of 16 bits, and its value is calculated and written by the smart network card firmware during the driver initialization phase based on the number of physical ports of the network card, the RSS receiver scaling configuration parameters of each port, and the on-chip SRAM buffer capacity. If the read total number of queues is zero, it is determined that the smart network card hardware initialization is not complete, and the thread extension unit reads the register cyclically at a period of 100 microseconds until its value is non-zero. When starting the packet receiving processing thread, a number of thread instances equal to the total number of queues are created through the POSIX standard pthread_create interface, and the thread identifier of each packet receiving processing thread is stored in a thread identifier array, with the array index corresponding one-to-one with the hardware receive queue number.

[0084] The queue binding unit is configured to: obtain the physical core identifier of each dedicated physical core in the dedicated physical core set; set the CPU affinity mask of the j-th packet receiving processing thread to only contain the mask bits corresponding to the physical core identifier of the j-th dedicated physical core, so that the j-th packet receiving processing thread and the j-th dedicated physical core are bound one-to-one; and pass the write pointer register address of the physical receive ring buffer of the j-th hardware receive queue to the j-th packet receiving processing thread, so that the j-th packet receiving processing thread performs independent busy polling on the write pointer register value of the physical receive ring buffer of the j-th hardware receive queue, where j is an integer from 1 to the total number of queues.

[0085] In this embodiment, the physical core identifier of each dedicated physical core in the dedicated physical core set is obtained by the resource isolation module during the initialization phase from the kernel boot parameter isolcpus. The parsed physical core identifiers are arranged in ascending order and stored in a dedicated physical core identifier array. The CPU affinity mask adopts a 1024-bit bitmap structure, with each bit corresponding to a physical core number. Writing 1 to the bit corresponding to the physical core identifier of the j-th dedicated physical core indicates that the core is included in the affinity binding range. The write pointer register address is obtained through the rte_eth_dev_rx_queue_info_get interface of the DPDK driver. The receive queue information structure returned by this interface contains a wptr_reg_addr field, which stores the physical address mapping value of the hardware receive queue's write pointer register in the PCIeBAR space of the smart network card.

[0086] The NUMA topology awareness unit is configured to: read the device local NUMA node attribute value in the directory corresponding to the smart network card in the system file system, and use the device local NUMA node attribute value as the NUMA node number connected to the smart network card; read the NUMA node ownership information corresponding to each dedicated physical core reserved by the kernel boot parameters in the system file system, and form a core-NUMA mapping pair with the physical core identifier of each dedicated physical core and its NUMA node number; traverse each core-NUMA mapping pair, and add the physical core identifier of the dedicated physical core corresponding to the core-NUMA mapping pair in which the NUMA node number is equal to the NUMA node number connected to the smart network card to the local NUMA domain dedicated physical core identifier list.

[0087] In this embodiment, the path for reading the device's local NUMA node attribute value is / sys / class / net / <NIC interface name> / device / numa_node. This file contains an integer value representing the NUMA node number to which the smart NIC is connected in the motherboard's physical topology. The path for reading the NUMA node affiliation information of each dedicated physical core is / sys / devices / system / cpu / cpu<core number> / node <n> / directory, where <core number> is the physical core identifier of the dedicated physical core, <n>This refers to the NUMA node number to which the core belongs. When there are multiple dedicated physical cores with the same NUMA node number as the NUMA node number connected to the smart network card, they are added to the local NUMA domain dedicated physical core identifier list in ascending order of physical core identifier. This list is generated once during the initialization phase and remains unchanged during operation.

[0088] The affinity binding execution unit is configured to: select the first physical core identifier from the local NUMA domain dedicated physical core identifier list as the binding target core identifier; construct a CPU affinity mask, setting the bit position corresponding to the binding target core identifier in the CPU affinity mask to 1 and setting the remaining bit positions to 0; call the thread affinity setting interface to set the CPU affinity mask of the packet receiving processing thread to the constructed CPU affinity mask, so that the packet receiving processing thread completes a one-to-one binding with the dedicated physical core corresponding to the binding target core identifier.

[0089] In this embodiment, after the affinity binding execution unit selects the target core identifier from the list of local NUMA domain dedicated physical core identifiers, it also records the selection timestamp and binding status flag of the core identifier, and moves the core identifier from the unassigned interval of the list to the assigned interval, so that other packet receiving processing threads will not repeatedly select the already bound dedicated physical core when performing NUMA-aware binding.

[0090] The interruption monitoring unit is configured to: after the packet receiving processing thread completes the Kth busy polling, read the current count value of the local interrupt count register of the dedicated physical core; subtract the current count value from the local interrupt history count value after the (K-1)th busy polling stored in the resource isolation module to obtain the interrupt increment value between the Kth and (K-1)th busy polling; compare the interrupt increment value with a preset interrupt increment threshold; if the interrupt increment value is greater than the interrupt increment threshold, it is determined that the dedicated physical core is interfered with by the non-smart network card hardware queue interrupt signal between the Kth and (K-1)th busy polling, and trigger the residual interrupt correction signal.

[0091] In this embodiment, the current count value of the local interrupt counter register is obtained by reading the sum of all interrupt counts in the corresponding column of the dedicated physical core in the / proc / interrupts file. This count value records the total number of all interrupt signals delivered to the dedicated physical core by the local Advanced Programmable Interrupt Controller (APIC) since system startup. User-mode programs cannot directly access the processor's internal APIC registers and must obtain interrupt statistics information through the / proc / interrupts file exported by the kernel. The interrupt increment threshold is set to 5, based on the following: under ideal interrupt-free conditions, the increment value of the local interrupt counter between two adjacent busy polls should be zero; considering that kernel timer interrupts may be misrouted in extreme cases, setting the threshold to 5 can tolerate a very small number of system management interrupts, while effectively detecting abnormal situations where a large number of hardware interrupts flood into the dedicated physical core due to configuration errors.

[0092] The residual interrupt correction unit is configured to: in response to the residual interrupt correction signal, traverse the interrupt service count increment value of each interrupt number in the system interrupt descriptor table, extract the interrupt numbers whose interrupt service count increment value is greater than zero and whose interrupt numbers do not belong to the set of interrupt numbers in the smart network card hardware queue, and use them as residual interference interrupt numbers; for each residual interference interrupt number, clear the bit corresponding to the dedicated physical core in its interrupt target core mask, and set at least one bit corresponding to the general physical core set, so that the interrupt signal of the residual interference interrupt number is no longer routed to the dedicated physical core.

[0093] In this embodiment, the system interrupt descriptor table is a data structure provided by the ` / proc / interrupts` file. Each line contains the interrupt name corresponding to an interrupt number, the cumulative interrupt count for each CPU core, and the affinity configuration information of the interrupt controller. The filtering conditions for residual interference interrupt numbers also include: the interrupt service count increment is greater than zero, the interrupt number does not belong to the smart network interface card hardware queue interrupt number set, and the interrupt name does not contain prefix matching items for the strings NMI, MCE, and LOC, to avoid mistakenly including non-maskable interrupts, machine check exception interrupts, and local timer interrupts in the correction scope, which could lead to system anomalies. Modification of the interrupt target core mask is achieved by writing to the ` / proc / irq / <interrupt number> / smp_affinity` file. The written value is a hexadecimal representation of a bitmap mask with the bits corresponding to the general physical core set set to 1. In this embodiment, NMI represents a non-maskable interrupt, MCE represents a machine check exception interrupt, and LOC represents a local timer interrupt. The filtering conditions are set based on the following: NMI is triggered by internal processor hardware errors or external emergencies and is used to report unrecoverable system-level fatal errors. Its routing target cannot be modified through operating system interrupt affinity settings. MCE is used to report hardware-level anomalies such as processor bus errors and memory check errors. Its processing logic is directly managed by the hardware and kernel. If the corresponding bit of the dedicated physical core in its interrupt target core mask is cleared, critical hardware error information will not be reported in a timely manner. LOC is generated by the processor's local APIC timer and is used to trigger the kernel scheduler's clock tick processing. If its interrupt route is removed from the dedicated physical core, it will disrupt the normal operation of the kernel time subsystem and cause the scheduler's time slice calculation to be incorrect.

[0094] The idle monitoring unit is configured to: obtain the first write pointer value obtained by the packet receiving processing thread from the write pointer register of the physical receiving ring buffer in the current polling cycle, and compare it with the second write pointer value obtained in the previous polling cycle; if the first write pointer value is equal to the second write pointer value, then increment the idle polling count value by 1; if the first write pointer value is not equal to the second write pointer value, then set the idle polling count value to zero.

[0095] In this embodiment, both the first write pointer value and the second write pointer value are 32-bit unsigned integer values, representing the starting buffer offset of the latest written packet data in the physical receive circular buffer. The empty polling count value is a 64-bit unsigned integer stored in the global state structure of the resource isolation module, with an initial value of zero, established when the packet receiving processing thread first performs busy polling. The polling period is defined as the complete time interval between the packet receiving processing thread completing one write pointer register read and corresponding packet processing judgment.

[0096] The sleep control unit is configured to: when the idle polling count value is greater than a preset idle threshold, multiply the difference between the idle polling count value and the idle threshold by a preset step duration unit value to calculate the sleep duration value; after the packet receiving processing thread completes the current polling cycle, pause the polling operation of the packet receiving processing thread on the write pointer register until the sleep duration value is reached; after the pause ends, set the idle polling count value to zero and resume the busy polling operation of the packet receiving processing thread on the write pointer register.

[0097] In this embodiment, the idle threshold is set to 1000. This is based on the following: In a high-frequency trading scenario with an average message arrival interval of 10 microseconds, 1000 consecutive idle polls correspond to a window of approximately 10 milliseconds without message arrivals. Exceeding this threshold indicates a low-frequency market or trading halt, making the power consumption control mechanism a reasonable trigger. The step duration unit is set to 10 microseconds. This is based on the following: The step duration is on the order of a single message processing cycle, allowing the sleep duration to gradually increase in 10-microsecond increments as the idle period progresses. After a 10-millisecond window without message arrivals, a light sleep of approximately 10 milliseconds is initiated. If the window persists, the sleep duration gradually increases to the tens of millisecond level.

[0098] The abnormal wrapback determination unit is configured to: obtain the current write pointer value obtained by the packet receiving processing thread from the write pointer register of the physical receive ring buffer, and compare it with the previous write pointer value obtained from the previous read; if the current write pointer value is less than the previous write pointer value, calculate the difference between the previous write pointer value and the current write pointer value; if the difference is greater than a preset wrapback tolerance threshold, determine that a write pointer abnormal wrapback event has occurred, and generate a wrapback abnormal trigger signal.

[0099] In this embodiment, the wraparound tolerance threshold is set to half the total capacity of the physical receive ring buffer, i.e., 32768 bytes. The basis for this setting is as follows: during normal transmit and receive operations, when the value of the write pointer register wraps around from near the end of the buffer to the beginning of the buffer, the difference between the previous write pointer value and the current write pointer value should be close to the total capacity of the physical receive ring buffer. If the difference is significantly less than the total capacity (i.e., less than half of the total capacity), it indicates that the write pointer has jumped back significantly before reaching the end of the buffer. This behavior is not a normal wraparound operation, but an unexpected jump caused by firmware abnormality, and an abnormal handling process should be triggered.

[0100] The buffer reset correction unit is configured to: in response to the wraparound exception trigger signal, discard all message payload data read from the memory area between the current write pointer value and the previous write pointer value; set the read pointer register value of the physical receive ring buffer to the starting offset of the physical receive ring buffer to zero; update the previous write pointer value to the current write pointer value, so that subsequent readings are based on the updated write pointer value.

[0101] In this embodiment, the operation of discarding packet payload data specifically involves: reclaiming the packet buffer data structure storing this portion of packet payload data into the preset buffer pool linked list, and overwriting its payload data field to zero to prevent abnormal data from polluting the upper-layer service processing logic. The starting offset of zero for the physical receive ring buffer corresponds to the first buffer bit pointed to by the base address mapped to the physical receive ring buffer in the user-space virtual address space. The operation of setting the read pointer register value to the starting offset of zero is implemented through the rte_eth_dev_rx_queue_reset interface of the DPDK driver, which simultaneously clears the status information of all pending descriptors in the receive queue.

[0102] As a specific example, in this embodiment, the initialization and operation process of the resource isolation module includes:

[0103] After the system powers on and completes buffer aggregation and page table mapping for the user-mode pass-through module, the thread extension unit obtains the number of hardware receive queues by reading the receive queue count register at offset 0x60 in the smart network card's PCIe configuration space. The receive queue count register has a 16-bit width, and its value is calculated and written by the smart network card firmware during the driver initialization phase based on the number of physical ports of the network card, the RSS receiver scaling configuration parameters of each port, and the on-chip SRAM buffer capacity. In this embodiment, the register value is 4, which is recorded as the total number of queues Q=4. The basis for the total number of queues being 4 in this embodiment is consistent with the hardware receive queue count N=4 obtained by the buffer aggregation unit, both originating from the four hardware receive queues of the same smart network card. The thread extension unit calls the POSIX standard pthread_create interface to create four packet receiving thread instances, which are respectively named packet receiving threads T1, T2, T3 and T4. The four thread identifiers are stored in the thread identifier array ThreadIDs[4] in the order of the hardware receiving queue index, so that ThreadIDs[0] corresponds to the first hardware receiving queue, ThreadIDs[1] corresponds to the second hardware receiving queue, and so on.

[0104] The queue binding unit parses the dedicated physical core identifier array IsoCores[4]={4,5,6,7} from the kernel startup parameters isocpus=4,5,6,7, and arranges them in ascending order. For the first packet receiving thread T1, a CPU affinity mask AffinityMask1 is constructed. This mask is a bitmap structure with a length of 1024 bits. Each bit corresponds to a physical core number. The 4th bit is set to 1 and the remaining 1023 bits are set to 0, indicating that only physical core 4 is included in the affinity binding range. The pthread_setaffinity_np(T1, sizeof(AffinityMask1), &AffinityMask1) is called to bind T1 to physical core 4. This interface updates the cpus_allowed mask field of thread T1 to the value of AffinityMask1 in kernel mode. In subsequent scheduling decisions, the kernel scheduler will only place T1 to run on physical core 4. The DPDK driver's `rte_eth_dev_rx_queue_info_get(port_id=0, queue_id=0, &qinfo)` interface retrieves the receive queue information structure of the first hardware receive queue. From this, the write pointer register address `qinfo.wptr_reg_addr` is extracted. This address is the physical address mapping value of the write pointer register in the PCIe BAR space of the smart NIC. This address is passed to T1, enabling T1 to perform independent busy polling of the write pointer register value of the first hardware receive queue using this address. Sequentially, the second receive processing thread T2 binds bit 5 to physical core 5 and corresponds to the second hardware receive queue via AffinityMask2; T3 binds bit 6 to physical core 6 and corresponds to the third hardware receive queue via AffinityMask3; and T4 binds bit 7 to physical core 7 and corresponds to the fourth hardware receive queue via AffinityMask4, completing the one-to-one binding of all receive processing threads. The basis for the one-to-one binding strategy between the hardware receive queue and the dedicated physical core in this embodiment is that each packet receiving processing thread independently polls the write pointer register of a hardware receive queue, avoiding contention for the same write pointer register by multiple packet receiving processing threads. At the same time, each packet receiving processing thread runs on a different dedicated physical core and does not share the last-level cache and memory bandwidth with each other, thus eliminating false sharing of cache lines and memory controller contention between packet receiving processing threads.

[0105] The NUMA topology sensing unit reads the file / sys / class / net / enp3s0f0 / device / numa_node and obtains that the NUMA node number connected to the smart network card is 0. This file contains an integer value written by the kernel PCI subsystem during the device enumeration phase according to the correspondence between the PCIe root complex and the NUMA node in the ACPISRAT table. In this embodiment, the value of 0 indicates that the smart network card is installed in the PCIe slot belonging to NUMA node 0. Traverse the dedicated physical core identifier array IsoCores[4]={4, 5, 6, 7} and check the existence of the directories / sys / devices / system / cpu / cpu4 / node0 / , / sys / devices / system / cpu / cpu5 / node0 / , / sys / devices / system / cpu / cpu6 / node1 / , and / sys / devices / system / cpu / cpu7 / node1 / respectively. / sys / devices / system / cpu / cpu<core number> / node <n>The directory is created by the kernel during the startup phase based on the topological relationship between the CPU physical cores and NUMA nodes. The existence of the directory indicates that the CPU core belongs to NUMA node N. The NUMA node numbers of cores 4 and 5 are 0, and the NUMA node numbers of cores 6 and 7 are 1. The core-NUMA mapping pairs {(4,0), (5,0), (6,1), (7,1)} are constructed, and the mapping pairs whose NUMA node numbers are equal to the NUMA node number 0 of the smart network card are filtered to obtain the local NUMA domain dedicated physical core identifier list LocalCores[2]={4,5}. In this embodiment, the core basis for cores 4 and 5 to belong to the local NUMA domain is that they are connected to the memory controller of the same NUMA node 0 with the smart network card. Any memory access request of the packet data written by the packet processing thread running on cores 4 or 5 to the smart network card DMA is completed through the local memory controller of NUMA node 0, without generating remote memory access latency across NUMA nodes. Remote memory access across NUMA nodes adds approximately 60 nanoseconds of latency compared to local memory access on the dual-socket server platform in this embodiment. This latency is eliminated through NUMA-aware binding.

[0106] The affinity binding execution unit selects the first physical core identifier 4 from the LocalCores list as the binding target core identifier for packet receiving thread T1, records the selection timestamp Ts_bind=1704067200000000 nanoseconds and the binding status flag BindStatus=OCCUPIED (value 0x55), and moves core 4 from the pending allocation interval to the allocated interval. For packet receiving thread T2, core 5 is selected from the updated pending allocation interval as the binding target core identifier, the binding is completed, and it is marked as allocated. Since the local NUMA domain only has two available cores, cores 4 and 5, while there are 4 packet receiving threads, for the remaining packet receiving threads T3 and T4, the affinity binding execution unit selects cores 6 and 7 from the general physical core set {6, 7} for binding, but records IsLocalNUMA=false in the binding flag to distinguish it from local NUMA binding. The allocation strategy in this embodiment, which prioritizes allocating local NUMA domain-dedicated physical cores to packet receiving threads, is based on the following: local NUMA cores are allocated sequentially according to the packet receiving thread numbers in ascending order, so that as many packet receiving threads as possible can obtain local memory access advantages; when the local NUMA cores are exhausted, the remaining packet receiving threads are bound to non-local NUMA cores, and the non-local NUMA binding status is recorded in the binding flag, providing a basis for decision-making for possible subsequent thread reallocation optimization.

[0107] After packet receiving thread T1 enters the steady-state busy polling phase, the interrupt interference monitoring unit continuously monitors the interrupt increment between every two adjacent busy polls. After T1 completes the 100th busy poll, the interrupt interference monitoring unit indirectly obtains the current count value of the local interrupt count register of the dedicated physical core 4 by reading the cumulative count value of the LOC line corresponding to the CPU4 column in the / proc / interrupts file. The LOC line in the / proc / interrupts file is the cumulative count of local timer interrupts, which records the total number of all local interrupt signals delivered to the dedicated physical core by the local APIC since system startup. IntrCnt_100 = 1523 is read, and the historical count value IntrCnt_99 = 1520 after the 99th busy poll is read from the global state structure of the resource isolation module. The interrupt increment value DeltaIntr = 1523 - 1520 = 3 is calculated. DeltaIntr=3 is compared with the preset interrupt increment threshold IntrThreshold=5. Since 3<5, it is determined that the dedicated physical core 4 was not interfered with by the non-smart NIC hardware queue interrupt signal between the 99th and 100th busy polls, and the residual interrupt correction signal is not triggered. When T1 completes the 200th busy poll, IntrCnt_200=1715, IntrCnt_199=1708, DeltaIntr=1715-1708=7, and 7>5. Therefore, it is determined that the dedicated physical core 4 was interfered with by the residual interrupt, and the residual interrupt correction signal is triggered. The basis for setting the interrupt increment threshold to 5 in this embodiment is as follows: Under ideal interruption-free conditions, the dedicated physical core is completely isolated and all smart NIC interrupts have been redirected to the general physical core set, and the local interrupt counter increment value between two adjacent busy polls should be zero; considering that kernel timer interrupts may be incorrectly routed to the dedicated physical core due to APIC configuration deviations in extreme cases, setting the threshold to 5 can tolerate a single or very small number (no more than 5) system management interrupts, while effectively detecting abnormal situations where a batch of hardware interrupts flood into the dedicated physical core due to interrupt affinity configuration errors, and avoiding unnecessary system overhead caused by accidental triggering of correction operations.

[0108] The residual interrupt correction unit responds to the residual interrupt correction signal by traversing the rows of interrupt numbers in the ` / proc / interrupts` file and extracting the interrupt service count increment value corresponding to each CPU core in each row. During the traversal, it is found that the interrupt service count increment value of interrupt number 125 (interrupt name nvme0q0, a non-volatile memory controller interrupt) in column 4 of CPU is 6>0, and interrupt number 125 does not belong to the smart NIC hardware queue interrupt number set {120, 121, 122, 123}, and the interrupt name nvme0q0 does not contain prefix matching items of the strings NMI, MCE, and LOC. Therefore, interrupt number 125 is determined to be a residual interference interrupt number. The current value of the ` / proc / irq / 125 / smp_affinity` file is read as ff (hexadecimal), corresponding to an 8-bit binary mask of 11111111, indicating that the target core of interrupt number 125 is all 8 physical cores. The corresponding bit (4th bit) of the dedicated physical core 4 is cleared, while the corresponding bits (6th and 7th bits) of the general physical core set {6, 7} are kept set, resulting in a corrected mask binary value of 11101011, which is hexadecimal eb. This value is written to the file / proc / irq / 125 / smp_affinity. The kernel interrupt subsystem will then route the interrupt signal of interrupt number 125 to cores 0, 1, 2, 3, 5, 6, and 7 only, and no longer to dedicated physical core 4, thus completing the correction of the residual interrupt.

[0109] During subsequent operation, the packet receiving thread T1 is in an idle period with no packets arriving. The idle monitoring unit reads the first write pointer value Wptr_500 = 0x8000 in the 500th polling cycle. This is a 32-bit unsigned integer representing the starting buffer offset of the latest written packet data in the physical receive ring buffer. It compares this with the second write pointer value Wptr_499 = 0x8000 read in the 499th polling cycle. Since Wptr_500 == Wptr_499, it is true, indicating that no new packet was acquired in the 500th poll, and the idle polling count is increased from 499 to 500. This idle polling count is a 64-bit unsigned integer stored in the global state structure of the resource isolation module, initially set to zero, and established when the packet receiving thread first executes busy polling. During the 501st to 1000th polling cycles, the write pointer value comparison results are always equal, and the empty polling count increases sequentially, reaching 1000 at the end of the 1000th polling cycle. In the 1001st polling cycle, Wptr_1001 = 0x8000 is read and compared with Wptr_1000 = 0x8000, which is still equal, increasing the empty polling count to 1001. The sleep control unit reads the empty polling count value IdleCnt = 1001, reads the preset idle threshold IdleThreshold = 1000, determines 1001 > 1000 as true, and calculates the sleep duration value SleepTime = (1001 - 1000) × 10 microseconds = 10 microseconds. In this embodiment, the idle threshold is set at 1000 because, in a high-frequency trading scenario with an average message arrival interval of 10 microseconds, 1000 consecutive idle polls correspond to a window of approximately 10 milliseconds (1000 times × 10 microseconds / time) without message arrivals. Exceeding this threshold indicates that the current market is in a low-frequency trading period or a trading halt, and triggering a power consumption control mechanism to reduce the risk of continuous high-temperature operation of the overclocked processor is reasonable. The step duration unit value is set at 10 microseconds because this value is consistent with the order of magnitude of a single message processing cycle, so that the sleep duration value gradually increases with the degree of idleness in 10-microsecond increments. After the 10-millisecond window of no message arrivals, it first enters a light sleep of approximately 10 microseconds. If the window continues, the sleep duration gradually increases to tens of microseconds or even milliseconds, achieving adaptive matching with the message arrival frequency and avoiding excessive delay in the first message response when sudden traffic arrives due to excessively long sleep times.

[0110] In extremely high throughput scenarios, if the smart network interface card firmware experiences an internal state management anomaly due to prolonged high-load operation, the anomaly wraparound determination unit, in the packet receiving processing thread T2, reads the write pointer register to obtain the current write pointer value Wptr_curr=0x1000, compares it with the previous write pointer value Wptr_prev=0x7F00 obtained and stored in the global state structure of the resource isolation module. If 0x1000 < 0x7F00, it is true, and the wraparound determination logic is entered. The difference Diff is calculated as: Diff = Wptr_prev - Wptr_curr=0x7F00 - 0x1000 = 0x6F00 = 28416 bytes. In this embodiment, the wraparound tolerance threshold is set to half of the total capacity of the physical receive ring buffer, which is 67,108,864 bytes (64MB), i.e., 33,554,432 bytes (32MB). The basis for this setting is as follows: During normal transmission and reception operations, when the value of the write pointer register wraps around from near the end of the buffer to the beginning of the buffer, the difference between the previous write pointer value and the current write pointer value should be close to the total capacity of the physical receive ring buffer, which is 67,108,864 bytes. If the difference is significantly less than half of the total capacity, i.e., 33,554,432 bytes, it indicates that the write pointer has bounced back significantly before reaching the end of the buffer. This behavior is not a normal hardware ring queue wraparound operation, but an unexpected jump caused by an abnormal state machine in the firmware. An exception handling process should be triggered to prevent the packet receiving thread from reading invalid memory areas based on an incorrect write pointer. In this embodiment, the wraparound tolerance threshold is set to half of the total capacity of the physical receive ring buffer, which is 67,108,864 bytes (64MB). This is based on the following: during normal transmission and reception operations, when the write pointer register wraps around from near the end of the buffer to the beginning, the difference between the previous write pointer value and the current write pointer value should be close to the total capacity of the physical receive ring buffer, 67,108,864 bytes. If the difference is significantly less than half the total capacity, i.e., 33,554,432 bytes, it indicates that the write pointer has bounced significantly before reaching the end of the buffer. This behavior is not a normal hardware ring queue wraparound operation, but rather an unexpected jump caused by an internal firmware state machine anomaly. An exception handling process should be triggered to prevent the receive processing thread from reading invalid memory areas based on an incorrect write pointer.

[0111] In response to the wraparound exception trigger signal, the buffer reset and correction unit marks all message buffer data structures corresponding to the message payload data read in the memory area between the current write pointer value 0x1000 and the previous write pointer value 0x7F00 as abnormal data. These message buffer data structures are then recycled to the available node queue of the pre-set buffer pool linked list. The `secure_zero_memory` function is called to perform a byte-by-byte zeroing write operation on the memory area from the start address of the message payload data field to the start address plus 2048 bytes, to prevent abnormal data from polluting the upper-layer business processing logic. The `rte_eth_dev_rx_queue_reset(port_id=0, queue_id=1)` interface of the DPDK driver is called to set the read pointer register value of the second hardware receive queue to the physical receive ring buffer start offset of zero, while simultaneously clearing the status information of all pending descriptors in this queue. The previous write pointer value Wptr_prev stored in the global state structure is updated to 0x1000, so that the receiving processing thread T2 uses the updated write pointer value as the basis for judging new packets during subsequent polling, thus completing the detection and reset correction of abnormal wraparound.

[0112] In this embodiment, the thread expansion unit starts a corresponding number of packet processing threads based on the number of hardware receive queues of the smart network interface card (NIC). The queue binding unit then binds each packet processing thread one-to-one with a dedicated physical core, allowing each thread to independently poll the write pointer register of a hardware receive queue. This avoids contention for the same write pointer register among multiple threads. Furthermore, each packet processing thread runs on a different dedicated physical core, without sharing the final-level cache and memory bandwidth, eliminating false cache line sharing and memory controller contention between threads. The NUMA topology awareness unit obtains the NUMA node numbers connected to the smart NIC and the NUMA node affiliation information of each dedicated physical core, and uses affinity... The binding execution unit prioritizes binding the packet receiving processing thread to the local physical core belonging to the same NUMA node as the smart NIC. This ensures that memory access requests for packet data written directly to the smart NIC by the packet receiving processing thread are handled by the local memory controller, eliminating remote memory access latency across NUMA nodes. The interrupt interference monitoring unit calculates the increment of the local interrupt counter between every two adjacent busy polls of the packet receiving processing thread and compares it with the interrupt increment threshold. This enables real-time detection of interrupt signal interference from the non-smart NIC hardware queue, preventing undetected anomalies caused by configuration errors leading to a surge of hardware interrupts into the dedicated physical core. Simultaneously, the interrupt increment threshold tolerates a very small number of system management interrupts, preventing accidental triggering of corrections. The operation incurs unnecessary system overhead. The residual interrupt correction unit, upon detecting residual interrupt interference, traverses the system interrupt descriptor table, filters out unredirectable interrupt types, and clears the corresponding bit in the dedicated physical core of the interrupt target core mask for the residual interference interrupt number, preventing the residual interrupt signal from being routed to the dedicated physical core. This achieves closed-loop maintenance of the interrupt isolation strategy. The idle monitoring unit compares the write pointer values ​​of adjacent polling cycles to determine the idle polling state and accumulates the idle polling count. When the idle polling count exceeds the idle threshold, the sleep control unit calculates a sleep duration proportional to the idle level and suspends the polling operation of the packet receiving thread. During idle periods without packet arrival, the processor's invalid polling is reduced in an adaptive step-by-step manner. The system monitors power consumption, avoiding the heat dissipation risk of overclocked processors under continuous high-frequency polling. Simultaneously, the sleep duration gradually increases with the degree of idle time, ensuring timely response to the first packet when sudden traffic recovery occurs. An abnormal wraparound determination unit calculates the difference when the current write pointer value is less than the previous write pointer value and compares it with the wraparound tolerance threshold. This distinguishes between normal circular queue wraparound operations and unexpected jumps caused by firmware anomalies, preventing the receiving thread from reading invalid memory areas based on erroneous write pointer values. The buffer reset and correction unit returns the affected packet buffer data to zero, reclaims it, resets the read pointer to the starting offset of zero, and clears the pending descriptor status information, restoring the receiving queue to a definite initial state and ensuring the reliability of subsequent packet access.

[0113] Example 4

[0114] Please see Figure 4 It should be further explained that the compilation optimization proxy module in this embodiment includes a pre-compilation startup unit, a compilation cache query unit, a vectorized dependency analysis unit, a replacement decision unit, a cache capacity statistics unit, a cache lock selection unit, an escape analysis unit, a lock elimination decision unit, a hotspot detection statistics unit, and a compilation trigger decision unit.

[0115] The pre-compilation unit is configured to: attach the compilation optimization agent to the Java Virtual Machine process after the Java Virtual Machine process completes class loading and before entering the main method execution; traverse the method table of the loaded classes and filter out the set of target methods marked with the vectorized optimization annotation; for each target method in the set of target methods, extract the starting memory address of its original bytecode instruction sequence, generate the starting memory address of the replacement vectorized single instruction multiple data stream instruction sequence, and store the starting memory address of the original bytecode instruction sequence as the key and the starting memory address of the replacement vectorized single instruction multiple data stream instruction sequence as the value in the pre-compilation cache mapping table.

[0116] In this embodiment, the timing of the additional compilation optimization proxy is chosen after the Java Virtual Machine process completes class loading and before entering the main method execution because: after class loading, all target methods marked with vectorized optimization annotations already exist in the method area, and performing pre-compilation scanning at this time can ensure the integrity of the target method set; and completing pre-compilation before entering the main method execution ensures that all vectorized replacement mapping relationships are constructed before the arrival of the first business message, so that when the just-in-time compiler compilation event is subsequently intercepted, only a hash table lookup operation needs to be performed, and the lookup complexity is constant. The pre-compilation cache mapping table is implemented using an open addressing hash table structure. The hash function takes the lower 16 bits of the binary representation of the starting memory address of the original bytecode instruction sequence as the hash slot index. If a hash collision occurs, a linear probing method is used to traverse backwards to find the first empty slot for insertion. This structure has an average of no more than 2 lookups when the number of key-value pairs does not exceed 70% of the hash table capacity.

[0117] The compilation cache query unit is configured to: when intercepting a code compilation event triggered by the just-in-time compiler, extract the starting memory address of the original bytecode instruction sequence of the method to be compiled in the code compilation event as a query key; search the pre-compiled cache mapping table for a cache entry with a key value equal to the query key; if it exists, return the value corresponding to the cache entry as the target instruction sequence, and skip the just-in-time compilation analysis stage of the method to be compiled by the just-in-time compiler.

[0118] In this embodiment, the specific operation of skipping the just-in-time (JIT) compilation analysis stage is as follows: The `HotSpotCompilationRequest.setResult()` method of the JVMCI interface is called to set the pre-compiled vectorized machine code as the compilation result. This causes the JIT compiler to no longer perform the processes of control flow graph construction, data flow analysis, intermediate representation generation, and machine code instruction selection for the method to be compiled. Instead, it directly returns the starting memory address of the replacement vectorized single instruction multiple data flow instruction sequence stored in the pre-compiled cache mapping table. This skipping operation reduces the compilation time from approximately hundreds of microseconds to approximately several microseconds of hash table lookup time, eliminating the CPU instruction cache refresh and compilation thread scheduling overhead caused by JIT compilation during the business processing stage.

[0119] The hotspot detection and statistics unit is configured to: perform runtime statistics on the target method marked with the vectorized optimization annotation during the interpretation and execution phase of the Java Virtual Machine process; record the number of times the target method is called, denoted as the method call count; and record the number of times the back edge instruction of the loop body within the target method is executed, denoted as the loop iteration count.

[0120] In this embodiment, the method call count is implemented by inserting call count increment stub code at the entry bytecode instruction of the target method. This stub code atomically increments the method call count by 1 each time the interpreter enters the target method. The loop iteration count is implemented by embedding back-edge count increment code in the loop body back-edge instruction processing logic of the target method. The interpreter atomically increments the loop iteration count by 1 each time it processes a back-edge instruction. Both counts are stored in the extended field of the corresponding target method in the pre-compiled cache mapping table, sharing the same hash table slot with the pre-compiled cache entry, without adding additional memory access overhead.

[0121] The compilation triggering decision unit is configured to: upon intercepting a code compilation event triggered by the just-in-time compiler, read the method call count and the loop iteration count; if the method call count is greater than a preset compilation heat threshold, or the loop iteration count is greater than a preset loop heat threshold, then determine that the target method meets the compilation triggering condition, and perform the instruction sequence replacement operation on the target method; if the method call count is less than or equal to the compilation heat threshold and the loop iteration count is less than or equal to the loop heat threshold, then determine that the target method does not meet the compilation triggering condition, and keep the original bytecode instruction sequence of the target method unchanged.

[0122] In this embodiment, the compilation heat threshold is set to 10,000 times, and the loop heat threshold is set to 1,000,000 times. The compilation heat threshold of 10,000 times is based on the following: according to actual test data from the trading gateway during the opening phase, if a target method is called more than 10,000 times within one minute, it means its average call frequency exceeds 167 times per second, classifying it as a high-frequency hot method. Vectorizing and replacing it can bring significant cumulative latency optimization benefits. The loop heat threshold of 1,000,000 times is based on the following: a single execution of a back-edge instruction takes approximately 3 to 5 nanoseconds, and 1,000,000 iterations correspond to a cumulative execution time of approximately 3 to 5 milliseconds. Exceeding this threshold indicates that the loop body is the core computationally intensive section within the target method, and vectorizing and replacing it can effectively compress the number of execution cycles of the loop body.

[0123] The vectorized dependency analysis unit is configured to: before executing the instruction sequence replacement operation, analyze the data access relationship between the Lth and L+1th iterations within the loop body of the target method; extract the target memory address of the write operation in the Lth iteration and extract the source memory address of the read operation in the L+1th iteration; if the target memory address is equal to the source memory address, it is determined that there is a cross-iteration write-read dependency; if the difference between the target memory addresses of any two memory access operations within the loop body is not equal to the byte length of a single data element, it is determined that there is a non-contiguous memory access pattern.

[0124] In this embodiment, the detection of cross-iteration write-after-read dependencies is based on the array subscript analysis results of the loop body during the intermediate representation stage of the just-in-time compiler. For each array access operation within the loop body, the vectorized dependency analysis unit extracts its array base address and subscript expression, substitutes the iteration number L into the subscript expression to calculate the memory address of the Lth iteration, and substitutes the iteration number L+1 into the subscript expression to calculate the memory address of the L+1th iteration. If an array element is written to in the Lth iteration and read from the same memory address in the L+1th iteration, then this dependency prevents vectorized replacement. The detection criterion for the non-contiguous memory access mode is: if the difference between the target memory addresses of two consecutive memory access operations is not equal to the byte width of the data type being operated on (e.g., 4 bytes for a 32-bit integer and 8 bytes for a 64-bit double-precision floating-point), it indicates that the memory access is a step-by-step access or random access, and the vectorized load instruction cannot read the data in a compact, packaged manner at once.

[0125] The replacement decision unit is configured to: if the vectorized dependency analysis unit determines that the loop body does not have cross-iteration write-after-read dependencies and does not have a non-contiguous memory access pattern, then perform the instruction sequence replacement operation; if it determines that there are cross-iteration write-after-read dependencies or a non-contiguous memory access pattern, then keep the original bytecode instruction sequence of the target method unchanged.

[0126] In this embodiment, when it is determined that the original bytecode instruction sequence of the target method should remain unchanged, the replacement decision unit also generates a dependency analysis failure record, which includes the target method name, failure type (cross-iteration write-after-read dependency or non-contiguous memory access mode) and the name of the array variable involved. The record is written to the compilation log buffer for developers to refer to for subsequent code structure optimization.

[0127] The cache capacity statistics unit is configured to: before performing the last-level cache locking operation, traverse all heap memory objects marked with the cache resident annotation, obtain the data length of each heap memory object, and sum the data lengths of each heap memory object to obtain the total cache resident data; read the last-level cache capacity value of the dedicated physical core, multiply the last-level cache capacity value by a preset ratio threshold to obtain the capacity upper limit value, and compare the total cache resident data with the capacity upper limit value.

[0128] In this embodiment, the final cache capacity of the dedicated physical core is obtained by reading the cache parameter register returned by the leaf function of the CPUID instruction. The final cache capacity returned varies for different processor models. In this embodiment, the final cache capacity of the AMD EPYC 9575F processor is 256MB. The preset ratio threshold is set to 75%, that is, the upper limit of capacity is 256MB × 0.75 = 192MB. The basis for setting this threshold is to reserve at least 25% of the final cache space for hot data that must reside in the cache, such as the operating system kernel, Java Virtual Machine runtime, just-in-time compiler code cache, and network protocol stack, so as to avoid other critical path data being frequently evicted to memory after the cached resident data occupies the entire final cache.

[0129] The cache locking selection unit is configured to: if the total amount of cached resident data is less than or equal to the capacity limit, perform the last-level cache locking operation on all heap memory objects marked with the cache resident annotation; if the total amount of cached resident data is greater than the capacity limit, obtain the access frequency value of each heap memory object within a preset statistical period, sort them in descending order of access frequency value, and attempt to select them sequentially starting from the first sorted heap memory object, calculate the sum of the data length of the currently selected object and the data length of the object to be selected, if the sum is less than or equal to the capacity limit, select the object to be selected and update the sum; if the sum is greater than the capacity limit, skip the object to be selected and end the selection, and perform the last-level cache locking operation on the selected heap memory objects.

[0130] In this embodiment, the preset statistical period is a 5-second time window from the start of the Java Virtual Machine process to the current moment. The length of this window is chosen because a 5-second window is sufficient to cover the hot data access patterns during active trading periods such as the opening and closing phases, while avoiding misjudgment of occasional accesses due to an excessively short statistical window. The access frequency value is calculated by dividing the total number of accesses to the heap memory object within the preset statistical period by the number of seconds in the preset statistical period. The last-level cache locking operation is implemented by calling the operating system's memory locking system call `mlock`. For objects marked with cache resident annotations, their data is stored in off-heap memory allocated through `sun.misc.Unsafe.allocateMemory`, and the off-heap memory address is stored in the object's `nativeAddress` field. When locking, the off-heap memory address and data length are passed in, and the physical page frame where the selected heap memory object is located is marked as non-swappable and locked in the last-level cache.

[0131] The escape analysis unit is configured to: before removing the bytecode synchronization instruction from the synchronization block marked with the lock removal annotation, obtain the object identifier of the shared object accessed within the synchronization block; scan the stack frame local variable table and operand stack of all active threads in the Java Virtual Machine process using the IterateOverThreads and GetLocalObjects methods of the JVMTI interface to check the number of threads holding the object identifier; check whether the shared object is used as the return value of the method it belongs to; check whether the shared object is passed to an external method that has not been inlined at compile time; if the number of threads is equal to 1, the shared object is not used as the return value of the method it belongs to, and the shared object is not passed to an external method that is not inlined, then it is determined that the shared object has not escaped the access scope of the current thread; otherwise, it is determined that the shared object has escaped the access scope of the current thread.

[0132] In this embodiment, the check of the number of threads holding object identifiers in the thread stack is implemented through the JVMTI interface. IterateOverThreads to traverse all active threads, and for each thread, GetLocalObjects is called to retrieve all reference type variables in its stack frame. If the object identifier only appears in the current thread's stack frame and not in other threads' stack frames, the thread count is determined to be 1. The criterion for determining if an external method is not inlined is as follows: during the just-in-time compiler's optimization phase, if the instruction calling the target method is compiled to directly jump to the machine code block of the called method, it is determined to be inlined; if the call instruction is compiled to a function call instruction, it is determined to be not inlined. Passing an object to a non-inlined external method means that the compiler cannot determine at compile time whether the external method will publish the object reference to other threads, therefore, it is conservatively considered to have an escape risk.

[0133] The lock elimination decision unit is configured to: if the escape analysis unit determines that the shared object has not escaped the access range of the current thread, then perform the removal operation of the bytecode synchronization instruction on the synchronization block; if the escape analysis unit determines that the shared object has escaped the access range of the current thread, then keep the bytecode synchronization instruction of the synchronization block unchanged.

[0134] In this embodiment, the removal of bytecode synchronization instructions is implemented by directly modifying the bytecode instruction sequence of the target method through the ASM bytecode framework during the pre-compilation stage. Specifically, in the `ClassFileTransformer.transform` method, the bytecode instructions of the target method are traversed, and the `monitorenter` and `monitorexit` instruction pairs are identified. If the synchronization block is marked with a lock removal annotation and verified through escape analysis, these two instructions and their corresponding operand stack adjustment instructions are deleted from the bytecode instruction sequence, preventing the generation of atomic locking and unlocking operations in the final executed machine code stream. This removal operation eliminates the execution overhead of lock record creation, lock contention detection, and atomic comparison and exchange instructions involved in the synchronization block, saving approximately 40 to 60 nanoseconds of CPU time per lock removal.

[0135] As a specific example, in this embodiment, the initialization and operation process of the compilation optimization proxy module is as follows:

[0136] After the system powers on and completes buffer aggregation and page table mapping for the user-mode passthrough module, and thread expansion and core binding for the resource isolation module, the Java Virtual Machine (JVM) process starts. When the JVM process has completed class loading of all startup classes and is about to enter the main method execution, the startup pre-compilation unit attaches a compilation optimization agent to the JVM process. The timing of attaching the compilation optimization agent in this embodiment is based on the following: after class loading, all target methods marked with vectorized optimization annotations already exist in the method area; performing pre-compilation scanning at this time ensures the integrity of the target method set. Furthermore, completing pre-compilation before entering the main method execution ensures that all vectorized replacement mapping relationships are constructed before the arrival of the first business message. Subsequent interception of the just-in-time (JIT) compiler compilation event only requires a hash table lookup operation, with a constant-order complexity, avoiding CPU instruction cache refresh and compilation thread scheduling overhead caused by JIT compilation during the business processing phase. The pre-compilation unit traverses the method table of loaded classes, filtering out target methods marked with vectorized optimization annotations. In this embodiment, this includes three target methods: OrderValidate.processOrder, MarketData.decodeQuote, and RiskCheck.checkPosition. For each target method, the starting memory address OrigAddr of its original bytecode instruction sequence is extracted. The vectorized code generator is then called to generate the starting memory address VecAddr of the corresponding replacement vectorized single instruction multiple data stream instruction sequence. OrigAddr is stored as the key and VecAddr as the value in the pre-compilation cache mapping table. In this embodiment, the pre-compilation cache mapping table is implemented using an open addressing hash table structure. The hash function uses the lower 16 bits of OrigAddr as the hash slot index, with an initial capacity of 1024 slots. If a hash collision occurs, a linear probing method is used to traverse backwards to find the first empty slot for insertion. This structure ensures that the average number of lookups does not exceed 2 when the number of key-value pairs does not exceed 70% of the hash table capacity.

[0137] After the Java Virtual Machine process enters the interpreted execution phase, the hotspot detection and statistics unit begins performing runtime statistics on target methods marked with vectorized optimization annotations. For the `OrderValidate.processOrder` method, each time the interpreter enters this method, the call count increment stub code at the entry bytecode atomically increments the method call count by 1; when the interpreter processes the back-edge instruction of the loop body within this method, the back-edge count increment code atomically increments the loop iteration count by 1. Both counts are stored in the extended field of the corresponding entry for `OrderValidate.processOrder` in the pre-compiled cache mapping table, sharing the same hash table slot with the pre-compiled cache entry, without adding additional memory access overhead. At the first minute of system operation, the method call count for `OrderValidate.processOrder` reaches 15,000, and the loop iteration count reaches 2,500,000.

[0138] The compilation cache lookup unit intercepts the code compilation event triggered by the just-in-time (JIT) compiler for the OrderValidate.processOrder method, extracts the starting memory address OrigAddr=0x7f0012000000 of the original bytecode instruction sequence of the method to be compiled as the lookup key, calculates the lower 16 bits of the hash slot index 0x7f0012000000 (0x0000) in the pre-compiled cache mapping table, linearly probes for cache entries whose key value equals the lookup key, and if a corresponding entry is hit, returns the value corresponding to that entry as the target instruction sequence. The processing result flag of the code compilation event is set to the pre-compiled hit state, preventing the JIT compiler from executing the control flow graph construction, data flow analysis, intermediate representation generation, and machine code instruction selection processes for the OrderValidate.processOrder method. In this embodiment, skipping the JIT compilation analysis stage reduces the compilation time from approximately 350 microseconds to approximately 8 microseconds of hash table lookup time, eliminating the CPU instruction cache refresh and compilation thread scheduling overhead caused by JIT compilation triggering during the business processing stage.

[0139] When intercepting a code compilation event, the compilation trigger decision unit reads the method invocation count value 15000 of OrderValidate.processOrder and the loop iteration count value 2500000. The method invocation count value 15000 is compared with the preset compilation popularity threshold 10000, and 15000>10000 is true, so it is determined that the target method meets the compilation trigger condition. The loop iteration count value 2500000 is compared with the preset loop popularity threshold 1000000, and <2500000>1000000 is also true, which further confirms that the compilation trigger condition is met. The basis for setting the compilation popularity threshold to 10000 times in this embodiment is that: according to the actual measured data of the transaction gateway during the market opening phase, if a target method is invoked more than 10000 times within 1 minute, its average invocation frequency per second exceeds 167 times, which belongs to a high-frequency hot method, and performing vectorization replacement on it can bring significant cumulative delay optimization benefits; the basis for setting the loop popularity threshold to 1000000 times is that: a single execution of the back-edge instruction takes about 3 to 5 nanoseconds, and 1000000 iterations correspond to a cumulative execution time of about 3 to 5 milliseconds. Exceeding this threshold indicates that the loop body is a core computation-intensive segment in the target method, and performing vectorization replacement on it can effectively reduce the number of execution cycles of the loop body, and under the transaction throughput of 100,000 transactions per second, the delay of each transaction can be reduced by about 30 to 50 nanoseconds.

[0140] Before executing the instruction sequence replacement operation, the vectorized dependency analysis unit analyzes the data access relationship between the Lth and L+1th iterations within the loop body of the OrderValidate.processOrder method. It extracts the target memory address AddrW=0x7f0030001000 for the write operation in the Lth iteration and the source memory address AddrR=0x7f0030001004 for the read operation in the L+1th iteration. Comparing 0x7f0030001000 and 0x7f0030001004, they are not equal, indicating no cross-iteration write-read dependency. It then checks the difference in target memory addresses between two consecutive memory access operations within the loop body. The first memory access target address is 0x7f0030001000, and the second is 0x7f0030001004. The difference is 4 bytes, equal to the byte width of a 32-bit integer, indicating no non-contiguous memory access pattern. In this embodiment, the detection of cross-iteration write-after-read dependencies is based on the array subscript analysis results of the loop body in the intermediate representation stage of the just-in-time compiler. For each array access operation, the array base address and subscript expression are extracted. The iteration counts L and L+1 are substituted into the subscript expression to calculate the memory address and then compared. The detection criterion for non-contiguous memory access mode is whether the difference between the target memory addresses of two consecutive memory access operations is equal to the byte width of the data type being operated on. If it is not equal, it indicates that the memory access is a step-by-step access or random access, and the vectorized load instruction cannot read the data in a compact package at once.

[0141] The replacement decision unit, based on the judgment result of the vectorized dependency analysis unit, confirms that the loop body does not have cross-iteration write-after-read dependencies and does not have non-contiguous memory access patterns, and performs an instruction sequence replacement operation. The original scalar instruction sequence of the OrderValidate.processOrder method loop body is replaced with the vectorized access and operation sequences _mm512_load_si512, _mm512_add_epi32, and _mm512_store_si512 from the AVX-512 instruction set. The replaced vectorized instructions process 16 32-bit integer elements at a time, while the original scalar instructions process only one element at a time. The execution cycle count for a single loop iteration is reduced from approximately 48 clock cycles to approximately 12 clock cycles.

[0142] The cache capacity statistics unit iterates through all heap memory objects marked with cache resident annotations after global garbage collection and before performing the last-level cache locking operation. In this embodiment, the objects marked with cache resident annotations include QuoteCache (data length 32MB), OrderBook (data length 64MB), and RiskRules (data length 48MB). The data lengths of the three objects are added together to obtain the total cache resident data: TotalCache = 32MB + 64MB + 48MB = 144MB. The L3 cache capacity parameter returned from the ECX register is read using the leaf function EAX = 0x80000006 of the CPUID instruction, and the last-level cache capacity of the AMD EPYC 9575F processor is found to be 256MB. Multiplying 256MB by a preset percentage threshold of 75% yields the capacity limit: CacheLimit = 256MB × 0.75 = 192MB. Comparing 144MB and 192MB, 144MB < 192MB, indicating that the total cache resident data is less than the capacity limit. The basis for setting the upper limit of the capacity of the last-level cache in this embodiment to 75% is that at least 25% of the last-level cache space should be reserved for hot data that must reside in the cache, such as the operating system kernel, Java Virtual Machine runtime, just-in-time compiler code cache and network protocol stack. This is to avoid the situation where the cached resident data occupies the entire last-level cache, causing other critical path data to be frequently evicted into memory, which would lead to the degradation of the overall system performance.

[0143] The cache locking selection unit, based on the comparison result from the cache capacity statistics unit, determines that TotalCache <= CacheLimit, and performs a last-level cache locking operation on all three heap memory objects marked with cache resident annotations. By calling the operating system's memory locking system call `mlock`, the physical page frames containing the market snapshot cache, order book cache, and risk control rule cache are marked as non-swappable and locked in the last-level cache. The `mlock` system call completes memory locking by setting the `PG_locked` flag in the process page table entry, marking the physical page frames corresponding to the specified virtual address range as non-swappable, ensuring they always reside in physical memory and are not swapped to disk. After locking with `mlock`, the physical page frames containing the data of the three objects (market snapshot cache, order book cache, and risk control rule cache) are fixed in physical memory, allowing the processor to hit the last-level cache in most cases, thereby reducing memory access latency from approximately 100 nanoseconds to approximately 12 nanoseconds for a last-level cache hit. The effectiveness of this locking operation is related to the processor cache replacement strategy. The last-level cache itself still dynamically manages its storage content according to the cache replacement algorithm. mlock does not provide a hard guarantee of last-level cache exclusivity or resident status for the locked page frame. Instead, it eliminates page faults caused by page frames being swapped to disk and subsequent data reloading, ensuring that the aforementioned hot data remains stably present in physical memory. This achieves a higher last-level cache retention priority at the cache replacement strategy level, resulting in an actual acceleration effect equivalent to permanent resident. In this embodiment, "permanent resident" in operating system memory management terminology specifically refers to marking the physical page frame corresponding to the specified virtual address range with the PG_locked flag through the mlock system call, so that the physical page frame is locked in physical memory and will not be swapped to disk by the operating system's page swapping mechanism under any circumstances.

[0144] Before processing synchronized blocks marked with lock removal annotations, the escape analysis unit obtains the object identifier ObjID=0x7f0050000000 of the shared object OrderCounter accessed within the synchronized block. It scans the local variable tables and operand stacks of all active threads in the Java Virtual Machine process, checking whether each stack frame stores a reference type variable with a value equal to 0x7f0050000000. The scan results show that ObjID=0x7f0050000000 only appears in the stack frame of the current packet receiving thread T1, and not in the stack frames of other threads T2, T3, T4, or Java Virtual Machine background threads, determining that the number of threads holding this object identifier is equal to 1. It then checks whether OrderCounter is used as the return value of its method. Analyzing the bytecode instruction sequence of the processOrder method, it searches for areturn instructions and their operands in all return paths of the method body. No areturn instructions with OrderCounter as an operand are found, indicating that it is not used as a return value. The algorithm checks whether `OrderCounter` is passed to an external method that is not inlined at compile time. It analyzes the compilation state of all calls using `OrderCounter` as a parameter after JIT compiler optimization. The `incrementCounter` method call instruction is compiled into a `jmp` instruction that directly jumps to the machine code block of the called method, thus it is determined to be inlined. The `logTransaction` method call instruction is compiled into a `call` function call instruction, and the machine code of the called method is not embedded in the current method's machine code stream, thus it is determined to be non-inlined. If the condition "the shared object is not passed to a non-inlined external method" is not met, the shared object is determined to have escaped the current thread's access scope. In this embodiment, the basis for determining the risk of escape when an object is passed to a non-inlined external method is that the compiler cannot determine at compile time whether the external method will publish the object reference to other threads. Therefore, it conservatively determines that there is a risk of escape to avoid data races and inconsistencies caused by error-eliminating locks and concurrent access by multiple threads.

[0145] Based on the escape analysis unit's determination, the lock elimination decision unit ensures that `OrderCounter` escapes the current thread's access scope, while maintaining the `monitorenter` and `monitorexit` bytecode synchronization instructions of the synchronized block unchanged. During the just-in-time (JIT) compiler's machine code generation process, `monitorenter` is correctly translated into a sequence of locking atomic operation instructions, and `monitorexit` into a sequence of unlocking atomic operation instructions. For another shared object `LocalBuffer` accessed within a synchronized block marked with a lock elimination annotation, the escape analysis unit determines that it is only referenced by the current thread, not returned as a return value, and not passed to any non-inlined external methods, thus determining that it has not escaped the current thread's access scope. The lock elimination decision unit performs a bytecode synchronization instruction removal operation on this synchronized block. During the translation of the bytecode instruction sequence into a machine code instruction sequence, it skips the generation of machine code corresponding to `monitorenter` and `monitorexit`, preventing the generation of locking and unlocking atomic operation instructions in the final executed machine code stream. In this embodiment, a single lock elimination can save approximately 50 nanoseconds of CPU time, which consists of three parts: approximately 15 nanoseconds for memory allocation and initialization of the lock record object, approximately 25 nanoseconds for atomic comparison-swapping contention detection of the lightweight lock, and approximately 10 nanoseconds for write barrier and memory synchronization during lock release.

[0146] This embodiment, by initiating a pre-compilation unit, attaches a compilation optimization proxy to the Java Virtual Machine process after class loading and before the main method execution. It then iterates through the method table of loaded classes, selecting the set of target methods marked with vectorized optimization annotations. A pre-generated sequence of vectorized single-instruction multiple-data-stream instructions for replacement is generated and stored in the pre-compilation cache mapping table. This ensures that all vectorized replacement mapping relationships are constructed before the arrival of the first business message. Subsequent interception of just-in-time (JIT) compiler compilation events only requires a hash table lookup operation, resulting in constant-time complexity. The compilation cache lookup unit, upon intercepting code compilation events, uses the starting memory address of the original bytecode instruction sequence of the method to be compiled as the lookup key. The compiler performs a lookup in the pre-compiled cache mapping table. Upon a match, it directly returns the corresponding vectorized single instruction multiple data flow instruction sequence and sets the code compilation event processing result flag to the pre-compiled hit state. This allows the just-in-time (JIT) compiler to skip the control flow graph construction, data flow analysis, intermediate representation generation, and machine code instruction selection processes, eliminating the processor instruction cache refresh and compiler thread scheduling overhead caused by JIT compilation during the business processing phase. During the interpretation and execution phase, the hotspot detection and statistics unit performs runtime statistics on the number of calls and loop iterations of the target method. The compilation triggering decision unit then compares these statistics with compilation heat thresholds and loop heat thresholds to determine if the compilation triggering conditions are met, ensuring that vectorized replacement only... It is applied to high-frequency hotspot methods and core computationally intensive segments, avoiding ineffective optimization of low-frequency methods. The vectorized dependency analysis unit analyzes the target memory address of write operations and the source memory address of read operations between the Lth and (L+1)th iterations of the loop body before executing instruction sequence replacement, detecting cross-iteration write-read dependencies. It also detects non-contiguous memory access patterns by comparing the difference in target memory addresses between two consecutive memory access operations with the byte width of the operated data type. The replacement decision unit executes the instruction sequence replacement operation only when it confirms that there are no cross-iteration write-read dependencies and no non-contiguous memory access patterns, ensuring the correctness and safety of vectorized replacement. The cache capacity statistics unit calculates all cache capacity before the last-level cache is locked. The total amount of data in heap memory objects marked with cache resident annotations is compared with the upper limit of capacity calculated based on the last-level cache capacity and a preset ratio threshold. The cache locking selection unit performs last-level cache locking on all objects when the total amount of data does not exceed the upper limit of capacity. When the upper limit of capacity is exceeded, objects that do not exceed the upper limit of capacity are selected for locking after being sorted by access frequency from largest to smallest. This reserves reasonable last-level cache space for hot data that must reside in the cache, such as the operating system kernel, Java virtual machine runtime, just-in-time compiler code cache, and network protocol stack. It avoids the overall system performance degradation caused by other critical path data being frequently evicted into memory after cache resident data occupies the entire last-level cache.Before removing bytecode synchronization instructions from synchronized blocks marked with lock removal annotations, the escape analysis unit checks the number of threads holding the shared object in the Java Virtual Machine process thread stack, whether it is used as the return value of its own method, and whether it is passed to an external method that is not inlined. The lock removal decision unit only removes the synchronization instructions if the shared object has not escaped the current thread's access scope. This avoids data races and inconsistencies caused by erroneous lock removal due to concurrent multi-threaded access. Furthermore, it eliminates the overhead of lock record creation, lock contention detection, and atomic comparison and swap instructions involved in synchronized blocks when safety is confirmed.

[0147] Example 5

[0148] Please see Figure 5 This embodiment provides a network acceleration method based on DPDK kernel bypass, including:

[0149] S1. Obtain the pass-through data path between the user-mode application process and the smart network card. The pass-through data path is established by mapping the physical receive ring buffer of the smart network card to the virtual address space of the user-mode application process through the DPDK driver. It is used to enable the user-mode application process to bypass the operating system kernel protocol stack and directly access the packet payload data in the packet buffer data structure.

[0150] S2. Obtain the dedicated physical core bound to the packet receiving processing thread in the user-mode application process. The dedicated physical core is reserved by the kernel startup parameters and local interrupts are disabled. The smart network card hardware queue interrupt is redirected to the general physical core set so that the packet receiving processing thread performs busy polling on the write pointer register value of the physical receive ring buffer in an uninterrupted state.

[0151] S3. When the busy polling determines that a new message has arrived, the message payload data is handed over to the Java Virtual Machine process running on the dedicated physical core for processing, and during the processing of the Java Virtual Machine process, the code compilation event triggered by the just-in-time compiler is intercepted.

[0152] S4. Based on the code compilation event, perform an instruction sequence replacement operation on the target method marked with vectorized optimization annotation in the Java Virtual Machine process, and replace the loop body operation instruction sequence of the target method with the vectorized single instruction multiple data stream instruction sequence supported by the dedicated physical core.

[0153] S5. Based on the heap memory objects marked with cache resident annotations in the Java Virtual Machine process, after the global garbage collection is completed, perform the last-level cache locking operation to keep the data of the heap memory objects resident in the last-level cache; at the same time, for the synchronization blocks marked with lock removal annotations, remove their corresponding bytecode synchronization instructions.

[0154] Example 6

[0155] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement a network acceleration method based on DPDK kernel bypass.

[0156] A computer-readable storage medium storing computer instructions that, when executed, perform a network acceleration method based on DPDK kernel bypass.

[0157] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of the present invention without departing from the spirit and scope of the claims. All of these variations are within the protection scope of the present invention.< / n> < / n> < / n>

Claims

1. A network acceleration system based on DPDK kernel bypass, applied to an edge computing node, wherein the edge computing node is deployed on a direct data channel between the physical receiving port of the smart network interface card and the service processing port of the user-mode application process, characterized in that, include: The user-mode passthrough module is configured to: obtain the passthrough data path between the user-mode application process and the smart network card. The passthrough data path is established by mapping the physical receive ring buffer of the smart network card to the virtual address space of the user-mode application process through the DPDK driver. It is used to enable the user-mode application process to bypass the operating system kernel protocol stack and directly access the packet payload data in the packet buffer data structure. The resource isolation module is configured to: acquire a dedicated physical core bound to the packet receiving processing thread in the user-mode application process, wherein the dedicated physical core is reserved by kernel startup parameters and local interrupts are disabled, and redirect the smart network card hardware queue interrupt to the general physical core set, so that the packet receiving processing thread performs busy polling on the write pointer register value of the physical receive ring buffer in an uninterrupted state.

2. The DPDK kernel-bypass based network acceleration system of claim 1, wherein, The network acceleration system also includes: The compilation optimization agent module is configured to: when the busy polling determines that a new message has arrived, hand over the message payload data to the Java Virtual Machine process running on the dedicated physical core for processing, and intercept the code compilation event triggered by the just-in-time compiler during the processing of the Java Virtual Machine process; Based on the code compilation event, for the target method marked with vectorized optimization annotation in the Java Virtual Machine process, an instruction sequence replacement operation is performed to replace the loop body operation instruction sequence of the target method with the vectorized single instruction multiple data stream instruction sequence supported by the dedicated physical core; Based on the heap memory objects marked with cache resident annotations in the Java Virtual Machine process, after global garbage collection is completed, a last-level cache locking operation is performed to retain the data of the heap memory objects in the last-level cache; at the same time, for synchronization blocks marked with lock removal annotations, their corresponding bytecode synchronization instructions are removed.

3. The DPDK kernel-bypass based network acceleration system of claim 2, wherein, The user-mode passthrough module includes a buffer aggregation unit and a page table mapping execution unit. The buffer aggregation unit is configured as follows: Obtain the number N of hardware receive queues reported by the smart network card; Traverse the N physical receive ring buffers corresponding to the N hardware receive queues, and extract the physical base address and buffer length of each physical receive ring buffer; In the virtual address space of the user-mode application process, a continuous virtual address range with a total length equal to the sum of the buffer lengths of the N physical receive ring buffers is calculated and reserved, and the starting virtual address of the continuous virtual address range is used as the aggregation base address.

4. The DPDK kernel-bypass based network acceleration system of claim 3, wherein, The page table mapping execution unit is configured to: use the aggregated base address as the starting address of the mapping target, use the list of physical base addresses of the N physical receive ring buffers as the set of mapping source addresses, and use the list of buffer lengths of the N physical receive ring buffers as the set of mapping size parameters, call the single-batch page table mapping interface provided by the DPDK driver to establish a complete page table mapping relationship between the continuous virtual address range and the N physical receive ring buffers in one go, so that the physical base address of the i-th physical receive ring buffer corresponds to the virtual address offset position obtained by adding the aggregated base address to the cumulative value of the buffer lengths of the previous (i-1) physical receive ring buffers, where i is an integer from 1 to N.

5. The DPDK kernel-bypass based network acceleration system of claim 4, wherein, The user-space pass-through module further includes a direct reference establishment unit and a zero-copy access encapsulation unit: the direct reference establishment unit is configured to: obtain the address of the memory pointer field used to store the message payload data in the message buffer data structure; update the pointer value stored at the memory pointer field address to the virtual address obtained by adding the mapping base address of the physical receive ring buffer in the user-space virtual address space to the offset of the current message data in the physical receive ring buffer, so that the message payload data pointer in the message buffer data structure directly points to the original storage location of the message data in the physical receive ring buffer.

6. The DPDK kernel-bypass based network acceleration system of claim 5, wherein, The zero-copy access encapsulation unit is configured to: provide a message data access interface to the upper-layer business processing logic; when the upper-layer business processing logic requests to read the message payload data through the message data access interface, return a copy of the message payload data pointer, so that the upper-layer business processing logic can directly perform a memory read operation on the original storage location of the message data in the physical receive ring buffer through the copy of the message payload data pointer, bypassing the heap memory allocator of the user-mode application process.

7. The DPDK kernel-bypass based network acceleration system of claim 6, wherein, The user-mode pass-through module further includes a status marking unit and a recycling synchronization unit: the status marking unit is configured to: after the application process completes the processing of the message payload data in the message buffer data structure, obtain the address of the status marking field in the message buffer data structure used to record the occupancy status of the buffer bits; update the status value stored at the status marking field address from a first status value indicating the occupancy status to a second status value indicating the release status, so that the buffer bits in the physical receive ring buffer corresponding to the message buffer data structure are marked as pending recycling.

8. The DPDK kernel-bypass based network acceleration system of claim 7, wherein, The recycling synchronization unit is configured to: traverse the message buffer data structure in which the status flag field value is the second status value, extract the offset of the buffer bit marked as to be recycled in the physical receive ring buffer, call the buffer release interface of the DPDK driver with the offset as a recycling parameter, so that the smart network card returns the corresponding buffer bit to the allocable space pool of the physical receive ring buffer according to the offset, and updates the free bit field of the descriptor table entry in the physical receive ring buffer to the allocable state.

9. The method of claim 1-8 based on DPDK kernel-bypass based network acceleration system implementation, characterized in that, include: Obtain the pass-through data path between the user-mode application process and the smart network card. The pass-through data path is established by mapping the physical receive ring buffer of the smart network card to the virtual address space of the user-mode application process through the DPDK driver. It is used to enable the user-mode application process to bypass the operating system kernel protocol stack and directly access the packet payload data in the packet buffer data structure. Obtain the dedicated physical core bound to the packet receiving processing thread in the user-mode application process. The dedicated physical core is reserved by the kernel startup parameters and local interrupts are disabled. The smart network card hardware queue interrupt is redirected to the general physical core set so that the packet receiving processing thread can perform busy polling on the write pointer register value of the physical receive ring buffer in an uninterrupted state.

10. The DPDK kernel-bypass based network acceleration method of claim 9, wherein, The network acceleration method further includes: When the busy polling determines that a new message has arrived, the message payload data is handed over to the Java Virtual Machine process running on the dedicated physical core for processing, and during the processing of the Java Virtual Machine process, the code compilation event triggered by the just-in-time compiler is intercepted; Based on the code compilation event, for the target method marked with vectorized optimization annotation in the Java Virtual Machine process, an instruction sequence replacement operation is performed to replace the loop body operation instruction sequence of the target method with the vectorized single instruction multiple data stream instruction sequence supported by the dedicated physical core; Based on the heap memory objects marked with cache resident annotations in the Java Virtual Machine process, after global garbage collection is completed, a last-level cache locking operation is performed to retain the data of the heap memory objects in the last-level cache; at the same time, for synchronization blocks marked with lock removal annotations, their corresponding bytecode synchronization instructions are removed.

Citation Information

Patent Citations

  • A low-latency, high-frequency trading system and method

    CN110460540B

  • Efficient parallel lock-free flow table state processing method and system based on DPDK

    CN117614901A

  • CPU resource dynamic control method for DPDK real-time traffic

    CN117978694A