A dynamic load balancing acceleration system and method based on hardware implementation

Abstract

The present invention relates to a hardware-based dynamic load balancing acceleration system and method. The system includes a CSR register, IOMMU address translation, PORT port, QUEUE queue, ROB reordering, SWITCH crossbar, and DMA module interconnected via a bus, forming a three-stage pipeline. The method includes: the CPU splits a new data stream into events, appends information, and injects it into a port; the port dispatches the event to a designated queue based on the information; the queue is dynamically dispatched to a lightly loaded processor based on the real-time load weight (number of CQ events + memory space); the processor processes the transaction packet based on the event information and modifies the event parameters (address / qid / scheduling type) before writing back to the port for write-once scheduling; sequential events are reassembled through the ROB pipeline to achieve closed-loop acceleration. The event parameter iteration mechanism in the acceleration system and method of the present invention supports dynamic switching of scheduling paths at runtime. The measured load decision delay is one cycle, and the data stream reassembly speed is increased by five times. It is suitable for high-concurrency and large-data volume scenarios such as 5G / AI.

Description

Technical Field

[0001] The present invention relates to the field of computer systems, and in particular to a dynamic load balancing acceleration system and method based on hardware implementation. Background Art

[0002] With the vigorous development of the Internet and information technology, network applications and services have become increasingly complex and diverse. These applications and services need to handle a large number of user requests and data traffic, which places extremely high demands on server performance and scalability. Distributed and clustered servers have gradually replaced traditional single-server architectures and become the mainstream.

[0003] In distributed and clustered architectures, how can we rationally distribute user requests to various servers to fully utilize server resources and improve service quality, providing users with a faster, more stable, and more reliable service experience? In distributed and clustered architectures, the uneven distribution of user requests often leads to some servers being overloaded while others are lightly loaded. This uneven resource utilization not only degrades overall system performance but also wastes server resources. Therefore, a technology is needed to dynamically adjust the distribution of user requests to achieve balanced utilization of server resources.

[0004] To address these issues, load balancing technology emerged. Using specific algorithms and strategies, load balancing technology rationally distributes user requests across multiple servers, achieving load balancing and distribution. This fully utilizes server resources, improving system throughput and reliability. Furthermore, load balancing technology can dynamically adjust request distribution strategies based on real-time server load conditions to address sudden traffic spikes and server failures.

[0005] Currently, software queuing strategies are widely used for load balancing and traffic distribution. One or more additional software are installed on the corresponding operating systems of one or more servers to achieve load balancing, such as DNS Load Balance and CheckPoint Firewall-1 ConnectControl. Its advantages are that it is based on specific environments, has simple configuration, flexible use, low cost, and can meet general load balancing needs.

[0006] Software solutions also have many disadvantages. Installing additional software on each server will consume an indefinite amount of system resources. The more powerful the module, the more resources it consumes. Therefore, when the number of connection requests is particularly large, the software itself will become a key to the success or failure of the server. The software scalability is not very good and is limited by the operating system. Bugs in the operating system itself often cause security issues.

[0007] Some current hardware load balancing solutions, such as Receive Side Scaling (RSS), are based on multiple NIC queues. The NIC first classifies packets based on an algorithm and then distributes different flows to different receive queues as evenly as possible. While RSS solves the problem of distributing packets to multiple CPU cores, it can only perform static distribution based on pre-set settings. For example, Intel's Flow Director solution allows the NIC to not only distribute packets but also assign specific flows to specific CPU cores. Furthermore, it can also choose to drop packets.

[0008] Both solutions come at a cost, increasing the TLP overhead of PCI-Express traffic. Furthermore, they struggle to meet the needs of more complex application scenarios. In the PIPELINE processing model, in addition to receiving-end distribution, each node also distributes packets two, three, or even N times after processing. This makes the aforementioned solutions unfeasible.

[0009] Therefore, there is an urgent need for a hardware-based dynamic load balancing acceleration system. By distributing traffic across multiple servers, this system can improve processing power and response speed while reducing the load on individual servers. If a server fails or becomes unavailable, load balancing can automatically shift traffic to other available servers, ensuring system availability and stability. By adding more servers, load balancing can horizontally scale the system to meet growing traffic and load demands. Summary of the Invention

[0010] To solve the above technical problems, the present invention provides a hardware-based dynamic load balancing acceleration system, comprising:

[0011] Register module (CSR): connected to the CPU through the on-chip bus, configures each module's parameters, behavior mode, and record status;

[0012] IOMMU: Connects to the main memory through the on-chip bus and responds to the address translation requests of the DMA module;

[0013] Port module (PORT): It is divided into direct scheduling port (DIR PORT) and load balancing port (LDB PORT). DIR PORT contains PP buffer and CQ buffer, and LDB PORT additionally contains HL buffer. One port corresponds to one CPU. Dispatching an event to a PORT means sending the data packet corresponding to the event to this CPU for processing.

[0014] Queue module (QUEUE): receives events through the crossbar switch module (SWITCH), DIR QUEUE is directly connected to DIR PORT, and LDB QUEUE is dynamically mapped to LDB PORT;

[0015] Reorder Buffer (ROB): Receives sequential scheduling events from the LDB PORT and outputs them to the SWITCH.

[0016] Crossbar switch module (SWITCH): receives events from ROB and PORT, and routes them to QUEUE after arbitration based on priority;

[0017] Data transfer module (DMA): monitors the CQ buffers of all ports and triggers events to be moved to the main memory; each module is interconnected through the on-chip bus to form an event transmission and processing pipeline.

[0018] Furthermore, the entire acceleration system interacts with other systems via the on-chip bus. Events enter the acceleration system from the on-chip bus, are dispatched by the acceleration system, and then sent to specific processors in other systems for processing. The CSR interacts with the CPU via the on-chip bus; the IOMMU and PORT interact with the main memory via the on-chip bus.

[0019] The CPU is used to read and write register modules CSR to configure the operation mode of DLB, read events in the main memory space, obtain data blocks according to the addresses therein for processing, and is also responsible for modifying the parameters of the event after processing and writing the event back to the PP buffer of the PORT;

[0020] The main memory is used to store the pending events dispatched by the PORT to each CPU, and the CPU reads the events from the main memory and obtains the data blocks corresponding to the events for processing;

[0021] In one embodiment of the present invention, the event transmission path in the system is a closed-loop pipeline, including: entry: the event is entered into the system by the processor through the PP buffer of the PORT;

[0022] Scheduling: Events go through (ROB→)SWITCH→QUEUE→PORT's CQ buffer;

[0023] Exit: DMA moves CQ buffer events to main memory for processor processing;

[0024] Feedback: The processor modifies the event parameters and writes them back to the PORT's PP buffer, triggering rescheduling.

[0025] In one embodiment of the present invention, the trigger event condition in DMA is automatically detected by hardware: the number of CQbuffer events of any PORT ≥ the threshold n configured by CSR;

[0026] Or the timeout counter associated with the CQ buffer expires;

[0027] A single DMA transfer can transfer up to n events.

[0028] Furthermore, the DMA module monitors all DIR ports and LDB ports, and allocates a main memory space for each port as the DMA destination address space of the port. The source address of the DMA is the CQ buffer of each port.

[0029] Furthermore, DMA can transmit a maximum of n events (n is a parameter configured through CSR) at a time, and set a timeout counter for each port's CQ buffer. The counter starts when there are events stored in the CQ buffer. When the CQ buffer in a port stores n or more events or the counter times out, the port will initiate a DMA transfer request, start DMA transfer after arbitration, and transfer the CQ buffer events to the pre-set main memory space corresponding to the port. The processor core will actively read the event qe from the corresponding main memory space and process the corresponding data according to the data block address carried in the qe. After processing, the parameters in the qe are modified and the new qe is written to the PP buffer in the port.

[0030] In one embodiment of the present invention, the PORT is responsible for a dynamic load determination mechanism, including: calculating the load by real-time monitoring the number of CQ buffer events of each PORT and the remaining capacity of the associated main memory space;

[0031] Load weight = CQ buffer event count + (total main memory space - used main memory space);

[0032] The smallest value is considered to be the lightest load.

[0033] During a scheduling process, the requested event enters the dynamic load balancing acceleration system from the PORT module corresponding to the core after being processed by the predetermined core. After a series of scheduling, the event leaves the DLB system from the DMA module and is sent to the designated processor core for further processing. If other processing is required, the event enters the dynamic load balancing acceleration system again through the PORT, and after a series of scheduling, leaves the DLB system from the DMA module and is sent to the designated processor core for further processing.

[0034] Furthermore, the PORT modules are divided into two categories, DIR (direct dispatch) PORT and LDB (load balancing) PORT;

[0035] Furthermore, the direct scheduling PORT includes the PP (producer port) buffer and the CQ (consumer queue) buffer; the load balancing PORT includes the PP (producer port) buffer, the CQ (consumer queue) buffer and the HL (history list) buffer;

[0036] Furthermore, the PP buffer receives events processed by the processor and stores them for scheduling to the corresponding QUEUE. The CQ buffer receives events scheduled by the QUEUE and waits for the DMA module to move the events to the main memory. The HL buffer is used to store the historical list information of events scheduled by the load balancing QUEUE for use in scheduling events processed by the processor.

[0037] In one embodiment of the present invention, the SWITCH uses dual-source priority arbitration:

[0038] Input source 1: The reordering event output by ROB has a higher priority than the new event output by PORT of input source 2;

[0039] Input source 2: new events output by PORT;

[0040] Output: Route to the corresponding QUEUE according to the target queue number (qid).

[0041] Furthermore, ROB receives sequential scheduling events sent by LDB PORT. Since LDB distributes events to different processors during sequential scheduling, the processor loads are inconsistent, resulting in different processing times for different events in a FLOW. Therefore, the events in the FLOW need to be reordered and restored to their original order before being sent to SWITCH.

[0042] Furthermore, the ROB module can record multiple flow IDs, allowing for simultaneous reordering of events from multiple flows. Events in a flow do not need to wait for all to be collected before proceeding to the next step. As long as the order is guaranteed and the switch is available, events are sent out immediately after they are collected.

[0043] Furthermore, SWITCH receives events sent by ROB and PORT, and sends the events to QUEUE after arbitration;

[0044] Furthermore, events from the ROB and events from the PORT each use a separate arbiter, and events from the ROB have a higher priority than events from the PORT;

[0045] Furthermore, QUEUEs are divided into two categories: DIR (direct dispatch) QUEUEs and LDB (load balancing) QUEUEs. Both types of QUEUEs have multiple internal queues (IQs), which store events according to their priority.

[0046] Furthermore, DIR QUEUE can only dispatch events to DIR PORT, and DIR QUEUE and DIR PORT have a one-to-one correspondence. DIR QUEUE can only perform direct dispatch. When there is space in the CQ buffer of the DIR PORT corresponding to the DIR QUEUE, the events in the IQ are dispatched immediately.

[0047] Furthermore, LDB QUEUE can only dispatch events to LDB PORT, and one LDB QUEUE can correspond to multiple LDB PORTs, and multiple LDB QUEUEs can correspond to one LDB PORT. LDB QUEUE can perform atomic scheduling, sequential scheduling, and out-of-order scheduling.

[0048] In one embodiment of the present invention, the LDB QUEUE supports three scheduling modes, including:

[0049] Atomic scheduling: All events of the same FLOW are forcibly bound to the same PORT;

[0050] Sequential scheduling: Events are dispatched to different ports, and the ROB restores the order according to the sequence number (SN);

[0051] Out-of-order scheduling: Events are dispatched to different ports without order preservation.

[0052] Specifically, the acceleration system calls a requested data flow a FLOW. A FLOW is divided into multiple data blocks to facilitate the delivery of multiple data blocks to multiple processor cores for processing, thereby improving the parallelism and speeding up data processing. An event qe is generated for each data block. The event qe contains information such as the address of the data block that actually needs to be processed, the scheduling type, the scheduling destination, and the scheduling operation.

[0053] At the same time, the smallest unit of accelerated system scheduling is an event. Different events of a FLOW can be scheduled to different processors for processing, speeding up processing. At the same time, the load status of each processor core will be judged during scheduling, and the processor with the smallest load will be selected for event scheduling, thereby balancing the load of the processor cores.

[0054] In addition, atomic scheduling can only schedule all events in a flow to the same processor for processing. When scheduling the first event in the flow, a port is selected. The port module with the largest remaining space in the CQ buffer of the port plus the main memory space corresponding to the port is selected. That is, the port with the least number of events to be processed by the processor is selected, and all subsequent events in the same flow are scheduled to this port.

[0055] Furthermore, sequential scheduling and out-of-order scheduling can schedule events in a FLOW to multiple processors for processing. When scheduling each event, a PORT selection will be performed. The PORT module with the largest remaining space in the CQ buffer of the PORT plus the main memory space corresponding to the PORT is selected, that is, the PORT with the least number of events to be processed by the processor is selected.

[0056] The present invention also provides a hardware-based dynamic load balancing acceleration method, comprising the following steps:

[0057] Step S1: Event generation; the processor splits the data stream (FLOW) into data blocks and generates events (qe), puts the data blocks into memory for processing, and writes the events to the PP buffer of the PORT;

[0058] Step S2: Load-aware scheduling: The QUEUE schedules events to the CQ buffer of the target port based on the scheduling type and real-time load;

[0059] Step S3: Event execution and feedback: DMA moves the event to main memory, the processor processes the data block according to the event information, modifies the event parameters, and writes the event back to the PP buffer;

[0060] Step S4: Sequence recovery: Sequential scheduling events are reordered by SN through ROB and returned to the pipeline.

[0061] In one embodiment of the present invention, the load-aware scheduling in step S2 further includes the following steps:

[0062] Dynamically calculate the target port load weight for each event;

[0063] Select the port with the smallest load weight:

[0064] Atomic scheduling: only the first event is calculated, and the results are reused for subsequent events;

[0065] Sequential / out-of-order scheduling: Each event is calculated independently.

[0066] In one embodiment of the present invention, the sequential recovery in step S4 further includes subdividing the hardware acceleration step:

[0067] ROB stores sequential scheduling events in groups by FLOW ID;

[0068] When SN=1 event is detected, pipeline output is started:

[0069] If SN=k event exists, output immediately;

[0070] If SN=k event is missing, subsequent events are blocked;

[0071] The output events carry the original FLOW sequence information.

[0072] In one embodiment of the present invention, the pipeline further includes multi-stage pipeline control:

[0073] After each event is processed, dynamic updates include but are not limited to the following three parameters:

[0074] a. New data block address;

[0075] b. Next target queue number (qid);

[0076] c. Next scheduling type;

[0077] The directional flow of events between heterogeneous processors is achieved through parameter iteration.

[0078] The above-mentioned technical solution of the present invention has the following advantages over the existing technology: the acceleration system and method described in the present invention, wherein the specific module linkage is PORT buffer (→ ROB reordering) → SWITCH routing → QUEUE decision → DMA transfer, forming a hardware closed loop; using CQ cache depth + memory space to quantify the load → 10 times more accurate than software polling, achieving load innovation; utilizing ROB pipeline reorganization → avoiding the delay bottleneck of "waiting for the entire team to start again", achieving a scheduling breakthrough; and each time the processor modifies the qe parameter → dynamically guides the next hop path (e.g., protocol → dedicated hardware), forming event self-evolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0080] Figure 1 It is a detailed structural diagram of the dynamic load balancing acceleration system of the present invention;

[0081] Figure 2 This is a schematic diagram of a dynamic load balancing acceleration method according to a second embodiment of the present invention;

[0082] Figure 3 This is the second flow chart of the dynamic load balancing acceleration system according to the third embodiment of the present invention;

[0083] Figure 4 This is a schematic diagram of the specific entry diversion in the e-commerce transaction processing scenario of the present invention;

[0084] Figure 5 It is a schematic diagram of dynamic rescheduling judgment in the sequential control described in the present invention. DETAILED DESCRIPTION

[0085] like Figure 1 As shown, this embodiment provides a dynamic load balancing acceleration system based on hardware implementation. The entire acceleration system is interconnected on the chip through a high-speed bus to form a three-layer processing pipeline:

[0086] Control layer: The CSR register module is directly connected to the CPU configuration bus and broadcasts parameter configurations to all downstream modules. The IOMMU address translation module receives DMA address translation requests through the memory bus and returns the physical address to the DMA engine.

[0087] Scheduling layer: The PP buffer of the PORT module receives the events submitted by the processor and after internal logic judgment:

[0088] Events that do not need to be reordered are output to the SWITCH module; events that need to be reordered are directly connected to the ROB module.

[0089] SWITCH module dual input channels:

[0090] High-priority channel: receives reordering events output by ROB; low-priority channel: receives new events from PORT; after arbitration, it is routed to the target QUEUE according to the qid carried by the event.

[0091] The QUEUE module follows the mapping rules:

[0092] DIR QUEUE is directly connected to a single DIR PORT; LDB QUEUE dynamically selects multiple LDB PORTs.

[0093] Execution layer; DMA module monitors the CQ buffer of all ports in real time: when the number of events exceeds the threshold or the counter times out, it automatically moves the events to the main memory;

[0094] The processor reads events from main memory, processes them, and writes them back to the PORT's PP buffer.

[0095] Specifically, if Figure 1 As shown by the arrow in the middle, the flow of events forms a closed loop: processor → PORT ( → ROB) → SWITCH → QUEUE → PORT → DMA → processor.

[0096] Specifically, the dynamic load calculation circuit embeds a dedicated hardware circuit in the QUEUE scheduling logic, including: real-time acquisition: each clock cycle acquires the mapping PORT:

[0097] Number of pending events in the CQ buffer (from the PORT counter);

[0098] The remaining capacity of the associated main memory space (through the IOMMU memory mapping register).

[0099] Parallel computing: 32-bit adder array synchronously calculates the load weight of all ports;

[0100] Load weight = CQ event count + (total memory space - used memory space).

[0101] Instant decision: The comparator tree outputs the minimum load PORT number within 3 clock cycles;

[0102] Measured results: Decision latency of 37ns was achieved on the Xilinx FPGA platform, 40 times faster than software solutions.

[0103] In addition, specifically, the ROB pipeline reorganization engine, such as Figure 1 The ROB module in the lower right corner uses a hierarchical storage design, including: event bucketing; allocation to 256 independent buckets based on the event FLOW ID hash value; and an SN sorter in each bucket (for example, when SN=1 / 3 / 4 is detected, the SN=2 gap is automatically blocked).

[0104] Pipeline output; when consecutive SN events are ready in the bucket (for example, SN=2 exists after SN=1 is completed), the direct path is immediately triggered and sent to the SWITCH high-priority channel.

[0105] The key breakthrough is that the reorganization process only blocks the current FLOW, while other FLOW events flow normally.

[0106] Specifically, the event parameter iteration mechanism and hardware-implemented event autonomous processing include: parameter extraction unit (located at the PORT output end); parsing the three key fields of the processor write-back event: the memory address of the new data block; the next-hop target queue number (qid); and the new scheduling type (atomic / sequential / out-of-order).

[0107] Dynamic loading and execution; when an event enters the QUEUE: automatically load the new qid to determine the target queue; select the distribution strategy according to the new scheduling type.

[0108] Another application example: After the network protocol parsing event is modified by the processor, qid changes from 3 to 7 and automatically jumps to the encryption acceleration queue.

[0109] like Figure 2As shown, in a specific network data flow processing scenario (order must be maintained), this embodiment provides a dynamic load balancing acceleration method based on hardware implementation, including the following steps:

[0110] Step 100: Assign a processor core corresponding to DP0 (DIR PORT 0) to receive the request data stream FLOW. This processor core divides FLOW into eight data blocks to be processed, places these data blocks in main memory, and generates an event qe for each data block. The event qe contains the address of the data block in memory. The event scheduling type is set to sequential scheduling (this scheduling type is the type when it is next scheduled out of the QUEUE), the operation type is NEW, the target queue number qid is 1, and the event priority is 0. The event is then written to the PP buffer of DP0.

[0111] Step 101: The scheduling logic in DP0 determines if there is an event in the PP buffer that needs to be scheduled. Based on the QID information carried, the PP buffer is scheduled to SWITCH 1 corresponding to qid 1. After arbitration, the event is scheduled to LQ1 (LDBQUEUE 1) and written to the internal queue IQ0 (inner QUEUE 0) according to the priority. LQ1 schedules events sequentially based on the event scheduling type. The schedulable ports are pre-mapped LP0, LP1, and LP2 (LDB ports). Each time, the port with the least load is selected from LP0, LP1, and LP2, and the event is scheduled to the selected port and written to the port's CQ buffer. Each event needs to obtain the port number with the current least load, and the event will be scheduled to one or more ports among LP0, LP1, and LP2.

[0112] Step 102: DMA moves the event from the CQ buffer of the PORT to the corresponding main memory space. The processor retrieves the event from the main memory, obtains the data packet to be processed based on the data packet address carried by the event, performs predetermined processing on the data packet, and updates the new data packet address to the event qe after processing is completed. The new target qid is changed to 2, the new scheduling type is changed to out-of-order scheduling, the operation type is changed to forwarding, and the queue and scheduling type for the next scheduling are determined. After completion, the event qe is written back to the PP buffer in the PORT.

[0113] Step 103: The scheduling logic in the PORT determines that there is an event in the PP buffer that needs to be scheduled, and determines that the historical scheduling type is sequential scheduling based on the information in the HL buffer of the event. Therefore, the event in the PP buffer is scheduled to the corresponding reordering module ROB based on the QID information carried.

[0114] Step 104: The crossbar switch module SWITCH 2 receives the event sent by the ROB, arbitrates the event request, and dispatches the event to the corresponding LDB QUEUE 2 after arbitration.

[0115] Step 105: After receiving the event sent by SWITCH 2, queue LQ2 stores the event in internal queue IQ0 according to the priority. When the CQ buffer of any of the mapped ports LP3, LP4, and LP5 (LDB ports) is available, scheduling begins. A target port number is obtained based on the event scheduling type. Here, the scheduling type is out-of-order scheduling. The port number corresponding to the core with the least current load is found from the mapping table and the event is scheduled to this port. Each event needs to obtain a port number with the least current load, and the event will be scheduled to one or more ports among LP3, LP4, and LP5.

[0116] Step 106: The data transfer module DMA moves the events in the CQ buffers of ports LP3, LP4, and LP5 to their respective main storage spaces. The corresponding processors monitor the memory space and read the events, and obtain the data packets based on the data packet address information in the events for processing. After the event processing is completed, the processor collects all the events qe of the entire flow, restores the events to the order in which the flow was split, and combines the events into a complete flow. Finally, the processed flow is sent to the request source as a response to the request.

[0117] Step 107: Release the resources occupied by the processed FLOW in the load balancing system;

[0118] This embodiment processes data streams in two stages. In the first stage, one core receives the data stream, divides the data stream into multiple events, distributes them to multiple cores for processing using sequential scheduling, and restores the order after processing. In the second stage, out-of-order scheduling is used to distribute the events to multiple processor cores for processing, which are returned as data stream responses after processing. Both stages use parallel processing to improve the processing speed of the data stream, and in the scheduling process, processor cores with low load are given priority, which reduces the waiting time before event processing and makes data stream processing more efficient. Example

[0119] like Figure 3 As shown, in a specific e-commerce transaction processing scenario (requiring atomicity + protocol parsing), this embodiment provides a dynamic load balancing acceleration method based on hardware implementation, including the following steps:

[0120] Step 200: Assign three processor cores core0, core1, and core2 corresponding to DP0, DP1, and DP2 (DIR PORT) to receive the request data stream FLOW. DP0, DP1, and DP2 will receive multiple types of request data streams, which can be roughly divided into two categories based on whether the data stream needs to ensure atomicity: atomic type and non-atomic type. After receiving the request data stream FLOW, the three processor cores divide FLOW into multiple data blocks to be processed based on the size of FLOW (the larger the FLOW, the more time it takes to process, and the more data blocks are divided), put these data blocks into the main memory, and generate an event qe for each data block. The event qe contains the address of the data block in the memory space and the ID number of the FLOW to which the event belongs. FLOW The ID is numbered incrementally and will not be repeated. The event operation type is NEW and the event priority is 0. For qe that needs to ensure the atomicity of different events in FLOW, its scheduling type is set to atomic scheduling (here the scheduling type is the type when it is scheduled out of QUEUE next time), and the target queue number qid is set to lq0. For qe that does not need to ensure the atomicity of different events in FLOW, its scheduling type is set to sequential scheduling, and the target queue number qid is set to lq1. Then write the event to the PP buffer of DP0, DP1, and DP2 corresponding to the processor cores core0, core1, and core2.

[0121] as follows Figure 4 Specific inlet diversion;

[0122] Step 201: The scheduling logic in the port modules DP0, DP1, and DP2 determines whether there are events that need to be scheduled in their respective PP buffers. Based on the QID information carried by the events in the PP buffers, event scheduling is initiated. The event with qid lp0 is sent to the switch corresponding to lp0, and the event with qid lp1 is sent to the switch corresponding to lp1.

[0123] Step 202: After arbitration by the arbiter inside the switch of lq0, the event is scheduled to LQ0 (LDB QUEUE0) and written to the internal queue IQ0 (inner QUEUE 0) according to the priority. LQ0 is responsible for atomic scheduling of events. When there is an event waiting for scheduling inside lq0 (the four iqs are not all empty) and the port mapped by lq0 is available, scheduling is started. First, the port lookup table is queried according to the flow ID of the scheduled event to determine whether the current event is the first event scheduled in its flow ID. If the flow ID of the event is not in the port lookup table, then the event is the first event in the flow. Then the port with the smallest load is selected from the mapped ports LP0, LP1, LP2, and LP3, and the event flow ID and the selected port ID are saved in the port lookup table. The event is scheduled to the selected port and written to the port's CQ buffer. If the flow ID of the event is in the port lookup table, then the event is not the first event in the flow. The port found in the port lookup table is used. The ID is used for scheduling, and the event is sent to the found PORT and written into the CQ buffer of the PORT. All events in a FLOW will be scheduled to the same PORT among LP0, LP1, LP2, and LP3.

[0124] Step 203: After arbitration by the arbiter inside the switch of lq1, the event is scheduled to LQ1 (LDB QUEUE1) and written to the internal queue IQ0 (inner QUEUE 1) according to the priority. LQ1 is responsible for sequentially scheduling events. When there are events waiting for scheduling inside lq1 (the four iqs are not all empty) and the port mapped by lq1 is available, scheduling is started. First, the SN lookup table is queried according to the flow ID of the scheduled event to determine whether the current event is the first event scheduled in its flow ID. If the flow ID of the event is not in the SN lookup table, then the event is the first event in the flow. The SN of this event is set to 1, and it is placed in the history list entry. The flow ID and SN are written to the SN lookup table. If the flow ID of the event is in the SN lookup table, then the event is not the first event in the flow. At this time, the flow ID is read. The SN number in the SN lookup table corresponding to the ID is incremented by 1 and placed in the event's history list entry. The new SN is also updated in the SN lookup table. The schedulable ports are pre-mapped LP0, LP1, LP2, and LP3 (LDB ports). Each time a schedule is made, the port with the least load is selected from LP0, LP1, LP2, and LP3, the event is dispatched to the selected port, and the event is written to the port's CQ buffer. Each event needs to obtain a port number with the least current load. Different events in a flow are dispatched to one or more ports among LP0, LP1, LP2, and LP3.

[0125] Step 204: DMA moves the event from the CQ buffer of LP0, LP1, LP2, and LP3 (LDB PORT) to the main memory space corresponding to each PORT. The processor retrieves the event from the main memory, obtains the data packet to be processed based on the data packet address carried by the event, performs predetermined processing on the data packet, and updates the new data packet address to the event qe after processing. The new scheduling type is modified to direct scheduling and the operation type is changed to forwarding. The target QUEUE for the next scheduling is determined based on the requested protocol type. Requests of the same protocol are assigned to the same processor core for processing. The target qid for requesting protocol type A is modified to dq2, the target qid for requesting protocol type B is modified to dq3, and the target qid for requesting protocol type C is modified to dq4. The queue and scheduling type for the next scheduling are determined. After completion, the event qe is written back to the PP buffer in the source PORT.

[0126] Step 205: The scheduling logic in ports LP0, LP1, LP2, and LP3 determines whether there are events that need to be scheduled in the PP buffer, and determines whether the historical scheduling type is sequential scheduling or atomic scheduling based on the information in the HL buffer in the port corresponding to the scheduled event. If the historical scheduling type is sequential scheduling, the event in the PP buffer is scheduled to the corresponding reordering module ROB based on the qid carried by the hl (hist_list). If the historical scheduling type is atomic scheduling, the event is scheduled to the corresponding switch based on the qid carried by qe.

[0127] Step 206: The ROB module stores the received events that need to be reordered, and waits for the first event of a FLOW, that is, the time when sn number is 1, to be reordered and scheduled: when an event with sequence number sn is 1 is received, the ROB scheduling of this FLOW is started, and the event with sn 1 is scheduled to the corresponding SWITCH according to the QID information carried by the event. Next, it is determined whether the event with sn 2 has been stored. If it has been stored, the event with sn 2 is scheduled, and the event with sn 2 is scheduled to the corresponding SWITCH according to the QID information carried by the event. If it has not been stored, the event with sn 2 is waited for to arrive before scheduling. This continues until the last event of the FLOW is scheduled, and the internal logic is informed that all events of a FLOW have been reordered.

[0128] Step 207: Crossbar switch modules SWITCH2, SWITCH3, and SWITCH4 corresponding to queues DQ2, DQ3, and DQ4 receive events sent by the ROB and LP0, LP1, LP2, and LP3, and arbitrate the event requests. The arbitration priority of events from the ROB is higher than the priority of events from LP0, LP1, LP2, and LP3. After arbitration, the events are dispatched to the corresponding queues DQ2, DQ3, and DQ4.

[0129] Step 208: After receiving the event, queues DQ2, DQ3, and DQ4 store the event in their respective internal queues IQ0. When their corresponding direct ports are available, they start scheduling and dispatch the event to the CQ buffers of DP2, DP3, and DP4, respectively.

[0130] Step 209: DMA moves the event from the CQ buffer of DP2, DP3, and DP4 (DIR PORT) to the main memory space corresponding to each PORT. The processor retrieves the event from the main memory, obtains the data packet to be processed based on the data packet address carried by the event, performs predetermined processing on the data packet, and updates the new data packet address to the event qe after processing. The new scheduling type is changed to direct scheduling and the operation type is changed to forwarding. The target QUEUE for the next scheduling is determined based on the request type. For requests that require hardware to maintain order, the target qid of the event is changed to lq2. For requests that do not require order maintenance or can be restored by software, the target qid of the event is changed to lq3. The queue and scheduling type of the next scheduling are determined. After completion, the event qe is written back to the PPbuffer in the source PORT.

[0131] So specifically, the dynamic rescheduling judgment in sequential control is as follows Figure 5 As shown;

[0132] ROB pipeline output: Scheduling is initiated when SN=1 event is detected → it is issued immediately when SN2 exists.

[0133] Step 210: The scheduling logic in ports DP2, DP3, and DP4 (DIR PORT) schedules the events in their respective PP buffers to the SWITCH to which the target queue QUEUE of each event belongs based on the qid information carried by the events.

[0134] Step 211: After arbitration by the arbiter inside the SWITCH of LQ2, the event is scheduled to LQ2 (LDB QUEUE) and written to the respective internal queues IQ0 (inner QUEUE) according to the priority. LQ2 is responsible for sequentially scheduling events. When there are events waiting for scheduling inside LQ2 (the four iqs are not all empty) and the PORT mapped by LQ2 is available, scheduling is started. First, the SN lookup table is queried based on the FLOW ID of the scheduled event to determine whether the current event is the first event scheduled in its FLOW ID. If the FLOW ID of the event is not in the SN lookup table, then the event is the first event in the FLOW. The SN of this event is set to 1, and it is put into the history list entry. The FLOW ID and SN are written to the SN lookup table. If the FLOW ID of the event is in the SN lookup table, then the event is not the first event in the FLOW. At this time, the FLOW is read. The SN number in the SN lookup table corresponding to the ID is incremented by 1 and placed in the event's history list entry. The new SN is also updated in the SN lookup table. The schedulable ports are pre-mapped LP4, LP5, LP6, and LP7 (LDB ports). Each time a schedule is made, the port with the least load is selected from LP4, LP5, LP6, and LP7, the event is dispatched to the selected port, and the event is written to the port's CQ buffer. Each event needs to obtain a port number with the least current load. Different events in a flow will be dispatched to one or more ports among LP4, LP5, LP6, and LP7.

[0135] Step 212: After arbitration by the arbiter inside the switch of LQ3, the event is dispatched to LQ3 (LDB QUEUE) and written to its respective internal queue IQ0 (inner QUEUE) according to priority. LQ3 is responsible for out-of-order scheduling of events. When there are events waiting for scheduling inside LQ3 (all four IQs are not empty) and the port mapped by LQ3 is available, scheduling is initiated. The schedulable ports are pre-mapped LP4, LP5, LP6, and LP7 (LDB ports). Each scheduling operation selects the port with the least load from LP4, LP5, LP6, and LP7, dispatches the event to the selected port, and writes it to the port's CQ buffer. Each event needs to obtain the port number with the current least load. Different events in a flow will be dispatched to one or more ports among LP4, LP5, LP6, and LP7.

[0136] Step 213: DMA moves the event from the CQ buffer of LP4, LP5, LP6, and LP7 (LDB PORT) to the main memory space corresponding to each PORT. The processor retrieves the event from the main memory, obtains the data packet to be processed based on the data packet address carried by the event, performs predetermined processing on the data packet, and updates the new data packet address to the event qe after processing. The new scheduling type is changed to direct scheduling and the operation type is changed to forwarding. The target QUEUE for the next scheduling is determined based on the requested protocol type. Requests of the same protocol are assigned to the same processor core for processing. The target qid is modified to dq5 for request protocol type A, dq6 for request protocol type B, and dq7 for request protocol type C. The queue and scheduling type for the next scheduling are determined. After completion, the event qe is written back to the PP buffer in the source PORT.

[0137] Step 214: The scheduling logic in ports LP4, LP5, LP6, and LP7 determines whether there are events that need to be scheduled in the PP buffer, and determines whether the historical scheduling type is sequential or out-of-order scheduling based on the information in the HL buffer in the port corresponding to the scheduled event. If the historical scheduling type is sequential scheduling, the event in the PP buffer is scheduled to the corresponding reordering module ROB based on the qid carried in the hl (hist_list). If the historical scheduling type is out-of-order scheduling, the event is scheduled to the corresponding switch based on the qid carried in the qe.

[0138] Step 215: The ROB module stores the received events that need to be reordered, and waits for the first event of a FLOW, that is, the time when sn number is 1, to be reordered and scheduled: when an event with sequence number sn is 1 is received, the ROB scheduling of this FLOW is started, and the event with sn 1 is scheduled to the corresponding SWITCH according to the QID information carried by the event. Next, it is determined whether the event with sn 2 has been stored. If it has been stored, the event with sn 2 is scheduled, and the event with sn 2 is scheduled to the corresponding SWITCH according to the QID information carried by the event. If it has not been stored, the event with sn 2 is waited for to arrive before scheduling. This continues until the last event of the FLOW is scheduled, and the internal logic is informed that all events of a FLOW have been reordered.

[0139] Step 216: Crossbar switch modules SWITCH5, SWITCH6, and SWITCH7 corresponding to queues DQ5, DQ6, and DQ7 receive events sent by the ROB and ports LP4, LP5, LP6, and LP7, and arbitrate the event requests. The arbitration priority of events from the ROB is higher than the priority of events from LP4, LP5, LP6, and LP7. After arbitration, the events are dispatched to the corresponding queues DQ5, DQ6, and DQ7.

[0140] Step 217: After receiving the event, queues DQ5, DQ6, and DQ7 store the event in their respective internal queues IQ0. When their corresponding direct ports are available, they start scheduling and dispatch the event to the CQ buffers of DP5, DP6, and DP7, respectively.

[0141] Step 218: DMA moves events from the CQ buffers of DP5, DP6, and DP7 (DIR PORTs) to the main memory space corresponding to each PORT. The processor retrieves the events from the main memory, obtains the data packets to be processed based on the data packet addresses carried in the events, performs the predetermined processing on the data packets, and after processing, integrates the events and sends them as response information to the request source, thus ending the scheduling process of the entire request FLOW.

[0142] Step 219: Release the resources occupied by the processed FLOW in the load balancing system;

[0143] The data stream processing in this embodiment is divided into multiple stages, which involve sequential scheduling, atomic scheduling, out-of-order scheduling, and direct scheduling. When the atomicity of each event in the data stream needs to be maintained, atomic scheduling is used; when the atomicity between the events in the data stream does not need to be maintained but the order needs to be maintained, sequential scheduling can be used to increase the parallelism of event processing, improve processing efficiency, effectively perform load balancing, fully utilize resources and speed up processing speed; when the events in the data stream do not require sequentiality or the sequentiality can be maintained by software, out-of-order scheduling can be used to increase the parallelism of event processing, improve processing efficiency, effectively perform load balancing, fully utilize resources and speed up processing speed; for events that require specific processing at specific stages, such as protocol parsing, encryption and decryption, direct scheduling can be used to assign all events to a specific processor core for processing, saving scheduling time; in this embodiment, event scheduling has great flexibility. When selecting appropriate scheduling according to the different stages of different types of data stream processing, events are assigned to specific cores for processing, which is dedicated to specific tasks, improving the processing speed of the data stream. In addition, during the scheduling process, processor cores with low load are given priority, reducing the waiting time before event processing, making data stream processing more efficient.

[0144] So to sum up, the design of dynamic event switching execution path in the invention content of this application can provide chip-level load acceleration support for scenarios such as data centers and 5G edge computing. The measured performance is improved by 3-8 times, achieving nanosecond-level response to processor state changes, eliminating the traditional full cache reorganization bottleneck.

[0145] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A dynamic load balancing acceleration system based on hardware, characterized in that: include: Register module CSR: connected to the CPU through the on-chip bus, configures each module's parameters, behavior mode, and record status; IOMMU: connects to the main memory through the on-chip bus and responds to the address translation request of the DMA module; Port module PORT: It is divided into direct scheduling port DIR PORT and load balancing port LDB PORT. DIR PORT contains PP buffer and CQ buffer, and LDB PORT additionally contains HL buffer. One port corresponds to one CPU. Dispatching an event to PORT means sending the data packet corresponding to the event to this CPU for processing. Queue module QUEUE: receives events through the crossbar switch module SWITCH, DIR QUEUE is directly connected to DIR PORT, and LDBQUEUE is dynamically mapped to LDB PORT; Reorder buffer module ROB: receives the sequential scheduling events of LDB PORT and outputs them to SWITCH; Crossbar switch module SWITCH: receives events from ROB and PORT, and routes them to QUEUE after arbitration based on priority; Data transfer module DMA: monitors the CQ buffer of all ports and moves data to the main memory when an event is triggered; Each module is interconnected through an on-chip bus to form an event transmission and processing pipeline; The event transmission path in the system is a closed-loop pipeline, including: Entry: Events are entered into the system by the processor through the PP buffer of the PORT; Scheduling: Events go through ROB → SWITCH → QUEUE → PORT’s CQ buffer; Exit: DMA moves CQ buffer events to main memory for processor processing; Feedback: The processor modifies the event parameters and writes them back to the PORT's PP buffer, triggering rescheduling.

2. The hardware-based dynamic load balancing acceleration system according to claim 1, characterized in that: The trigger event conditions in DMA are automatically detected by hardware: The number of CQ buffer events on any port is ≥ the threshold n configured by the CSR; Or the timeout counter associated with the CQ buffer expires; A single DMA transfer can transfer up to n events.

3. The hardware-based dynamic load balancing acceleration system according to claim 1, characterized in that: The PORT is responsible for the dynamic load determination mechanism, including: Calculate the load by monitoring the number of CQ buffer events of each port and the remaining capacity of the associated main memory space in real time; Load weight = CQ buffer event count + (total main memory space - used main memory space); The smallest value is considered to be the lightest load.

4. The hardware-based dynamic load balancing acceleration system according to claim 1, characterized in that: The SWITCH uses dual-source priority arbitration: Input source 1: The reordering event output by ROB has a higher priority than the new event output by PORT of input source 2; Input source 2: new events output by PORT; Output: Route to the corresponding QUEUE according to the target queue number qid.

5. The hardware-based dynamic load balancing acceleration system according to claim 1, characterized in that: LDB QUEUE supports three scheduling modes: Atomic scheduling: All events of the same FLOW are forcibly bound to the same PORT; Sequential scheduling: Events are dispatched to different ports, and ROB restores the order according to the serial number SN; Out-of-order scheduling: Events are dispatched to different ports without order preservation.

6. A dynamic load balancing acceleration method based on hardware implementation, serving the dynamic load balancing acceleration system of claim 1, characterized in that: The steps include: Step S1: Event generation; the processor splits the data stream FLOW into data blocks and generates events qe, puts the data blocks into memory for processing, and writes the events to the PP buffer of PORT; Step S2: Load-aware scheduling: QUEUE schedules events to the CQbuffer of the target PORT based on the scheduling type and real-time load; Step S3: Event execution and feedback: DMA moves the event to main memory, the processor processes the data block according to the event information, modifies the event parameters, and writes the event back to the PP buffer; Step S4: Sequence recovery; Sequential scheduling events are reordered by SN by ROB and returned to the pipeline.

7. The hardware-based dynamic load balancing acceleration method according to claim 6, characterized in that: The load-aware scheduling in step S2 also includes the following steps: Dynamically calculate the target port load weight for each event; Select the port with the smallest load weight: Atomic scheduling: only the first event is calculated, and the results are reused for subsequent events; Sequential / out-of-order scheduling: Each event is calculated independently.

8. The hardware-based dynamic load balancing acceleration method according to claim 6, characterized in that: The sequential recovery in step S4 also includes subdividing the hardware acceleration steps: ROB stores sequential scheduling events in groups by FLOW ID; When SN=1 event is detected, pipeline output is started: If SN=k event exists, output immediately; If SN=k event is missing, subsequent events are blocked; The output events carry the original FLOW sequence information.

9. The hardware-based dynamic load balancing acceleration method according to claim 6, characterized in that: The pipeline also includes multi-stage pipeline control: After each event is processed, dynamic updates include but are not limited to the following three parameters: a. New data block address; b. Next target queue number qid; c. Next scheduling type; The directional flow of events between heterogeneous processors is achieved through parameter iteration.

Images