Stateful programmable data plane structure based on speculative execution, and chip

Abstract

The present application relates to the technical field of programmable data planes, and in particular relates to a stateful programmable data plane structure based on speculative execution, and a chip. The data plane structure comprises: multiple stage processors, the multiple stage processors forming a data processing pipeline, and each stage processor processing one pipeline stage; and a side ring, the side ring comprising multiple ring nodes, and each ring node being connected to each stage processor in one-to-one correspondence. A read processor reads the state required for a stateful data packet. A write processor writes the state required for a stateful data packet, identifies a state-modified data packet requiring re‑submission, uploads the data packet for re-submission to a corresponding ring node, and sends the data packet via the side ring to a ring node corresponding to the read processor, and the read processor then downloads the data packet requiring re‑submission from the corresponding ring node. Thus, the present invention solves problems in the prior art such as the inability of pipeline technology to support relatively complex stateful operations, and the high development costs.

Description

Stateful programmable data plane structure and chip based on speculative execution

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 202410360790.8 and application date of March 27, 2024, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to the technical field of programmable data planes, and in particular to a stateful programmable data plane structure and chip based on speculative execution. Background Art

[0004] Programmable switches consist of a programmable data plane and a control plane. The data plane is responsible for protocol parsing and de-parsing, as well as packet processing, while the control plane is responsible for issuing parsing rules and flow table matching rules. High-performance programmable switching chips have further unlocked the potential of data plane devices, from customized protocol forwarding to supporting in-network computing applications. The demand for high throughput has made hardware pipelining a primary choice for switch chip architectures. While pipelining offers unparalleled throughput, it assumes a forward processing flow, hindering effective support for stateful functions, which are critical to many valuable applications.

[0005] The stateful data plane functionality can be summarized as an EFSM (extended finite state machine). The packet state is read from the flow state table; an action is performed based on the current state and input. This action may result in a state update, which is then written back to the flow state table. The inconsistency between writing back to the flow state table and the forward processing flow of the pipeline makes existing pipeline technology unable to support more complex stateful operations.

[0006] Summary of the Invention

[0007] The present application provides a stateful programmable data plane structure and chip based on speculative execution to solve the problems that the pipeline technology in the related art cannot support more complex stateful operations and has high development costs.

[0008] The first aspect of the present application provides a stateful programmable data plane structure based on predictive execution, including: multiple stage processors, wherein the multiple stage processors form a data processing pipeline, and each stage processor processes one pipeline stage; a side ring, wherein the side ring includes multiple ring nodes, and the multiple ring nodes are connected to each stage processor in a one-to-one correspondence; any stage processor among the multiple stage processors is configured as a read processor or a write processor, the read processor reads the state required to be read for at least one stateful data packet, and predicts that the state of the stateful data packet remains unchanged; the write processor writes the state required to be written for the stateful data packet, identifies the data packet that needs to be resubmitted with the modified state, uploads the data packet that needs to be resubmitted to the corresponding ring node of the write processor, and sends it to the corresponding ring node of the read processor through the side ring, and the read processor downloads the data packet that needs to be resubmitted from the corresponding ring node.

[0009] Optionally, the stage processor includes a configurable scheduling module, wherein the configurable scheduling module of the read processor is configured as a read scheduling module, the configurable scheduling module of the write processor is configured as a write scheduling module, and the configurable scheduling modules of other stage processors are not activated.

[0010] Optionally, both the read scheduling module and the write scheduling module include: a dirty table stored in a content-addressable memory, wherein the keyword used to match in the dirty table is a state table index of the flow, wherein the dirty table of the write scheduling module contains a table of keywords, and the dirty table of the read scheduling module contains a table of keywords and corresponding data; the write scheduling module writes the flow corresponding to the data packet with modified state into the dirty table, and sends a write-back control signal to the read scheduling module through the side ring, and the read scheduling module writes the flow corresponding to the data packet with modified state into the dirty table according to the write-back control signal.

[0011] Optionally, the read processor also includes: a resubmission buffer for storing resubmitted data packets; the read scheduling module also includes: a data packet buffer for storing data packets that hit the dirty table, and the data packets of each flow are linked in a first linked list and a second linked list, the first linked list is used to store newly arrived data packets on the pipeline, and the second linked list is used to store data packets that need to be resubmitted; an order-preserving check module for scheduling data packets in the data packet buffer, and when the data packets in the data packet buffer are scheduled, whether the data packets that need to be resubmitted of the flow where the scheduled data packets are located have all arrived in the resubmission buffer and have all been moved to the second linked list of the data packet buffer.

[0012] Optionally, the order-preserving checking module is further configured to: start a timer until the timer times out, and determine that all data packets that need to be resubmitted have arrived at the resubmission buffer.

[0013] Optionally, a heartbeat address field is set on the bus of the side ring. The heartbeat address field is independent of other fields on the bus, and each pipeline stage occupies one bit of the heartbeat address field, and the signal carrying the heartbeat is written into the heartbeat address field; when the timer receives the signal carrying the heartbeat sent from the corresponding write processor, the timing is advanced by one moment.

[0014] Optionally, the sequence checking module is also used to: write the serial number of each data packet into the counter table when each data packet reads the status; if the serial number of the data packet that needs to be resubmitted is equal to the serial number in the counter table, it is determined that all the data packets that need to be resubmitted have arrived at the resubmission buffer.

[0015] Optionally, the read processor is also used to write a flush signal to the corresponding ring node when the data packet of a flow in the data packet buffer starts to be scheduled, send it to the ring node corresponding to the write processor through the side ring, and download it to the write processor; the write processor removes the corresponding flow from the dirty table according to the flush signal.

[0016] Optionally, the read scheduling module includes first to third levels of state consistency, wherein the first level of state consistency includes: the read operation of the data packet is after the write operation of all data packets before the data packet; the second level of state consistency includes: on the stream where the target data packet is located, the read operation of K data packets after the target data packet is allowed to be before the write operation of K data packets, and starting from the K+1th data packet, the read operation of the data packet is after the write operation of all data packets before the data packet; the third level of state consistency includes: the order of the read operation and the write operation of the data packet is arbitrary.

[0017] A second embodiment of the present application provides a programmable switching chip, including: a stateful programmable data plane structure based on speculative execution according to any one of the above embodiments.

[0018] Therefore, this application has the following beneficial effects:

[0019] The embodiment of the present application adds a simple side ring to the pipeline, so that each pipeline stage is connected to it. On the side ring, the data flow is opposite to the direction of the pipeline, providing a fast backward communication path. The read processor reads the state required to read the data packet with state, and the write processor can write back the state across stages when writing back the data packet that needs to modify the state. The whole process is very convenient and more efficient, and supports complex stateful functions, thereby reducing the development cost, deployment cost and maintenance cost of in-network computing applications as a whole. As a result, the problems of pipeline technology in related technologies that cannot support more complex stateful operations and have high development costs are solved.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 shows the technical architecture of Banzai in related art;

[0023] FIG2 is a circuit diagram of an atomic operation of Banzai in the related art;

[0024] FIG3 is a technical architecture diagram of a distributed reconfigurable matching table in related art;

[0025] FIG4 is a block diagram of a stateful programmable data plane structure based on speculative execution according to an embodiment of the present application;

[0026] FIG5 is a structural diagram of a scheduling module according to an embodiment of the present application;

[0027] FIG6 is a schematic diagram of a finite state machine under bounded stale consistency provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0029] The following describes the stateful programmable data plane structure and chip based on predictive execution of the embodiment of the present application with reference to the accompanying drawings. In response to the problems mentioned in the above background technology, the present application provides a stateful programmable data plane structure based on predictive execution, which adds a simple side ring to the pipeline so that each pipeline stage is connected to it. On the side ring, the data flow is opposite to the direction of the pipeline, providing a fast backward communication path. When the read processor reads the state required to read the state of the data packet with state, and when the write processor writes back the data packet that needs to modify the state, the state can be written back across stages. The whole process is very convenient and more efficient, and supports complex stateful functions, thereby reducing the development cost, deployment cost and maintenance cost of in-network computing applications as a whole. As a result, the problems that the pipeline technology in the related art cannot support more complex stateful operations and has high development costs are solved.

[0030] Before introducing the specific content of this application, the relevant technical advances involved in this application are described in detail, including the following:

[0031] 1. Related Technology 1

[0032] As shown in Figure 1, Banzai is a technical prototype of Intel's programmable switch, Tofino. Users program Tofino's protocols and functions using a protocol-agnostic language and operate the on-chip flow tables and state tables using Intel-provided runtime software. As shown in Figure 1, Banzai uses a programmable parser to parse the packet header fields upon incoming data packets. These two identical pipelines constitute Banzai's programmable packet processing logic. Each pipeline consists of a series of stages, known as the Match-Action Table (MAT), which is the prototype's basic programmable unit. Users program the MAT using a protocol-agnostic language to determine which flow table the packet should query at each stage and the processing logic to execute after the query results are obtained.

[0033] In Banzai, the processing logic of each stage is implemented using a set of parallel atomic operations. The circuitry used for these atomic operations, shown in Figure 2, performs a "read-add / subtract-writeback" function. The target of an atomic operation can be a field in a packet header or a register in a state table. However, due to the simplicity of atomic operations, only simple stateful functions (such as counters) can be implemented using atomic operations. For more complex functions, the logic for determining the next state may involve a series of operations and table accesses, which cannot be implemented using atomic operations. To avoid pipeline bottlenecks, this technical solution cannot simply expand the atomic operation circuitry or enhance the atomic operation functionality to support these stateful functions. Due to the unidirectional nature of the pipeline, there is no effective mechanism for writing state data back to earlier stages. The only solution is to re-loop (re-send the packet to the pipeline head for further processing). However, this approach may disrupt the order of packets in the same flow within the pipeline, resulting in unavoidable read-after-write (WAR) conflicts and causing state consistency issues (i.e., reading outdated state).

[0034] 2. Related Technology 2

[0035] The distributed reconfigurable matching table (dRMT) utilizes multi-core, multi-threaded, run-to-completion (RTC) processors to support stateful functionality. dRMT decouples memory and processors on the switch chip, allowing processors to access memory resources in the memory cluster through the switch network, as shown in Figure 3. If dRMT uses shared memory to store the state table, access scheduling and locking are required to address state consistency issues, resulting in unpredictable wait times on some processors. This undermines the deterministic latency inherent in its integer linear programming-based compilation algorithm, making such programs uncompilable. If each processor maintains its own copy of the state table, complex synchronization mechanisms are required and consumes significant amounts of additional memory.

[0036] 3. Related Technology 3

[0037] FlowBlaze uses a queuing and scheduling mechanism to ensure state consistency. At each FlowBlaze stage, a packet queries a separate flow context table to determine if its flow is currently being processed. Once a packet enters stateful processing (matching and performing operations on the ESFM table), FlowBlaze blocks subsequent packets from the same flow to ensure state consistency. This approach assumes that all packets entering a stateful function may change state, which can cause pipeline stalls and create performance bottlenecks. In reality, stateful functions may only apply to a subset of flows; different flows under stateful processing may access different states; and packets within the same flow may not change state. This makes it unlikely that consecutive packets will read and update the same state. Furthermore, the additional flow context table and scheduling module consume a large chip area and cannot support more complex stateful functions that cannot be handled by a single EFSM table.

[0038] Specifically, FIG4 is a block diagram of a stateful programmable data plane structure based on speculative execution provided by an embodiment of the present application.

[0039] As shown in FIG4 , the stateful programmable data plane structure based on speculative execution includes: multiple stage processors, side rings, multiple ring nodes, etc.

[0040] Among them, multiple stage processors form a data processing pipeline, and each stage processor processes one pipeline stage; the side ring includes multiple ring nodes, and the multiple ring nodes are connected to each stage processor one-to-one; any stage processor among the multiple stage processors is configured as a read processor or a write processor, the read processor reads the state that needs to be read for at least one stateful data packet, and predicts that the state of the stateful data packet remains unchanged; the write processor writes the state that needs to be written for the stateful data packet, identifies the data packet that needs to be resubmitted with the modified state, uploads the data packet that needs to be resubmitted to the corresponding ring node of the write processor, and sends it to the corresponding ring node of the read processor through the side ring, and the read processor downloads the data packet that needs to be resubmitted from the corresponding ring node.

[0041] In one embodiment of the present application, the stage processor includes a configurable scheduling module, wherein the configurable scheduling module of the read processor is configured as a read scheduling module, the configurable scheduling module of the write processor is configured as a write scheduling module, and the configurable scheduling modules of other stage processors are not activated.

[0042] It is understandable that the embodiment of the present application proposes a low-cost, high-performance data plane architecture that supports cross-stage data write-back: RAPID (Ring-Augmented Pipeline Dataplane). The overall technical architecture of RAPID is shown in Figure 4. In RAPID, the embodiment of the present application can add a simple side ring to the pipeline and connect each pipeline stage to it. As a specific example, on the side ring, the direction of the data flow and the pipeline can be opposite, thereby providing a fast backward communication path. In addition, the direction of the data flow and the pipeline can also be the same, or a bidirectional data path can be used, that is, data can flow in both directions between adjacent nodes; those skilled in the art can make specific choices or settings without specific limitations.

[0043] Secondly, the embodiment of the present application proposes a behavioral abstraction of "local read-remote write" in RAPID. For complex stateful functions, it involves a series of stage processors in the pipeline. The flow state table is stored on the first processor in the series, and the data packet reads the state on the first processor. After being processed by several processors, the state is written back across stages on the last processor in the series. The control signals, addresses, and data written back across stages are uploaded by the last processor to its corresponding ring node, sent to the ring node corresponding to the first processor through the side ring, and then written into the state table stored in the first processor.

[0044] In order to ensure state consistency, a configurable scheduling module is installed on each stage processor of the embodiment of the present application, as shown in Figure 5. According to the user program, the module can be reconstructed as a read scheduling module or a write scheduling module. In a "local read-remote write" series of processors, the scheduling module on the read processor is configured as a read scheduling module, the scheduling module on the write processor is configured as a write scheduling module, and the scheduling module on the middle processor is not activated. With the help of data prefetching technology, the scheduling module is implemented as a bypass without being inserted into the pipeline, so the activated scheduling module will not introduce additional delays for pipeline processing. When the user program does not need to write back data across stages, the scheduling module and its connected side rings are lossless to the pipeline performance.

[0045] Furthermore, the read scheduling module and the write scheduling module of the embodiment of the present application both include: a dirty table stored in a content-addressable memory, wherein the keyword used to match in the dirty table is a state table index of the flow, wherein the dirty table of the write scheduling module contains a table of keywords, and the dirty table of the read scheduling module contains a table of keywords and corresponding data; the write scheduling module writes the flow corresponding to the data packet with modified status into the dirty table, and sends a write-back control signal to the read scheduling module through the side ring, and the read scheduling module writes the flow corresponding to the data packet with modified status into the dirty table according to the write-back control signal.

[0046] It is understandable that both the read scheduling module and the write scheduling module of the embodiment of the present application are equipped with a small content addressable memory as a dirty table for registering the "dirty" flow currently undergoing stateful processing. The keyword used for matching in the dirty table is the state table index of the flow (for example, the hash value of the flow ID). The dirty table in the write scheduling module is a table containing only keywords, while the dirty table in the read scheduling module also contains related data.

[0047] Furthermore, the read processor of the embodiment of the present application also includes: a resubmission buffer for storing data packets that need to be resubmitted; the read scheduling module also includes: a data packet buffer for storing data packets that hit the dirty table, and the data packets of each flow are linked in a first linked list and a second linked list, the first linked list is used to store newly arrived data packets on the pipeline, and the second linked list is used to store data packets that need to be resubmitted.

[0048] In this embodiment of the present application, packets for flows not in the dirty table are processed without blocking, while packets for resubmitted and blocked dirty flows are opportunistically scheduled only during idle pipeline cycles. Therefore, resubmitted and blocked packets require a resubmission buffer (RB). Under normal traffic conditions, the switch pipeline is lightly loaded, leaving ample idle cycles to process buffered packets. Packets newly arriving from the pipeline and hitting the dirty table in the read scheduling module are stored in the read scheduling module's packet buffer (PB).

[0049] Both RB and PB are queues. Once the first packet in the RB has a chance to enter the pipeline, it moves from the RB to the PB. In the PB, packets for each flow are linked in two linked lists: the first linked list (NL) that stores newly arrived packets in the pipeline, and the second linked list (RL) that stores resubmitted packets. Metadata for the two linked lists is maintained in the dirty table, while the link pointer for each linked list node is stored in a dedicated register set.

[0050] Secondly, to ensure state consistency, the order in which packets of a flow are processed by the stateful function should be the same as the order in which they arrive at the switch. To maintain packet order, the read scheduling module of the embodiment of the present application also includes an order-preserving check module for scheduling packets in the packet buffer and detecting whether, when the packets in the packet buffer are scheduled, all packets that need to be resubmitted for the flow to which the scheduled packets belong have arrived at the resubmission buffer and have all been moved to the second linked list (RL) of the packet buffer.

[0051] The read scheduling module can schedule data packets of a flow in the PB only under the following conditions:

[0052] Condition (1) There are no data packets in the pipeline between local read and remote write of this flow, that is, all packets have been resent to RB;

[0053] Condition (2) All packets of the flow that need to be resubmitted in RB have been moved to PB.

[0054] Condition (2) can only be met after condition (1) is met. Condition (2) is guaranteed by the resubmitted packet counter maintained in the dirty table. Each time the RB receives a packet that needs to be resubmitted, the corresponding counter is incremented; each time a packet that needs to be resubmitted is moved from the RB to the PB, the corresponding counter is decremented. A counter value of 0 means that condition (2) is met. When a flow meets both (1) and (2), the RL is inserted as a whole before the NL to prepare for scheduling, and the new RL is reset to an empty linked list.

[0055] Furthermore, the embodiment of the present application designs two different checking schemes for condition (1) to ensure that when a data packet of a flow in the PB is scheduled, all data packets previously read in the stale state have been resubmitted to the read processor, as follows:

[0056] Solution 1: A timer-based solution starts a timer until the timer times out and determines that all packets that need to be resubmitted have arrived at the resubmission buffer.

[0057] Specifically, the timer-based solution maintains a timer field in the dirty table of the read scheduling module. When the read scheduling module receives a remote write signal, it writes the flow corresponding to the modified state packet to the dirty table and starts the timer. The timer is a countdown that continuously decreases over time. When the timer expires, it indicates that enough time has passed for all previously sent packets to return to the RB.

[0058] Because the data plane pipeline is non-blocking, the time it takes for a packet to be read locally and written remotely in the pipeline depends solely on the pipeline parameters and the number of stages between them. This is a compiler-determined constant (measured in master clock cycles). Furthermore, the time it takes for a packet to be resubmitted from the write processor to the read processor via the side ring, if it's not blocked, can also be predetermined by the compiler. The timer is initialized to the sum of these two values ​​at startup. Therefore, if there's no blocking on the side ring, the timer advances by one tick each cycle, just enough to ensure that all previously sent packets reach the RB.

[0059] It can be seen that the timer-based solution ensures that the timer value will only decrease in an unblocked period by designing the communication protocol on the side ring, so that even if the side ring is blocked, it can still ensure that all previously sent data packets arrive at the RB when the timer times out. The side ring can transmit three signals: cross-stage status writeback, data packets that need to be resubmitted, and clearing signals. In addition to the control tags, addresses, and data buses required by these three signals, the bus on the side ring of the embodiment of the present application adds an additional field to represent the heartbeat address.

[0060] The heartbeat address is a bitmap, with each bit corresponding to a stage on the chip. The write scheduler continuously sends heartbeats to the read scheduler. Because each pipeline stage's heartbeat occupies only one bit in the heartbeat address field, and the heartbeat address field is independent of other fields in the bus, any signal (considered an idle signal when the bus is idle) can simultaneously carry any number of heartbeats. At each ring node on the side ring, signals from the side ring compete with signals from the current stage being uploaded to the ring node. Except for the idle signal, which is directly discarded, other signals are stored in queue buffers in both directions, and bus arbitration is performed by the scheduler. If a signal carrying a heartbeat is blocked at a ring node, the read scheduler will not receive the heartbeat for a period of time. The read scheduler's timer advances by one time if and only if a heartbeat is received, thus achieving accurate timing. Whether idle cycles are blocked does not affect timing accuracy; the heartbeat carried by the idle signal can be directly transferred to another signal when it is discarded.

[0061] Solution 2: A counter-based solution. The sequence checking module of the embodiment of the present application is also used to write the serial number of each data packet into the counter table when each data packet is in the read state; if the serial number of the data packet that needs to be resubmitted is equal to the serial number in the counter table, it is determined that all the data packets that need to be resubmitted have arrived at the read processor.

[0062] Specifically, the counter-based solution adds a small amount of memory to each pipeline stage. These memories are used to store counter tables on the read processor. When the read scheduling module receives the remote write signal sent back, it writes the flow corresponding to the modified state data packet into the dirty table. The counter table maintains a sequence number for each flow. A metadata field is used to store the sequence number of each data packet. It is written and incremented when the data packet enters the pipeline, and the sequence number maintained by the counter table is the maximum sequence number of the stateful data packet that has passed. The sequence number of the data packet that needs to be resubmitted is compared with the sequence number in the local counter table. If the two are equal, it means that all the data packets sent before have returned to the RB.

[0063] It's important to note that the two solutions above correspond to two different use cases. The timer-based solution only requires adding additional fields to the dirty table, offering advantages in terms of chip area and power consumption. The counter-based solution is simpler to implement and doesn't rely on precise time counting, offering better scalability.

[0064] In one embodiment of the present application, the read processor is also used to start scheduling data packets of a flow in the data packet buffer, write a flushing signal to the corresponding ring node, send it to the corresponding ring node of the write processor through the side ring, and download it to the write processor; the write processor removes the corresponding flow from the dirty table according to the flushing signal.

[0065] It can be understood that when the write scheduling module detects that a packet will cause a change in its flow state (i.e., a remote write is triggered), the flow is registered in the write scheduling module's dirty table. When the write-back control signal reaches the target processor from the side ring, it simultaneously notifies the read scheduling module on that processor to register the flow in its own dirty table. If a subsequent packet in the write scheduling module hits the dirty table, it indicates a prediction error. The packet is uploaded to the side ring and resent to the read processor, re-entering the local read-remote write stateful processing flow. On the read processor side, the flow registered in the read scheduling module's dirty table serves as a safeguard, preventing newly arriving packets of that flow from entering stateful processing. During its residency, any packets of that flow that have already been allowed to pass will experience a prediction failure and be resubmitted to the read processor. When all prediction-failed packets of a flow have been resubmitted, the read scheduling module sends a flush signal to notify the write scheduling module to remove the flow from its dirty table. Once all backlogged packets have been sent, the flow is removed from the read scheduling module's local dirty table.

[0066] Because the dirty table records hash values ​​rather than flow IDs, hash collisions are possible. In the event of a hash collision, a flow that is not actually dirty may be found in the dirty table, causing it to be incorrectly blocked and resubmitted, resulting in a certain degree of performance loss. Under typical traffic conditions and hash functions, the probability of hash collisions is very low, so the impact of hash collisions on overall packet processing performance is almost negligible.

[0067] In one embodiment of the present application, the read scheduling module includes first to third levels of state consistency, wherein the first level of state consistency includes: the read operation of a data packet is after the write operation of all data packets before the data packet; the second level of state consistency includes: on the stream where the target data packet is located, the read operation of K data packets after the target data packet is allowed to be before the write operation of K data packets, and starting from the K+1th data packet, the read operation of the data packet is after the write operation of all data packets before the data packet; the third level of state consistency includes: the order of the read operation and the write operation of the data packet is arbitrary.

[0068] Among them, the first level of state consistency can be strong consistency, the third level of state consistency can be weak consistency, the second level of state consistency can be bounded staleness consistency, the target data packet can be any data packet, and the state of the flow where the target data packet is located obeys the bounded staleness consistency requirements. The preset number can be specified according to actual conditions, such as setting it to K.

[0069] Specifically, implementing strong consistency for all applications may not be worth the performance penalty, so embodiments of the present application allow users to specify the state consistency level for each application. Strong consistency means that data packets are never allowed to read stale states, that is, the read operation of a data packet must be after the write operations of all previous data packets; weak consistency means that data packets are allowed to read stale states, that is, the order of data packet read and write operations can be arbitrary; bounded stale consistency means that up to K data packets are allowed to ignore the read of stale states, and when K is reached, they must synchronize to the new state, where K is a user-specified constant.

[0070] It is understandable that embodiments of the present application can use predictive execution technology to process data packets. By default, the read processor always predicts that a stateful data packet will not modify its state. When a data packet does not modify its state, it does not cause state consistency issues, so there is no need to block the processing of subsequent data packets, thereby not affecting the packet processing performance of the switching chip. However, when a data packet needs to modify its state, it indicates that the previous prediction was wrong. At this time, not only does it need to write back the state data across stages, but it also needs to trigger the state consistency guarantee mechanism.

[0071] Specifically, for strong consistency, every packet that hits the dirty table in the write scheduling module should be resubmitted. For weak consistency, when a packet causes a state change, only the new state is written back without registering the flow in the dirty table, thus preventing the resubmission of subsequent packets. This is suitable for scenarios where the state can eventually converge even if some packets read stale state (for example, a rate limiter).

[0072] For bounded staleness consistency, a finite state machine (FSM) is used to guide the behavior of the write scheduling module under bounded staleness consistency, as shown in Figure 6. In the "normal" state, a flow is not in the dirty table of the write scheduling module and can pass normally without resubmission. When a packet causes a state change, the FSM transitions to the "expiring" state, and a decrementing counter is initialized to K. In this state, up to K packets of the flow can pass without resubmission. When the counter reaches 0, the FSM transitions to the "expired" state, and the flow is registered in the dirty table, starting a round of strict state synchronization. Only the flush signal from the read scheduling module can remove the flow from the dirty table of the write scheduling module and restore the state of the flow to the "normal" state.

[0073] In the actual implementation process, the embodiment of the present application can be based on the P4_16 language and enhance it by introducing a new abstract structure - the mutable table (muTable), which is equivalent to the table in P4. Compared with the table, the muTable can be modified by the data plane, so it can be used to implement the flow state table. User-written programs can apply read and write primitives to the muTable object to customize the processing logic of the stateful function. When compiling the application, the compiler decides whether it can be implemented in a single stage or multiple stages based on the complexity of the stateful function. Simple stateful functions that can be implemented by atomic operations will be compiled into a single stage, and the cross-stage data write-back function will not be enabled; complex stateful functions that cannot be implemented by atomic operations will be compiled into multiple stages and implemented using cross-stage data write-back.

[0074] If stateful processing involves modifying some data (for example, reducing the TTL field in the IPv4 protocol), packet resubmission may cause the data to be modified multiple times, resulting in a read-after-write conflict and incorrect results. The compiler uses a "field renaming" technique to solve this problem. For fields that are modified between the local read and the remote write, the compiler creates a copy of the field in the metadata. When reading the field between the local read and the remote write, it is read only from the location of the copy, not from the location of the field itself, which may have been modified once.

[0075] In summary, this application implements a low-cost, high-performance data plane structure to support complex stateful functions, which can offload more in-network computing applications from servers or middleboxes to programmable switches, thereby reducing the communication delay of in-network computing, reducing the use of network bandwidth, reducing the network system's demand for servers or middlebox devices, and simplifying the programming method of in-network computing applications, thereby reducing the overall development cost, deployment cost and maintenance cost of in-network computing applications.

[0076] The following describes in detail the stateful programmable data plane structure based on speculative execution according to the embodiment of the present application in conjunction with several specific application scenarios:

[0077] 1) Application scenario 1:

[0078] Load balancers (LBs) are an essential feature in cloud networks. Currently, more and more data centers are using stateful LBs instead of stateless LBs. In a stateful LB, client packets are sent to a virtual destination IP address, and the LB assigns a physical IP address to each packet. If an input packet finds an existing mapping from its flow to a physical IP address in the state table, it is forwarded using that address. Otherwise, a policy or algorithm is applied to select a physical IP address for the packet, and the new mapping from the packet flow to the physical IP address is written back to the state table.

[0079] Determining the physical server for a newly arrived flow can require complex logic. The conventional approach is to have the controller software on the server make the decision and then write the result back to the flow state table. Multiple forwarding and communication delays can put pressure on the packet buffer on the switch, reducing throughput. The present embodiment offloads the logic for determining a new physical IP address for stateful LB to the data plane through cross-stage writeback, effectively improving LB performance.

[0080] 2) Application Scenario 2:

[0081] DDoS attacks, such as DNS floods, TCP SYN floods, and amplification attacks, have always been a significant challenge for networks. Traditional server- or middlebox-based solutions are costly and have limited throughput. The cross-stage data writeback feature proposed in this application provides a high-performance DDoS detection solution that can be implemented on the switch data plane:

[0082] First, incoming packets are classified using access lists (e.g., blacklists, whitelists, and graylists for attack flows, normal flows, and suspicious flows, respectively). The detection / analysis module then analyzes them based on the list search results. This module can be a sketch or a series of header field checks. If a packet on the whitelist is deemed suspicious, the decision module updates the list and moves the corresponding flow from the whitelist to the graylist. Other inter-list movements are similar.

[0083] 3) Application scenario 3:

[0084] Traffic shaping and policy control are critical in enforcing Quality of Service (QoS) policies and ensuring optimal bandwidth allocation. In this process, packets are first classified based on user-defined header fields and then evaluated by some rate control or scheduling algorithm based on the current queue or link state. Next, the packet is queued or dropped, and the queue state is updated (i.e., written back to the evaluation module). In this case, the writeback is from the egress pipeline to the ingress pipeline. Since the pioneering work of PIFO, many efforts have been made to make packet scheduling programmable. However, most of them cannot avoid cross-stage data writeback. On today's pipeline-based chips, these schemes have to use packet recycling or approximation methods, which affects scheduling performance or accuracy.

[0085] 4) Application Scenario 4:

[0086] A stateful firewall can use a finite state machine to track and monitor active network connections and detect malicious traffic based on the flow context. Compared with a stateless firewall, it provides higher security and stronger attack mitigation capabilities. In a stateful firewall, the state table is first checked to determine whether the packet belongs to an existing connection. If so, the connection state and the associated timeout value are updated; otherwise, the packet is evaluated according to the predefined firewall rules to determine whether the connection should be allowed or denied, and the matching rules are applied. If the new connection is allowed, the state table is updated with the connection information (e.g., flow ID and connection state). Due to the complex state transitions and message passing between different modules, if this solution is to be implemented on a pure data plane pipeline, the cross-stage data write-back support provided by this application is also required.

[0087] 5) Application scenario 5:

[0088] Top-K high-flow detection can be used for traffic-aware routing, traffic engineering, and real-time network monitoring. Because the hash table used for Top-K high-flow detection exceeds the storage space of a single pipeline stage, in order to use Top-K detection in the data plane, the table needs to be distributed across the stages. The packet first needs to pass through each stage to obtain the appropriate location, and then the data is written back to the selected stage. In current pipeline devices, such operations must require the help of a controller or packet recycling. The cross-stage data writeback provided by this application can make this process very convenient and more efficient.

[0089] According to the stateful programmable data plane structure based on speculative execution proposed in the embodiment of the present application, a simple side ring is added to the pipeline so that each pipeline stage is connected to it. On the side ring, the data flow is opposite to the direction of the pipeline, providing a fast backward communication path. The read processor reads the state required to read the state of the data packet with state, and the write processor can write back the state across stages when writing back the data packet that needs to modify the state. The whole process is very convenient and more efficient, and supports complex stateful functions, thereby reducing the development cost, deployment cost and maintenance cost of in-network computing applications as a whole. As a result, the problems that the pipeline technology in the related art cannot support more complex stateful operations and has high development costs are solved.

[0090] In addition, an embodiment of the present application further provides a programmable switching chip, including the stateful programmable data plane structure based on speculative execution of the above embodiment.

[0091] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0092] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0093] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A stateful programmable data plane structure based on speculative execution, characterized in that: include: A plurality of stage processors, wherein the plurality of stage processors form a data processing pipeline, each stage processor processes one pipeline stage, and each stage processor is configured as a read processor or a write processor; A side ring, wherein the side ring includes a plurality of ring nodes, and the plurality of ring nodes are connected to each of the stage processors in a one-to-one correspondence; Any stage processor among the multiple stage processors is configured as a read processor or a write processor, the read processor reads the state that needs to be read for at least one stateful data packet and predicts that the state of the stateful data packet remains unchanged; the write processor writes the state that needs to be written for the stateful data packet, identifies the data packet that needs to be resubmitted with the modified state, uploads the data packet that needs to be resubmitted to the corresponding ring node of the write processor, sends it to the corresponding ring node of the read processor through the side ring, and the read processor downloads the data packet that needs to be resubmitted from the corresponding ring node.

2. The stateful programmable data plane structure based on speculative execution according to claim 1, wherein: The stage processor includes a configurable scheduling module, wherein the configurable scheduling module of the read processor is configured as a read scheduling module, the configurable scheduling module of the write processor is configured as a write scheduling module, and the configurable scheduling modules of other stage processors are inactivated.

3. The stateful programmable data plane structure based on speculative execution according to claim 2, wherein: The read scheduling module and the write scheduling module both include: A dirty table is stored in a content-addressable memory, wherein the keyword matched in the dirty table is the state table index of the flow, wherein the dirty table of the write scheduling module contains a table of keywords, and the dirty table of the read scheduling module contains a table of keywords and corresponding data; the write scheduling module writes the flow corresponding to the data packet with modified state into the dirty table, and sends a write-back control signal to the read scheduling module through the side ring, and the read scheduling module writes the flow corresponding to the data packet with modified state into the dirty table according to the write-back control signal.

4. The stateful programmable data plane structure based on speculative execution according to claim 3, wherein: The read processor further includes: a resubmission buffer for storing resubmitted data packets; the read scheduling module further includes: A data packet buffer, used to store data packets that hit the dirty table, and the data packets of each flow are linked in a first linked list and a second linked list, the first linked list is used to store newly arrived data packets on the pipeline, and the second linked list is used to store the data packets that need to be resubmitted; The order checking module is used to schedule the data packets in the data packet buffer and detect whether the data packets that need to be resubmitted in the flow where the data packets are scheduled have all arrived at the The buffer is resubmitted and has all been moved to the second linked list of the packet buffer.

5. The stateful programmable data plane structure based on speculative execution according to claim 4, wherein: The sequence checking module is further used for: A timer is started until the timer times out, and it is determined that all the data packets that need to be resubmitted have arrived at the resubmission buffer.

6. The stateful programmable data plane structure based on speculative execution according to claim 5, wherein: A heartbeat address field is provided on the bus of the side ring, the heartbeat address field is independent of other fields on the bus, and each pipeline stage occupies one bit of the heartbeat address field, and writes a signal carrying a heartbeat into the heartbeat address field; The timer advances one time when receiving the signal carrying the heartbeat sent from the corresponding write processor.

7. The stateful programmable data plane structure based on speculative execution according to claim 4, wherein: The sequence checking module is further used for: Writing the sequence number of each data packet into the counter table when each data packet is in a read state; If the sequence number of the data packet that needs to be resubmitted is equal to the sequence number in the counter table, it is determined that all the data packets that need to be resubmitted have arrived at the resubmission buffer.

8. The stateful programmable data plane structure based on speculative execution according to claim 4, wherein: The read processor is further configured to write a sweep signal to a corresponding ring node when a data packet of a flow in the data packet buffer begins to be scheduled, send the sweep signal to a ring node corresponding to the write processor via the side ring, and download the sweep signal to the write processor; The write processor removes the corresponding flow from the dirty table according to the flush signal.

9. The stateful programmable data plane structure based on speculative execution according to claim 4, wherein: The read scheduling module includes first to third levels of state consistency, wherein, The first level of state consistency includes: the read operation of a data packet is after the write operation of all data packets before the data packet; The second level of state consistency includes: on the flow where the target packet is located, the read operation of the K packets after the target packet is allowed before the write operation of the K packets, and starting from the K+1th packet, the read operation of the packet is allowed after the write operation of all packets before the packet; The third level of state consistency includes: the read and write operations of data packets are arbitrary.

10. A programmable switching chip, characterized in that: The method comprises a stateful programmable data plane structure based on speculative execution as described in any one of claims 1 to 9.

Images