Integrated circuit for use in association with a network exchange

Abstract

This application relates to flow control visibility. Some embodiments provide a method for a service management circuit of a data plane forwarding circuit. The service management circuit receives data messages from a set of ingress pipelines and provides the data messages to a set of egress pipelines. The method identifies flow control events. The method provides metadata related to the flow control events to the message generation circuit via a bus between the service management circuit and the message generation circuit of the data plane forwarding circuit.

Description

[0001] This application is a divisional application of the invention patent application filed on March 8, 2019, with application number 201980021359.8 and invention title "Flow Control Visibility". Technical Field

[0002] This application relates to flow control. Background Technology

[0003] Network forwarding elements use priority flow control as a mechanism to prevent specific data flows from overloading their capacity. An overloaded forwarding element can send a flow control message to its link partner (another forwarding element), instructing the link partner to suspend sending specific traffic (or resume sending previously suspended specific traffic once congestion has cleared). Similarly, a network forwarding element can receive flow control messages from its link partner, specifying that the forwarding element suspends (or resumes) sending specific traffic to that link partner. The use of flow control within the network can indicate service patterns within the network, but it is not easily traceable. Summary of the Invention

[0004] Some embodiments of the present invention provide an integrated circuit for use in association with a network switch, the network switch being used for packet forwarding operations in a network, the network switch being associated with a remote server in the network, the integrated circuit comprising: an ingress pipeline configured to process packet data received by the integrated circuit; a service management circuit for queuing the packet data processed by the ingress pipeline; and an egress pipeline configured to generate egress packet data for exiting the integrated circuit based on the dequeued packet data; wherein: the ingress pipeline and the egress pipeline include a matching action level to perform a matching action operation; the ingress pipeline includes at least one parser to determine the packet header value to be provided to the matching action level; the matching action operation is based on the packet header. The matching of data tuples with matching table data determines the packet data processing action associated with the forwarding operation; the integrated circuit provides metadata associated with the flow control message; the metadata includes: timestamp data; port identification data; flow control data; and other data indicating whether the flow control message corresponds to a flow control message received at (1) the network switch or a flow control message sent from the network switch (2); the timestamp data indicates the receive / send time associated with the flow control message; the port identification data indicates the receive / send port associated with the flow control message; the flow control data is based on congestion-related information; the metadata is provided to the remote server via message data; the message data includes message header data; and the metadata is used by the remote server.

[0005] Some embodiments of the present invention also provide corresponding methods, machine-readable storage media, network switches, and systems.

[0006] Some embodiments of the present invention provide a data plane forwarding circuit that can be configured to track flow control events and provide information about the flow control events to a (local or remote) monitor. The data plane forwarding circuit of some embodiments includes a configurable message processing stage configured to perform ingress and egress pipelines for processing data messages, and a service management circuit that acts as a cross-switching mechanism (among other things) to receive data messages from the ingress pipelines and provide the data messages to the correct egress pipelines. In some embodiments, the service management circuit is configured to identify flow control events and provide metadata related to these flow control events to a data message generation circuit of the data plane forwarding circuit. In some embodiments, the data message generation circuit stores the metadata and generates a data message including the metadata for the data plane forwarding circuit to send to a monitoring system.

[0007] In some embodiments, data plane forwarding circuitry processes data tuples associated with data messages received by the data plane in order to forward the data messages within the network. In some embodiments, the data plane is part of a network forwarding element (e.g., a switch, router, etc.) that includes control plane circuitry (“control plane”) that configures the data plane. In some embodiments, message processing levels, service management circuitry, and / or data message generation circuitry may be configured to process data messages and perform related operations (e.g., monitor and report flow control events) through the control plane. In other embodiments, the control plane that configures the data plane forwarding circuitry operates outside the forwarding element of the data plane (e.g., on a remote server). In some embodiments, the local control plane is implemented by a control software layer executed by one or more general-purpose processors (e.g., CPUs) of the forwarding element, while the remote control plane is implemented by a control software layer executed by one or more CPUs of another forwarding unit or a remote computer (e.g., a server).

[0008] Flow control events detected by the service management circuitry may include receiving flow control messages at the link-layer port of a forwarding element and / or generating flow control messages by the data plane forwarding circuitry. When a link partner of a network forwarding element (i.e., another forwarding element with a link-layer connection to the network forwarding element) becomes excessively congested, based at least in part on receiving data messages of a specific flow from the network forwarding element, the link partner sends a flow control message to the port of the network forwarding element to request the network forwarding element to suspend sending data messages with a specific priority (corresponding to the data flow causing the congestion) to that port. Similarly, if the link partner has cleared its congestion, it can send a flow control message instructing the network forwarding element to continue sending data messages with a specified priority to the link partner. These flow control messages are processed by the data plane forwarding circuitry, which stops or continues sending these data messages as requested.

[0009] In some embodiments, the service management circuitry detects when the data plane forwarding circuitry receives these flow control messages and sends metadata about the flow control messages to the data message generation circuitry. In some embodiments, the service management circuitry detects the reception of flow control messages when they are passed from the ingress pipeline to the service management circuitry. In other embodiments, the service management circuitry receives a signal summarizing the received flow control messages from the port that receives the flow control messages.

[0010] Additionally, when the data plane forwarding circuit generates flow control messages to send to link partners, the service management circuit also sends metadata about the flow control events. In some embodiments, the service management circuit is configured to generate these flow control messages in response to the detection of congestion in a specific queue (managed by the service management circuit). In this case, the service management circuit has the information needed to generate the metadata for the flow control events.

[0011] In some embodiments, the metadata includes: (i) a timestamp for the flow control event (generated by the circuitry of the service management circuitry); (ii) a receive / send indicator (e.g., a bit) indicating whether the event is the reception of a flow control message or the generation and transmission of a flow control message; (iii) a port identifier specifying the port through which the flow control message is received or transmitted; and (iv) the content of the flow control message (e.g., priority).

[0012] In some embodiments, the service management circuitry may send metadata for a flow control event to the data message generation circuitry every clock cycle via a bus from the service management circuitry to the data message generation circuitry. If more than one flow control event occurs within a clock cycle, the service management circuitry in some embodiments includes a queue for storing metadata for up to a certain number of flow control events. If the queue is required, the service management circuitry sends metadata for a flow control event every clock cycle until the queue is cleared.

[0013] As described above, in some embodiments, the data message generation circuit stores metadata received from the service management circuit. For example, in some embodiments, the data message generation circuit includes a buffer for storing metadata. When a specific condition occurs (e.g., the buffer exceeds a threshold, a specific time period elapses, etc.), the data message generation circuit generates a data message that includes (i) a set of data message headers and (ii) at least a portion of the metadata stored in the buffer. In some embodiments, the data message headers are pre-configured and include a destination network address for a monitoring system (e.g., a remote monitoring server). In some embodiments, the data message generation circuit provides the generated data message along with mixed metadata to one of the ingress pipelines, causing the data message to be processed by a data plane forwarding circuit and sent to the monitoring system.

[0014] The foregoing summary is intended as a brief introduction to some embodiments of the present invention. It does not imply an introduction or overview of all inventive subjects disclosed in this document. The following detailed description and the accompanying drawings referenced in that detailed description will further describe the embodiments described in the summary and other embodiments. Therefore, a full review of the summary, detailed description, and drawings is necessary to understand all the embodiments described in this document. Furthermore, the claimed subject matter is not limited to the illustrative details in the summary, detailed description, and drawings, but is defined by the appended claims, as the claimed subject matter may be embodied in other specific forms without departing from the spirit of the claimed subject matter. Attached Figure Description

[0015] The novel features of this invention are set forth in the appended claims. However, for illustrative purposes, several embodiments of the invention are illustrated in the following drawings.

[0016] Figure 1 An example of a network forwarding element is conceptually shown, including data plane circuitry that can be configured to track flow control events and provide information about these events to a monitor.

[0017] Figure 2 A business manager is conceptually illustrated in some embodiments.

[0018] Figure 3 The process for storing flow control event metadata in a queue is conceptually illustrated.

[0019] Figure 4 This conceptually illustrates some embodiments of the process by which the business manager executes in each clock cycle of the data plane circuitry to send flow control event metadata to the message generator.

[0020] Figure 5The concept illustrates data transmitted via a bus from the business manager to the message generator in some embodiments.

[0021] Figure 6 A message generator with some embodiments is conceptually illustrated.

[0022] Figure 7 This conceptually illustrates some embodiments of the process for generating data messages based on flow control event metadata received from the business manager.

[0023] Figure 8 An electronic system that implements some embodiments of the present invention is conceptually illustrated. Detailed Implementation

[0024] Some embodiments of the present invention provide a data plane forwarding circuit that can be configured to track flow control events and provide information about the flow control events to a (local or remote) monitor. The data plane forwarding circuit of some embodiments includes a configurable message processing stage configured to perform ingress and egress pipelines for processing data messages, and a service management circuit that acts as a cross-switching mechanism (among other things) to receive data messages from the ingress pipelines and provide the data messages to the correct egress pipelines. In some embodiments, the service management circuit is configured to identify flow control events and provide metadata related to these flow control events to a data message generation circuit of the data plane forwarding circuit. In some embodiments, the data message generation circuit stores the metadata and generates a data message including the metadata for the data plane forwarding circuit to send to a monitoring system.

[0025] Figure 1 An example of a network forwarding element 100 is conceptually illustrated, including data plane circuitry 120 that can be configured to track flow control events and provide information about these events to a monitor. The forwarding element 100 forwards data messages within the network and can be any type of forwarding element (e.g., a switch, router, bridge, etc.).

[0026] exist Figure 1 In this diagram, the forwarding element forwards data messages to and from at least two link partners 105 and 110 (the dashed arrows specifically illustrate the path of a data message from the first link partner 105 forwarded through the forwarding element 100 to the second link partner 110). Link partners 105 and 110 are connected to the forwarding element 100 at the link layer (Layer 2 in the Open Systems Interconnection (OSI) model). These link partners are, for example, other forwarding elements (i.e., software forwarding elements or other hardware forwarding elements).

[0027] Forwarding element 100 can be deployed as an edge forwarding element at the network edge to connect to computing devices (e.g., standalone or host computers) that serve as sources and destinations for data messages within the network, or as a non-edge forwarding element to forward data messages among other forwarding elements in the network.

[0028] As shown in the figure, the forwarding element 100 includes: (i) a data plane forwarding circuit 120 (“data plane”) that performs forwarding operations of the forwarding element 100 to forward data messages received by the forwarding element to other devices; and (ii) a control plane circuit 125 (“control plane”) that configures the data plane circuit. The forwarding element 100 also includes a physical port 112 that receives data messages from and sends data messages to link partners 105 and 110 (and any other link partners of the forwarding element 100).

[0029] Control plane 125 configures data plane 120 to process data messages and perform related operations (e.g., monitoring and reporting flow control events). In some embodiments, the control plane includes one or more processors (e.g., a microprocessor with multiple processing cores or units) that execute instructions for performing control plane operations when executed by the processors. These instructions may be specified by (i) the manufacturer of the network forwarding elements including control plane 125 and data plane 120, (ii) a network administrator who deploys and maintains the network forwarding elements, or (iii) one or more automated processes executed on servers and / or network forwarding elements that monitor network conditions. The control plane processor or another circuitry of the control plane communicates with the data plane via an interface (e.g., configuring the data plane or receiving statistics from the data plane).

[0030] Data plane circuitry 120 includes ports 115 that receive data messages for further processing and transmission after the data messages have been processed. In some embodiments, these ports 115 are link-layer ports for receiving and transmitting flow control messages. In some embodiments, some ports 115 of data plane 120 are associated with physical ports 112 of forwarding element 100, while other ports 115 are associated with other components of data plane 120. Data plane 120 also includes message generator circuitry (“message generator”) 135, multiple ingress pipelines 140, multiple egress pipelines 142, and service management circuitry (“service manager”) 144. In some embodiments, data plane 120 is implemented on an application-specific integrated circuit (ASIC), and its components are defined on that integrated circuit.

[0031] Message generator 135 generates messages in the data plane. In some embodiments, these messages may instruct circuitry in the data plane to perform certain operations or store data in the messages for export over a network to the control plane or another device. In some embodiments, message generator 135 generates data messages that are processed through ingress and / or egress pipelines 140 and 142 and sent over a network to the device of monitoring forwarding element 100. For example, in some embodiments, message generator 135 receives flow control event metadata and packages the metadata into a data message that is sent over a network to the monitoring device.

[0032] Ingress and egress pipelines 140 and 142 process data messages received via forwarding elements (and generated by message generator 135) to forward these messages to their destinations within the network. Service manager 144 acts as a crossover switch directing messages from the ingress pipeline to the egress pipeline. Additionally, as shown, data plane circuitry 115 includes a bus 150 that provides data from service manager 144 (e.g., flow control event metadata) to message generator 135. References below... Figure 2 The business manager of some embodiments is described in more detail.

[0033] Each ingress or egress pipeline includes several configurable (i.e., programmable) message processing stages 132, which can be configured to perform data plane forwarding operations of the forwarding element 100 to process data messages and forward them to their destinations. These message processing stages perform these forwarding operations by processing data tuples (e.g., message headers) associated with data messages received by the data plane 120 to determine how to forward the messages. In some embodiments, although the ingress and egress pipelines 140 and 142 are described as separate pipelines, they actually execute simultaneously on the same message processing resources of the data plane circuitry 115. In other words, in some embodiments, each message processing stage 132 may process one ingress pipeline data message and one egress pipeline data message within the same clock cycle.

[0034] In some embodiments, message processing level 132 is a Matching Action Unit (MAU), as shown in this example. In some embodiments, the MAU includes a matching table that stores multiple records for matching against data tuples of a processed data message (e.g., message header values ​​stored in the header vector passed from one MAU to the next). When a data message matches a matching record, the MAU then executes an action specified by an action record associated with the identified matching record (e.g., an action record identified by the identified matching record).

[0035] In some embodiments, each MAU also includes a set of stateful arithmetic logic units (ALUs) that perform arithmetic operations based on parameters specified by a header vector and / or a matching table. The stateful ALUs may store the results of their operations in stateful tables they access and / or may write these results into the header vector (e.g., directly or by instructing other action ALUs to write these results into the header vector) for use by other MAU levels to process data messages.

[0036] In addition to MAU level 132, each inlet or outlet pipeline 140 or 142 also includes a parser 130 and a de-parser 134. The parser 130 extracts message header values ​​from the data messages received from the pipeline for processing. In some embodiments, the message header values ​​are stored in a header vector that is passed to the first MAU level (and subsequently to subsequent MAU levels). The header vector is processed by successive message processing levels 132, and in some cases modified, as part of their message processing operations. As the pipeline's message processing level 132 operates on the header vector, the pipeline's parser 130 passes the message payload to the de-parser 134. In some embodiments, the parser passes the entire message, rather than just the payload, to the de-parser 134.

[0037] When the pipeline has finished processing the data message and must provide the message to the service manager (in the case of the ingress pipeline) or port 115 (in the case of the egress pipeline), the pipeline's de-parser 134 reconstructs the data message (including any modifications to the message header values). In some embodiments, the de-parser constructs a message header based on the modified message header vector received from the last message processing stage 132 and combines it with the payload (or a portion of the entire data packet designated as the payload) received from the parser 130.

[0038] Figure 2 A business manager 200 of some embodiments is conceptually illustrated. As described above, the business manager 200 receives reconstructed data packets from the ingress pipeline 205 and provides these data packets to the appropriate egress pipeline 210. The business manager 200 of some embodiments includes an ingress message analyzer 215, a switching structure and message replicator 220, an output buffer and queue 225, a queue monitor 230, and a flow control event generator 235.

[0039] Upon receiving a data message from one of the ingress pipelines 205, the ingress message analyzer 215 determines how to process the data message. In some embodiments, the ingress pipeline 205 provides the service manager with the reconstructed data message along with additional metadata, such as the egress queue, multicast group identifier, etc., for the data message. In some embodiments, the ingress pipeline 205 may also provide instructions to the service manager 200, such as modifying queue credits (which affect queue scheduling) and opening or closing certain queues (effectively performing flow control). Additionally, the ingress pipeline may process flow control messages from link partners of forwarding elements and provide these flow control messages to the service manager 200 (allowing the service manager to suspend or resume queues specified by the flow control messages).

[0040] In some embodiments, the ingress message analyzer 215 analyzes the metadata and data messages provided by the ingress pipeline 205 to determine how the service manager 200 will process the data messages. For example, if the metadata includes a multicast group identifier, this information is provided along with the data messages to the exchange structure and message replicator 220. Additionally, if a flow control message is received, the ingress message analyzer 215 provides this information to the flow control event generator 235.

[0041] In some embodiments, the switching structure and message replicator 220 process adding data messages to appropriate queues 225. In some embodiments, for unicast packets, the ingress pipeline identifies a specific queue, and the switching structure 220 adds the message to that queue. In some embodiments, the data message is actually added to the output buffer 225, and a reference to the location in the output buffer where the data message is stored is also added to the queue. Additionally, if the data message is a multicast message, the switching structure and message replicator 220 uses a multicast group identifier determined by the ingress pipeline to determine multiple queues 225 to which the data message is added. In some embodiments, the switching structure and message replicator 220 also select from output queue groups (e.g., queues corresponding to equal-cost multipath routing paths or within link aggregation groups).

[0042] As described above, the output buffer and queue 225 store data messages until the data messages are dequeued and provided to the egress pipeline 210. In some embodiments, the scheduler determines the order in which data messages are dequeued from each egress pipeline 210 based on the priority of data messages in different queues and other factors (no more than one data message is dequeued to each pipeline in each clock cycle).

[0043] In some embodiments, queue monitor 230 monitors statistics for each output queue 225. This includes queue depth (i.e., the number of data messages and / or bytes currently stored in the queue) and other queue statistics. In some embodiments, queue monitor 230 determines when the queue depth of certain queues exceeds various thresholds and signals the queue statistics to the inlet and / or outlet pipelines. Additionally, when queue monitor 230 detects that a particular queue has exceeded a threshold congestion level (i.e., the queue depth exceeds a specific threshold), queue monitor sends a message related to that queue to flow control event generator 235, enabling the flow control event generator to (i) generate and send a flow control message for that queue, and (ii) store metadata about this flow control event.

[0044] Flow control event generator 235 detects flow control events, stores metadata about these events in queue 240, and sends the metadata to one or more message generators (i.e., message generator 135 of data plane circuit 120) via bus 245. As shown, in some embodiments, flow control event generator 235 includes flow control message generator 250, timestamp generator 255, and the aforementioned metadata queue 240. (Refer to...) Figure 3 and Figure 4 Describe the operation of the flow control event generator.

[0045] Figure 3 A process 300 for storing flow control event metadata in a queue is conceptually illustrated in some embodiments. In some embodiments, a component of flow control event generator 235 performs process 300. As shown, process 300 begins by detecting flow control events (305). These flow control events may include receiving flow control messages at a link-layer port of a forwarding element and / or generating flow control messages by data plane forwarding circuitry.

[0046] When a link partner of a network forwarding element becomes excessively congested due at least in part to the reception of data messages belonging to a specific data flow from the forwarding element, the link partner sends a flow control message to the port of the network forwarding element to request the network forwarding element to suspend the transmission of data messages with a specific priority (e.g., corresponding to the data flow causing the congestion) to that port. Similarly, if the link partner has cleared its congestion, it can send a flow control message instructing the network forwarding element to continue transmitting data messages with a specified priority to the link partner. These flow control messages are provided to the service manager 200, which modifies the queue scheduler to suspend or resume the transmission of data messages from the specified queue based on the link partner's request.

[0047] Additionally, the flow control event generator 235 receives flow control messages, or at least receives metadata indicating the content of the flow control messages. In some embodiments, the flow control messages are passed from one of the ingress pipelines 205 to a service manager (e.g., to an ingress message analyzer 215, which provides the data to the flow control event generator 235). In other embodiments, the flow control event generator 235 receives a signal summarizing the received flow control messages directly from a link-layer port where the data plane receives the flow control messages.

[0048] Network forwarding elements (i.e., the data plane) also generate flow control messages to send to link partners, which are also detected as flow control events by flow control event generator 235. In some embodiments, service manager 200 (e.g., flow control message generator 250) is configured to generate these flow control messages in response to the detection of congestion in a particular queue.

[0049] Reference Figure 3 Process 300 generates (310) metadata for detected flow control events. In some embodiments, the metadata generated for a flow control event includes: (i) a timestamp for the flow control event (generated by timestamp generator 255); (ii) an identification (e.g., a single bit) of whether the flow control event is a received or sent flow control message; (iii) a port identifier specifying the link layer port through which the flow control message is received or sent; and (iv) the content of the flow control message (e.g., the flow priority or multiple priorities being paused or resumed, the pause time, and any other data). For received flow control messages, in some embodiments the metadata is based on messages received from a link layer port or ingress pipeline. For flow control messages sent by a forwarding element, the metadata is determined based on the data used by flow control message generator 250 to generate the message.

[0050] Next, process 300 determines (315) whether the metadata queue is full. In some embodiments, such as Figure 2 As shown, the flow control event generator includes a FIFO queue for storing flow control events. Because multiple flow control messages can be received and / or generated in the same clock cycle, in some embodiments, queue 240 is used to store the metadata of these messages before sending the metadata to the message generator. For example, in different embodiments, the queue may hold the metadata of eight, sixteen, or other flow control events.

[0051] If the metadata queue is full (e.g., due to a large number of flow control messages being received and / or sent recently), the process discards (320) the metadata (i.e., does not store the data). However, if there is space remaining in the metadata queue, the process 300 adds (325) the metadata for the detected flow control event to the queue.

[0052] In some embodiments, the flow control event generator 235 sends metadata for a flow control event from the metadata queue 240 to the message generator via the bus 245 in each clock cycle. Figure 4 The process 400 of some embodiments, executed by a service manager (e.g., a flow control event generator) in each clock cycle of a data plane circuit to send flow control event metadata to a message generator, is conceptually illustrated. As shown, process 400 first determines (405) whether any flow control event metadata is stored in a metadata queue.

[0053] If the queue contains metadata for at least one flow control event, process 400 sends (410) the metadata for the first (earliest) event in the queue, where a signal marks the event as valid. However, if no metadata is stored in the queue, the process sends (415) a signal to the message generator, which is designated as invalid. In other words, the business manager sends a signal to the message generator via the bus at each clock cycle. However, if there is no metadata to send, in some embodiments, the validity bit is set to 0, causing the message generator not to store the signal as flow control metadata.

[0054] Figure 5 A conceptual illustration shows data 500 transmitted from a service manager 200 to a message generator via a bus in some embodiments. As shown, data 500 includes validity bits, a timestamp, send / receive bits, a port identifier, and flow control data. In some embodiments, the validity bit is a single bit that identifies whether a signal is valid flow control metadata. The timestamp indicates the time when a flow control event was detected (i.e., the time when a forwarding element receives a flow control message or generates a flow control message). In some embodiments, the send / receive bit is a single bit that indicates whether the event represents a sent or received flow control message. The port identifier specifies a link-layer port of a data plane circuit on which flow control messages are received or transmitted. Finally, in some embodiments, the flow control event data indicates data contained in the flow control message (e.g., the affected priority, whether the priority is paused or resumed, and the length of time the priority is paused).

[0055] As described above, metadata from the service manager is sent to one or more message generators in the data plane circuitry. In some embodiments, the message generators store the metadata received from the service management circuitry (e.g., in a buffer or a set of buffers) and, when a specific condition occurs, generate a data message, including a data message header and at least a portion of the metadata stored in the buffer. The message generators then send these generated data messages to the processing pipeline of the data plane (e.g., an ingress pipeline), causing the data messages to be processed by the data plane and transmitted over the network to a monitor.

[0056] Figure 6 A message generator 600 is conceptually illustrated in some embodiments. As described above, the service manager 600 generates data and / or control messages to be sent to ingress and / or egress pipelines. These messages may be generated in response to the reception of certain signals (e.g., sending a specific type of control message), in response to the occurrence of certain conditions, etc. As shown, the message generator 600 includes at least one set of message header templates 605, a message aggregator 610, a pair of buffers 615 and 620, a buffer manager 625, and buffer storage logic 630.

[0057] Message header template 605 is a configured message header (e.g., stored in RAM) used by message aggregator 610 to generate data and / or control messages. For example, for certain types of messages (e.g., bidirectional forwarding detection heartbeat messages), the entire message may be stored in the message header template. For other types of messages that package runtime data into a message, message aggregator 610 combines this runtime data with a set of message headers from template 605. For example, to generate a data message with flow control event metadata, message aggregator 610 retrieves data from one of buffers 615 or 620 and appends a set of message headers to that data to direct data message 635 to the monitoring device.

[0058] In some embodiments, a buffer manager 625 manages buffers 615 and 620 and buffer storage logic 630. In some embodiments, buffer storage logic 630 receives flow control event metadata from a service manager via bus 640 and determines (i) whether to store the received metadata in one of buffers 615 and 620, and (ii) in which buffer to store the metadata (if it should be stored). The first decision (whether to store the metadata) is determined by a validity bit sent along with the metadata signal from the service manager. If the bit indicates that the metadata signal is valid data, buffer storage logic 630 stores the data in one of buffers 615 and 620. In some embodiments, which buffer is used is based on a signal from buffer manager 625. Specifically, the buffer manager specifies that buffer storage logic 630 stores the received data in one of buffers 615 or 620 until message aggregator 610 reads the buffer into a data message. At this point, buffer manager 625 specifies for buffer storage logic 630 to direct incoming metadata to the other of buffers 615 and 620. It should be understood that other mechanisms or circuit structures can be used to buffer flow control event metadata and generate data messages to send that metadata to a local or remote monitor.

[0059] Figure 7 A process 700 of some embodiments is conceptually illustrated, which generates a data message based on flow control event metadata received from a service manager (in different embodiments, or from other sources). Process 700 is executed by a message generator (e.g., message generator 600 or similar circuitry). In this example, the process uses two buffers to store the flow control event metadata; however, as mentioned, other embodiments may use only a single memory.

[0060] As shown in the figure, process 700 determines (705) to send the contents of the currently active metadata buffer to the monitor. In some embodiments, message generator 600 is configured to send the contents of the flow control event metadata buffer after a specific time period (or first of any of them) or based on other circumstances, when the flow control event metadata buffer reaches a threshold capacity.

[0061] Based on this determination, the process sets the currently inactive buffer (710) as the new active buffer. In some embodiments, the buffer manager 625 sends a different signal to the buffer storage logic 630, causing the buffer storage logic to store subsequently received metadata in another buffer (i.e., the currently empty buffer). The process also reads (715) the contents of the previously active buffer (i.e., the buffer that has already stored flow control event metadata received from the business manager). This use of multiple buffers allows the message generator (e.g., buffer manager 625 or message aggregator 610) to read from one of the buffers, while newly received metadata can be stored in different buffers without affecting the read operation.

[0062] Next, the process adds (720) a message header to the content read from the previous active buffer to generate a data message. In some embodiments, this message header is pre-configured for all data messages containing flow control event metadata and is stored by the message generator as a message header template 605. Each time the content of one of buffers 615 and 620 is read, the message aggregator 610 adds this message header to that content (or to a subset of the content if multiple data messages are sent for the content of a buffer) to generate a data message.

[0063] The process then sends the generated data messages (at 725) to the data plane pipeline. The process then ends. In some embodiments, the message generator always sends data messages to the ingress pipeline (as if receiving data messages at a port on the data plane), while other embodiments send some of the generated data messages to the service manager for direct delivery to the egress pipeline. These data messages are processed by the data plane and, in some embodiments, are sent to the destination monitoring device via an intermediate network by network forwarding elements. In other embodiments, the data messages are sent to a local monitor (e.g., on the data plane circuitry or on a processor running the control plane).

[0064] Figure 8 An electronic system 800 is conceptually illustrated, utilizing some embodiments of the present invention. The electronic system 800 may be a computer (e.g., a desktop computer, personal computer, tablet computer, server computer, mainframe, blade computer, etc.), a telephone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 800 includes a bus 805, a processing unit 810, a system memory 825, a read-only memory 830, a permanent storage device 835, an input device 840, and an output device 845.

[0065] Bus 805 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of electronic system 800. For example, bus 805 communicatively connects processing unit 810 to read-only memory 830, system memory 825, and permanent storage device 835. From these various memory units, processing unit 810 retrieves instructions to be executed and data to be processed to perform the processes of the present invention. In different embodiments, one or more processing units may be a single processor or a multi-core processor.

[0066] Read-only memory (ROM) 830 stores static data and instructions required by processing unit 810 and other modules of the electronic system. On the other hand, permanent storage device 835 is a read-write storage device. This device is a non-volatile storage unit that can store instructions and data even when the electronic system 800 is in a powered-off state. Some embodiments of the invention use mass storage devices (e.g., magnetic disks or optical disks and their corresponding disk drives) as permanent storage device 835.

[0067] Other embodiments use removable storage devices (e.g., floppy disks, flash drives, etc.) as permanent storage devices. Similar to permanent storage device 835, system memory 825 is a read-write storage device. However, unlike storage device 835, system memory is volatile read-write memory, such as random access memory. System memory stores some instructions and data required by the processor during operation. In some embodiments, the processes of the present invention are stored in system memory 825, permanent storage device 835, and / or read-only memory 830. Through these various storage units, processing unit 810 retrieves instructions to be executed and data to be processed in order to perform the processes of some embodiments.

[0068] Bus 805 is also connected to input and output devices 840 and 845. Input devices enable users to communicate information to the electronic system and select commands to be sent to it. Input device 840 includes an alphanumeric keypad and an indicator device (also referred to as a "cursor control device"). Output device 845 displays images generated by the electronic system. Output devices include printers and display devices such as cathode ray tube (CRT) or liquid crystal displays (LCDs). Some embodiments include devices such as touchscreens that function as both input and output devices.

[0069] Ultimately, as Figure 8 As shown, bus 805 also couples electronic system 800 to network 865 via a network adapter (not shown). In this way, the computer can be part of a computer network (e.g., a local area network (“LAN”), wide area network (“WAN”), or intranet) or a network of networks (e.g., the Internet). Any or all components of electronic system 800 can be used in conjunction with this invention.

[0070] Some embodiments include electronic components, such as microprocessors, storage devices, and memories that store computer program instructions in machine-readable or computer-readable media (which may also be referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Examples of such computer-readable media include RAM, ROM, read-only compressed optical disc (CD-ROM), recordable compressed optical disc (CD-R), rewritable compressed optical disc (CD-RW), read-only digital versatile optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / writable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard disk drives, read-only and recordable Blu-ray discs... Optical discs, high-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable medium may store a computer program that can be executed by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include, for example, machine code generated by a compiler and files containing high-level code executed by a computer, electronic components, or a microprocessor using an interpreter.

[0071] While the above discussion primarily concerns microprocessors or multi-core processors that execute software, some embodiments are executed by one or more integrated circuits (e.g., application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs)). In some embodiments, these integrated circuits execute instructions stored on the circuit itself.

[0072] As used in this specification, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include a person or group of people. For illustrative purposes, the terms "displayed" or "to be displayed" mean to be displayed on an electronic device. As used in this specification, the terms "computer-readable medium," "machine-readable medium," and "computer-readable medium" are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0073] Although the invention has been described with reference to many specific details, those skilled in the art will recognize that it can be practiced in other specific forms without departing from the spirit of the invention. Furthermore, several figures (including...) Figure 3 , Figure 4 and Figure 7These processes are illustrated conceptually. The specific operations of these processes may not be performed in the exact order shown and described. Specific operations may not be performed in a continuous sequence, and different specific operations may be performed in different embodiments. Furthermore, the process may be performed using multiple sub-processes or as part of a larger macro-process. Therefore, those skilled in the art will understand that the invention is not limited to the foregoing exemplary details, but is defined by the appended claims.

Claims

1. An integrated circuit for use in association with a network switch, the network switch being used for packet forwarding operations in a network, the network switch being associated with a remote server in the network, the integrated circuit comprising: The inlet pipeline can be configured to process data packets received by the integrated circuit; The service management circuit queues the data packets processed by the ingress pipeline; as well as The export pipeline can be configured to generate export data packets for exporting from the integrated circuit based on the dequeued data packet data; in: The inlet pipeline and the outlet pipeline include a matching action level to perform a matching action operation; The ingress pipeline includes at least one parser to determine the packet header values ​​to be provided to the matching action level; The matching action operation determines the packet data processing action associated with the forwarding operation based on the matching of packet header data tuples and matching table data; The integrated circuit provides metadata related to flow control messages; The metadata includes: Timestamp data; Port identification data; Flow control data; and Other data indicates whether the flow control message corresponds to (1) a flow control message received at the network switch or (2) a flow control message sent from the network switch; The timestamp data indicates the receive / send time associated with the flow control message; The port identification data indicates the receive / transmit port associated with the flow control message; The flow control data is based on congestion-related information; The metadata is provided to the remote server via message data; The message data includes message header data; and The metadata is used by the remote server.

2. The integrated circuit according to claim 1, wherein: The metadata is used by the remote server associated with flow control.

3. The integrated circuit according to claim 2, wherein: The congestion-related information is associated with the queue depth.

4. The integrated circuit according to claim 3, wherein: The integrated circuit is included in a dedicated integrated circuit; and The integrated circuit includes a data plane that can be configured by a central processing unit of a remote computer.

5. The integrated circuit according to claim 4, wherein: The message data includes mixed metadata.

6. The integrated circuit according to claim 5, wherein: The received flow control message comes from another network switch.

7. A method implemented using an integrated circuit, wherein the integrated circuit is used in association with a network switch, the network switch being used for packet forwarding operations in a network, the network switch being associated with a remote server in the network, the integrated circuit including configurable ingress pipelines, configurable egress pipelines, and service management circuitry, the method comprising: The configurable inlet pipeline processes the data packets received by the integrated circuit. The service management circuit queues the data packets processed by the ingress pipeline; as well as The configurable exit pipeline generates exit data packets for exiting the integrated circuit based on the dequeued data packet data; in: The inlet pipeline and the outlet pipeline include a matching action level to perform a matching action operation; The ingress pipeline includes at least one parser to determine the packet header values ​​to be provided to the matching action level; The matching action operation determines the packet data processing action associated with the forwarding operation based on the matching of packet header data tuples and matching table data; The integrated circuit provides metadata related to flow control messages; The metadata includes: Timestamp data; Port identification data; Flow control data; and Other data indicates whether the flow control message corresponds to (1) a flow control message received at the network switch or (2) a flow control message sent from the network switch; The timestamp data indicates the receive / send time associated with the flow control message; The port identification data indicates the receive / transmit port associated with the flow control message; The flow control data is based on congestion-related information; The metadata is provided to the remote server via message data; The message data includes message header data; and The metadata is used by the remote server.

8. The method according to claim 7, wherein: The metadata is used by the remote server associated with flow control.

9. The method according to claim 8, wherein: The congestion-related information is associated with the queue depth.

10. The method of claim 9, wherein: The integrated circuit is included in a dedicated integrated circuit; and The integrated circuit includes a data plane that can be configured by a central processing unit of a remote computer.

11. The method of claim 10, wherein: The message data includes mixed metadata.

12. The method according to claim 11, wherein: The received flow control message comes from another network switch.

13. A computer-readable storage medium storing a computer program, said computer program being executed by at least one processor to implement the method of any one of claims 7 to 12.

14. A network switch for packet data exchange operations in a network, the network switch being associated with a remote server in the network, the network switch comprising: The physical port coupled to the network; and An integrated circuit coupled to the physical port, the integrated circuit comprising: The inlet pipeline can be configured to process data packets received by the integrated circuit via at least one of the physical ports; The service management circuit queues the data packets processed by the ingress pipeline; and The export pipeline can be configured to generate export data packets for exporting from the integrated circuit based on the dequeued data packet data; in: The inlet pipeline and the outlet pipeline include a matching action level to perform a matching action operation; The ingress pipeline includes at least one parser to determine the packet header values ​​to be provided to the matching action level; The matching action operation determines the packet data processing action associated with the forwarding operation based on the matching of packet header data tuples and matching table data; The integrated circuit provides metadata related to flow control messages; The metadata includes: Timestamp data; Port identification data; Flow control data; and Other data indicates whether the flow control message corresponds to (1) a flow control message received at the network switch or (2) a flow control message sent from the network switch; The timestamp data indicates the receive / send time associated with the flow control message; The port identification data indicates the receive / transmit port associated with the flow control message; The flow control data is based on congestion-related information; The metadata is provided to the remote server via message data; The message data includes message header data; and The metadata is used by the remote server.

15. The network switch according to claim 14, wherein: The metadata is used by the remote server associated with flow control.

16. The network switch according to claim 15, wherein: The congestion-related information is associated with the queue depth.

17. The network switch according to claim 16, wherein: The integrated circuit is included in a dedicated integrated circuit; and The integrated circuit includes a data plane that can be configured by a central processing unit of a remote computer.

18. The network switch according to claim 17, wherein: The message data includes mixed metadata.

19. The network switch according to claim 18, wherein: The received flow control message comes from another network switch.

20. A system for use in association with a network, the system comprising: Network switches used for packet data exchange operations in the network; A server used in association with a network switch in the network, at least one of the network switches including: The physical ports coupled to the network; and An integrated circuit coupled to the physical port, the integrated circuit comprising: The inlet pipeline can be configured to process data packets received by the integrated circuit via at least one of the physical ports; The service management circuit queues the data packets processed by the ingress pipeline; and The export pipeline can be configured to generate export data packets for exporting from the integrated circuit based on the dequeued data packet data; in: The inlet pipeline and the outlet pipeline include a matching action level to perform a matching action operation; The ingress pipeline includes at least one parser to determine the packet header values ​​to be provided to the matching action level; The matching action operation determines the packet data processing action associated with the forwarding operation based on the matching of packet header data tuples and matching table data; The integrated circuit provides metadata related to flow control messages; The metadata includes: Timestamp data; Port identification data; Flow control data; and Other data indicates whether the flow control message corresponds to (1) a flow control message received at the network switch or (2) a flow control message sent from the network switch; The timestamp data indicates the receive / send time associated with the flow control message; The port identification data indicates the receive / transmit port associated with the flow control message; The flow control data is based on congestion-related information; The metadata is provided to the server via message data; The message data includes message header data; and The metadata is used by the server.

21. The system according to claim 20, wherein: The metadata is used by the server associated with flow control; and / or The congestion-related information is associated with the queue depth.

22. The system according to claim 21, wherein: The integrated circuit is included in a dedicated integrated circuit; and The integrated circuit includes a data plane that can be configured by a central processing unit of a remote computer.

23. The system according to claim 22, wherein: The message data includes mixed metadata.

24. The system according to claim 23, wherein: The received flow control message comes from at least one other network switch among the network switches.

Images