Direct memory access (DMA) engine for diagnostic data

By introducing diagnostic direct memory access circuits into network-connected devices, the monitoring problems of failures and performance degradation of complex network-connected devices are solved, efficient diagnostic data collection and analysis are achieved, and the efficiency of network performance debugging is improved.

CN114756489BActive Publication Date: 2024-11-12MELLANOX TECHNOLOGIES LTD(IL)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210020941.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-01-10
Filing Date
2022-01-10
Publication Date
2024-11-12
Estimated Expiration
2042-01-10

AI Technical Summary

Technical Problem

The prior art is difficult to effectively monitor and diagnose failures and performance degradation of complex network-connected devices, especially under high-speed transmission of large amounts of data and complex forwarding schemes, which lead to challenging network debugging and performance tuning.

Method used

A network-connected device is designed, including a diagnostic direct memory access (DDC) circuit that can receive and respond to diagnostic events, generate and collect lists of diagnostic data related to events, and send this data over the network to a designated destination for remote or Local analysis.

Benefits of technology

It realizes efficient collection and output debugging-related information, simplifies fault diagnosis and performance monitoring of network connected devices, and improves the efficiency and accuracy of network performance analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114756489B_ABST
    Figure CN114756489B_ABST
Patent Text Reader

Abstract

A network connection device includes at least one communication port, a packet processing circuit, and a diagnostic direct memory access (DMA) circuit (DDC). The at least one communication port is configured to transmit packets over a network. The packet processing circuit is configured to receive, buffer, process, and transmit packets. The DDC is configured to receive (i) one or more diagnostic events, and (ii) for each diagnostic event, a definition of a corresponding list of diagnostic data generated in the packet processing circuit and related to the diagnostic event, and in response to the occurrence of the diagnostic event, collect the corresponding list of diagnostic data from the packet processing circuit.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates generally to computer networks and, more particularly, to remote diagnostics of network connected devices. Background Art

[0002] Monitoring and diagnosing networks, especially network-attached devices (e.g., network interface cards, host interface adapters, switches, and routers), present significant challenges due to the large amounts of data transmitted at high speeds and complex forwarding schemes. In "Distributed Network Monitoring and Debugging with SwitchPointer," presented at the 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI'18), April 9-11, 2018, the authors describe a network monitoring and debugging system that leverages end-host resources and programmability to collect and monitor telemetry data, providing network visibility by using switch memory as a "directory service" for monitoring functions. Summary of the invention

[0003] Embodiments of the invention described herein provide a network connection device, including at least one communication port, a packet processing circuit and a diagnostic direct memory access (DMA) circuit (DDC). At least one communication port is configured to transmit packets over a network. The packet processing circuit is configured to receive, buffer, process and transmit packets. The DDC is configured to receive (i) one or more diagnostic events, and (ii) for each diagnostic event, a definition of a corresponding diagnostic data list generated in the packet processing circuit and related to the diagnostic event, and in response to the occurrence of a diagnostic event, collect the corresponding diagnostic data list from the packet processing circuit.

[0004] In some implementations, the DDC is configured to, in response to occurrences of a plurality of diagnostic events, collect a plurality of respective diagnostic data lists from the packet processing circuitry within respective time periods that at least partially overlap with one another.

[0005] In some embodiments, the DDC includes a plurality of DDC instances configured to collect the diagnostic data for building the list. In one embodiment, at least two of the DDC instances are configured to collect the diagnostic data from respective different portions of the packet processing circuit. Additionally or alternatively, at least two of the DDC instances may be configured to collect the diagnostic data associated with the same diagnostic event.

[0006] In a disclosed embodiment, the packet processing circuit includes a packet processing pipeline having a plurality of pipeline stages, and the DDC is configured to collect the diagnostic data from two or more of the pipeline stages. In an example embodiment, the DDC is configured to collect the diagnostic data by reading one or more counter values ​​in the packet processing circuit.

[0007] In another embodiment, the DDC is configured to receive a diagnostic trigger and collect the corresponding diagnostic data list from the packet processing circuit in response to the diagnostic trigger.

[0008] In some embodiments, the DDC is configured to send the collected list to a specified destination. In one embodiment, the network connection device also includes a memory, and the specified destination includes a storage location in the memory. In another embodiment, the specified destination includes a processor of the network connection device, or a host hosting the network connection device. In yet another embodiment, the specified destination includes an address of another network connection device coupled to the network. In a disclosed embodiment, the packet processing circuit includes an egress queue for queuing egress packets, and the DDC is configured to send the diagnostic data list to the egress queue.

[0009] According to an embodiment of the present invention, a method is further provided, including receiving, buffering, processing and transmitting packets through a network using a packet processing circuit in a network connection device. Using a diagnostic direct memory access (DMA) circuit (DDC) in the network connection device, receiving (i) one or more diagnostic events, and (ii) for each diagnostic event, a definition of a corresponding diagnostic data list generated in the packet processing circuit and related to the diagnostic event. In response to the occurrence of a diagnostic event, the corresponding diagnostic data list is collected from the packet processing circuit using the DDC.

[0010] According to an embodiment of the present invention, there is further provided a method, comprising, in a network device, receiving, processing and transmitting packets through a network using a packet processing circuit. A diagnostic circuit is operated in the network device to receive (i) diagnostic events expected to occur in the network device, and (ii) for each diagnostic event, a definition of a corresponding diagnostic data list to be collected from the packet processing circuit for analysis, and when a diagnostic event occurs, the diagnostic data list corresponding to the diagnostic event is collected according to the definition.

[0011] The present invention will be more fully understood through the following detailed description of its embodiments in conjunction with the accompanying drawings, in which: BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1is a block diagram schematically illustrating a computer network according to an embodiment of the present invention;

[0013] Figure 2 is a block diagram schematically illustrating a network connection device (NCD) according to an embodiment of the present invention;

[0014] Figure 3 is a flow chart schematically illustrating a method for efficiently collecting diagnostic data from an NCD according to an embodiment of the present invention;

[0015] Figure 4 is a block diagram schematically illustrating a diagnostic event signaling circuit in an NCD according to an embodiment of the present invention;

[0016] Figure 5 is a block diagram schematically illustrating a direct memory access (DMA) scheme for transmitting diagnostic data in an NCD according to an embodiment of the present invention; and

[0017] Figure 6 is a block diagram schematically illustrating a diagnostic data output circuit that outputs diagnostic data from an NCD according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Traditional computer networks can be monitored for slow or faulty components, failed links, and incorrect or suboptimal forwarding decisions. Monitoring and debugging of large-scale networks is a challenging problem, for example, because network performance is affected by real-time decisions made by numerous network elements, including routing paths, packet loss, buffer sizes, various congestion relief parameters, etc.

[0019] Network connection devices (NCDs), such as routers, switches, network interface controllers (NICs), and host channel adapters (HCAs), for example in network fabrics such as Fat-tree, Dragonfly, etc., may include information that aggregates and accumulates forwarding decisions related to the device; such information may be useful for network debugging, network performance tuning (and other applications; e.g., machine learning of network behavior). This information typically includes cache hit rates, packet loss rates, buffer occupancy, routing decisions, security events, etc. In order to diagnose failures or performance degradation of the NCD, it may be necessary to output part of the accumulated data for analysis. However, this task can be complex for high-performance NCDs with rich feature sets, where the accumulated data spans various circuits of the NCD.

[0020] Embodiments of the present invention described herein provide methods and systems for efficiently collecting and outputting debugging related information. In an embodiment, NCD includes one or more diagnostic direct memory access circuits (DDCs). DDC is configured to receive the definition of diagnostic events and corresponding diagnostic data from a processor (local or remote), if a diagnostic event occurs, these corresponding diagnostic data should be monitored. In an embodiment, DDC then programs the various diagnostic event sources (e.g., inlet port processing circuit, outlet port processing circuit, shared buffer, forwarding table and queuing system) in the NCD to detect the occurrence of diagnostic events and send with a signal. In some embodiments, diagnostic events can also be software triggered (e.g., a user can send a DDC to interpret as a command that detects a diagnostic event).

[0021] When the DDC receives an indication that a diagnostic event has been detected by any diagnostic event source, the DDC may DMA the corresponding diagnostic data to the diagnostic data buffer. In some embodiments, the NCD is configured to send the contents of the diagnostic data buffer over a network for analysis by a remote diagnostic station; in other embodiments, the diagnostic data is analyzed locally. It should be noted that the diagnostic data required to analyze a diagnostic event is not limited to the diagnostic event source; rather, diagnostic data from various subunits of the NCD may be required.

[0022] Further details will be disclosed in the following system description with reference to example embodiments.

[0023] System Description

[0024] Figure 1 1 is a block diagram schematically illustrating a computer network 100 according to an embodiment of the present invention. The computer network 100 includes a network connection device (NCD) 102 that communicates with peer network connection devices via a communication network 104. The NCD 102 may include, for example, a switch, a router, a network adapter such as a network interface controller (NIC) or a host channel adapter (HCA), a network-enabled graphics processing unit (GPU), or any other suitable type of network device.

[0025] NCD 102 includes packet processing circuitry 106 that is configured to receive, buffer, process, and transmit packets, and may include a plurality of buffers, queues, routing tables, congestion relief circuits, forwarding circuits, and other circuits. In an embodiment, computer network 100 also includes a remote diagnostic station 110 coupled to communication network 104 and may be used to remotely diagnose NCD 102. Additionally or alternatively, a local diagnostic station 112 may be coupled to NCD 102, for example, via a serial link, and the DDC may be configured to send monitored data via the serial link.

[0026] To facilitate online monitoring and diagnosis, the NCD 102 includes a diagnostic direct memory access circuit (DDC) 108 that is configured to detect diagnostic events and, in response to the detection, send diagnostic data to a diagnostic station. The DDC receives definitions of diagnostic events—e.g., predefined forwarding decisions, packet loss, buffer overflows, or less severe events (such as cache misses). For each defined diagnostic event, the DDC further receives definitions of corresponding data to be monitored; for example, for a packet loss event, the corresponding monitoring data may include buffer occupancy and cumulative cache hit rate. When a predefined diagnostic event occurs, the DDC accumulates the corresponding monitoring data and then sends the monitoring data to a predefined destination received by the DDC (e.g., to a remote diagnostic station 110).

[0027] Therefore, according to Figure 1 In the illustrated example implementation, the DDC receives an indication of a detected diagnostic event, collects monitoring data corresponding to the event, and transmits the monitored data to a predefined diagnostic station, which may be local or remote.

[0028] Figure 2 1 is a block diagram schematically illustrating a network connection device (NCD) 102 according to an embodiment of the present invention. The NCD 102 includes an ingress port and a processing unit 200 configured to receive and process ingress packets; a shared global buffer 202 configured to temporarily store ingress packets; a forwarding table 204 configured to define the destination of the ingress packets; a queuing system 208 configured to queue egress packets; and an egress port and a processing unit 202 configured to modify and output egress packets.

[0029] Each ingress port and processing circuit 200 includes an ingress port 210 configured to receive data from the network 104 ( Figure 1 ) receives packets, a buffer 212 configured to buffer ingress packets, and a control plane 214 configured to control the operation of the ingress ports and processing. Each egress port and processing circuitry 202 includes a packet modifier 216 configured to modify the egress packet (typically adding a header), an egress port 218 configured to send the packet over the network 104, and a control plane 220 configured to control the operation of the egress port and processing circuitry.

[0030] When NCD 102 transmits packets through network 104, various events may occur, such as packet loss, security blocking of packets, transmission of congestion notification packets, etc.; and it may be necessary to detect and / or monitor the occurrence of some or all of such events. The event may originate from any one of circuits 200, 202, 204, 206 or 208 (which will be referred to as "diagnostic event sources" hereinafter). When a diagnostic event occurs, it may be necessary to monitor certain diagnostic data. For example, when a packet is discarded, it may be necessary to check the status of the corresponding transmission queue (in an embodiment, the status of the transmission queue is continuously monitored and stored in local memory; and, when packet discard is detected, the stored monitoring data should be sent for inspection).

[0031] To facilitate diagnosis, the NCD 102 also includes a diagnostic DMA circuit (DDC) 222 and a CPU 226. The DDC is configured to receive definitions of diagnostic events and corresponding diagnostic data from the CPU 226 (in some embodiments, the DDC may receive diagnostic event definitions from a remote processor). The DDC then programs the corresponding diagnostic event source to detect diagnostic events and send a detection signal to the DDC. When the DDC receives a diagnostic event detection signal from any diagnostic event source, the DDC collects the corresponding diagnostic data and DMA transfers the data to a buffer (not shown; reference will be made below). Figure 4 for discussion).

[0032] In some embodiments, NCD 102 includes multiple DDC circuits, and if multiple diagnostic events are detected simultaneously or very close in time to each other, the diagnostic events can be assigned to different DDC circuits, which will simultaneously DMA transfer corresponding diagnostic data. In an embodiment, diagnostic data from multiple sources can be collected at partially or completely overlapping time intervals. In some embodiments, two or more DDC instances can be configured to detect the same diagnostic event (and usually collect different data). In some embodiments, the packet processing circuit includes a pipeline, and the DDC is configured to collect diagnostic data from multiple pipeline stages.

[0033] The NCD may then send the diagnostic data collected by the DDC to a remote diagnostic station, which may be coupled to the network or coupled to the NCD 102. In an embodiment, the NCD sends the diagnostic data only when the data collected in the buffer is complete; in a second embodiment, the NCD sends the diagnostic data while data collection is still in progress; in a third embodiment, the NCD may be configured to send the data when the collection is complete, or to send the data while the collection is in progress; and, in a fourth embodiment, the NCD is configured to send the diagnostic data from the first set of diagnostic data sources when the data collection is complete, and to send the diagnostic data from the second set of diagnostic data sources while the collection is still in progress.

[0034] Understandably, Figure 2 The structure of NCD 102 shown and described above is an example embodiment cited as an example. Other circuits can be used in alternative embodiments. For example, in some embodiments, the NCD includes a circuit that initiates (rather than relays) a communication packet (e.g., generated by CPU 226). In some embodiments, an inlet / outlet bidirectional port is used. CPU 226 may include one or more general-purpose programmable processors that are programmed with software to perform the functions described herein. The software can be downloaded to the processor in electronic form, such as via a network, or it can be provided and / or stored in a non-transitory tangible medium, such as a magnetic, optical or electronic memory, alternatively or additionally.

[0035] Figure 3 300, which schematically illustrates a method for efficiently collecting diagnostic data from an NCD according to an embodiment of the present invention. Figure 2 )implement.

[0036] The flowchart begins with a get list step 302, in which the DDC gets a list of diagnostic events and corresponding diagnostic data (hereinafter referred to as a "DEDD-list"). The DEDD list can be obtained by the CPU 116 ( Figure 2 ) is provided, which typically generates the list in response to a debug request received by the CPU, for example from a diagnostic station 110 or 112.

[0037] Next, the DDC enters a program diagnostic sources step 304 and programs one or more diagnostic sources to indicate detection of a corresponding diagnostic event (e.g., a transmit queue may be programmed to signal a queue overflow event), and then enters a check diagnostic events step 306, where the DDC loops until a diagnostic event is detected and signaled by one (or more) diagnostic event sources.

[0038] When a diagnostic event is signaled by one or more diagnostic sources, the DDC enters DMA transfer step 308, consults the DEDD list (defined in step 302) and DMA transfers the diagnostic data corresponding to the diagnostic event to the diagnostic data buffer. The DDC then re-enters step 306 to wait for the next diagnostic event.

[0039] It should be understood that Figure 3The flowchart 300 shown in and described above is a conceptual flowchart referenced by way of example only for conceptual clarity. Alternative flowcharts may be used in alternative embodiments of the present invention. For example, in an embodiment, the DDC may receive updates to the DEDD list while waiting for a diagnostic event (e.g., looping on step 306) or while the DDC is in any other step. The DDC may refresh the DEDD list or portions thereof in response to, for example, instructions from the CPU 226. (See below for more information.) Figure 6 The example implementation shown is used to describe the mechanism for outputting diagnostic data from the NCD.

[0040] Diagnostic Event List

[0041] The following is a non-limiting list of examples of diagnostic events and diagnostic data. In some embodiments, some of the diagnostic events below may not be used; in other embodiments, additional diagnostic events may be used.

[0042] Entry ports and processing events:

[0043] 1. Buffer events (e.g., queue full, queue empty).

[0044] 2. Mirror events (eg, generate mirror packets).

[0045] 3. Forwarding events (eg, forwarding decisions).

[0046] 4. Supervision events (e.g., drop events when a maximum rate is exceeded, packets of a predefined type are dropped, or when a minimum or maximum rate for a predefined queue is reached).

[0047] 5. Shaping events (e.g., the maximum or minimum bandwidth set for a queue is not met).

[0048] 6. WRED events (eg, random packet loss events).

[0049] Global shared buffer events:

[0050] 1. Queue occupancy (for example, the occupancy exceeds a preset threshold).

[0051] 2. Buffer Histogram

[0052] 3. Tail drop counter (e.g., number of packets dropped by the tail drop algorithm)

[0053] 4. Congestion counters (e.g., counters that count congestion events)

[0054] 5. Pool occupancy (for example, the occupancy of the buffer pool)

[0055] Forwarding table events:

[0056] 1. Cache counters (e.g., cache hit / miss counters)

[0057] 2. Forwarding Action

[0058] 3. ACL (Action List) Counter

[0059] 4. Flow counter (eg, counting packets with predefined header field values).

[0060] Queuing System Events

[0061] 1. AQM (Active Queue Management) Events

[0062] 2.ETF (Enhanced Transmission Option)

[0063] 3. Shaping events (e.g., setting maximum bandwidth based on class of service)

[0064] Egress ports and processing events

[0065] 1. Port Histogram

[0066] 2. Latency Histogram

[0067] 3. Various port counters

[0068] Figure 4 4 is a block diagram 400 schematically illustrating a diagnostic event signaling circuit in NCD 102 according to an embodiment of the present invention. Each inlet port and processing circuit 200 ( Figure 2 ) includes an entry processing diagnostic event source circuit 402, which is configured to detect a pre-programmed diagnostic event and signal a pre-programmed diagnostic event (e.g., the event programmed in the program diagnostic event step 304, Figure 3 ).

[0069] In a similar manner, the shared global buffer 204 ( Figure 2 ) includes a shared global buffer diagnostic event source circuit 404, the forwarding table 206 includes a forwarding table diagnostic event source 406, the queuing system 208 includes a queuing system diagnostic event source 408 and each egress port and processing circuit 202 includes an egress processing diagnostic event source 410.

[0070] DDC 222 receives diagnostic signaling from all sources in the NCD and, in response, may initiate DMA transfers of corresponding diagnostic data (step 308 of flowchart 300, Figure 3 ).

[0071] It should be understood that Figure 4The diagnostic event signaling 400 shown in and described above is an example implementation cited as an example. Other diagnostic event signaling schemes can be used in alternative implementations. In some embodiments, for example, circuits 402 to 410 (or a subset thereof) can send a detection signal of a diagnostic event to a CPU (e.g., CPU 226), and the CPU can then send a diagnostic event signal to DDC 222. In other embodiments, the diagnostic event signaling can be performed by a serial or parallel bus.

[0072] Figure 5 5 is a block diagram 500 schematically illustrating a DMA scheme for transmitting diagnostic data in NCD 102 according to an embodiment of the present invention. Each inlet port and processing circuit 200, shared global buffer 204, forwarding table 206, queuing system 208 and egress port and processing circuit 202 ( Figure 2 ) (collectively referred to as diagnostic data sources) are configured to send corresponding diagnostic data to diagnostic buffer 504 via diagnostic data bus 502. DDC 222 controls DMA transfer, grants the diagnostic bus to one of the diagnostic data sources, and sets the start address and end address of the data to be transferred.

[0073] In some embodiments, the DDC DMA transfers the diagnostic data to the CPU 226 or CPU memory when the predefined destination for the diagnostic data is the CPU 226 or an address range in a memory of the CPU 226 (not shown). In other embodiments, the DDC DMA transfers the diagnostic data to the host and / or host memory over a system bus (not shown) when the predefined destination for the diagnostic data is the host CPU hosting the NCD or an address range in a memory of the host CPU.

[0074] It should be understood that Figure 5 The DMA scheme 500 shown in and described above is an example implementation cited as an example. Other DMA schemes may be used in alternative implementations. For example, in some implementations, the diagnostic data buffer 504 is distributed and coupled to the diagnostic data source via multiple buses. In an implementation, the diagnostic data buffer 504 is embedded in the shared global buffer 204 ( Figure 2 )middle.

[0075] Figure 6 6 is a block diagram 600 schematically illustrating a diagnostic data output circuit for outputting diagnostic data from an NCD according to an embodiment of the present invention. Figure 3 ) after which the diagnostic data is stored in the diagnostic data buffer 504 ( Figure 5 ), wherein the DDB DMA transfers diagnostic data associated with one or more detected diagnostic events.

[0076] Figure 6 The illustrated example embodiment depicts three paths, designated Path A, Path B, and Path C, in which diagnostic data may be output from the NCD 102 .

[0077] Path A can be used to send diagnostic data to, for example, a remote diagnostic station 110 ( Figure 1 ). In some embodiments, the predefined destination of the diagnostic data received by the DDC includes the address of a peer NCD (e.g., a CPU within the NCD or a host hosting the NCD); the diagnostic data from the diagnostic data buffer 504 is input to the packet encapsulator 602, which adds a header and trailer to the diagnostic data and sends the packet to the forwarding table 206. The forwarding table 206 receives the packaged diagnostic data (as well as non-diagnostic packets received by the forwarding table from the shared global memory 204) and sends the diagnostic packet to the network 104 through the queuing system 208 and one of the egress ports and processes 202.

[0078] Type B routing may be used when the diagnostic station is local, coupled to the NCD 102 via a local serial port 604 (e.g., a universal serial bus (USB)). Data is routed from the diagnostic data buffer 504 to a serial port interface 606, which outputs the diagnostic data to the diagnostic station coupled to the serial port. In this case, communication is bidirectional—the diagnostic station can extract diagnostic data from the diagnostic data buffer.

[0079] Type C routing may be used when the NCD performs diagnostics; for example, by diagnostic software running on CPU 226 (or any other processor embedded in NCD 102). CPU 226 may read the diagnostic data buffer or portions thereof and diagnose data associated with predefined diagnostic events.

[0080] It should be understood that Figure 6 The diagnostic data output circuit shown in and described above is an example implementation cited as an example. Other circuits can be used in alternative implementations. For example, in some embodiments, the diagnostic data buffer 504 is embedded in the shared global buffer 204. In an embodiment, the packet encapsulation is completed when the diagnostic data is stored in the diagnostic data buffer. In other embodiments, the diagnostic data buffer 504 is coupled to a system bus (e.g., a high-speed peripheral component interconnect (PCIe)), and the diagnostic data is read from the diagnostic data buffer by a processor coupled to the system bus.

[0081] Figures 1 to 6The illustrated configuration of the NCD 102 including the DDC 222 and the method of the flowchart 300 are example configurations and flowcharts, which are described purely for conceptual clarity. Any other suitable configurations and flowcharts may be used in alternative embodiments. The NDC and DDC may be implemented using suitable hardware, such as in one or more application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs), using software, using hardware, or using a combination of hardware and software elements.

[0082] In some embodiments, some or all of the functions performed by the DDC 222 may be performed by the CPU 226. In embodiments, the DDC 222 may be implemented using one or more general purpose programmable processors that are programmed in software to perform the functions described herein. The software may be downloaded to the processor in electronic form, such as over a network, or it may alternatively or additionally be provided and / or stored on a non-transitory tangible medium, such as a magnetic, optical, or electronic memory.

[0083] It should be understood that the above embodiments are cited by way of example, and the present invention is not limited to the contents specifically shown and described above. On the contrary, the scope of the present invention includes the combination and sub-combination of the various features described above, as well as changes and modifications thereof that would occur to those skilled in the art after reading the foregoing description and that are not disclosed in the prior art.

Claims

1. A network connection device, comprising: at least one communication port configured to communicate packets over a network; a packet processing circuit configured to receive, buffer, process and transmit the packet, the packet processing circuit comprising a plurality of subunits generating diagnostic data; Diagnostic data buffer; a diagnostic data bus connecting the plurality of subunits to a diagnostic data buffer, and A diagnostic direct memory access (DMA) circuit (DDC), the diagnostic direct memory access (DMA) circuit (DDC) being configured to: receiving (i) one or more diagnostic events and (ii) for each diagnostic event, a definition of a corresponding list of diagnostic data generated in the packet processing circuitry and associated with the diagnostic event; and In response to the occurrence of a diagnostic event, diagnostic data corresponding to the list of diagnostic events is collected from the sub-units of the packet processing circuit in a diagnostic data buffer over a diagnostic data bus.

2. The network connection device of claim 1 , wherein the DDC is configured to collect diagnostic data in a plurality of corresponding diagnostic data lists from the packet processing circuitry in response to occurrences of a plurality of diagnostic events during corresponding time periods that at least partially overlap with each other. 3 . The network connection device of claim 1 , wherein the DDC comprises a plurality of DDC instances configured to collect the diagnostic data in the list.

4. The network connection device of claim 3, wherein at least two of the DDC instances are configured to collect the diagnostic data from respective different portions of the packet processing circuitry. 5 . The network connection device of claim 3 , wherein at least two of the DDC instances are configured to collect the diagnostic data related to a same diagnostic event.

6. The network connection device of claim 1, wherein the packet processing circuitry comprises a packet processing pipeline having a plurality of pipeline stages, and wherein the DDC is configured to collect the diagnostic data from two or more of the pipeline stages.

7. The network connection device of claim 1, wherein the DDC is configured to collect the diagnostic data by reading one or more counter values ​​in the packet processing circuit.

8. The network connection device of claim 1, wherein the DDC is configured to receive a diagnostic trigger, collect the corresponding diagnostic data list from the packet processing circuit in response to the diagnostic trigger, and send the collected diagnostic data to a designated destination.

9. The network connection device of claim 1, wherein the DDC is configured to send the collected diagnostic data to a designated destination.

10. The network connection device of claim 9, further comprising a memory, wherein the specified destination comprises a storage location in the memory.

11. The network connection device of claim 9, wherein the designated destination comprises a processor of the network connection device, or a host hosting the network connection device.

12. The network connection device of claim 9, wherein the specified destination comprises an address of another network connection device coupled to the network.

13. The network connection device of claim 9, wherein the packet processing circuitry comprises an egress queue for queuing egress packets, and wherein the DDC is configured to send the collected diagnostic data to the egress queue.

14. A method for diagnosing a network connection device, comprising: In network connection equipment, packets are received, buffered, processed, and transmitted over a network using packet processing circuitry; as well as Using diagnostic direct memory access (DMA) circuitry (DDC) in the network connection device: receiving (i) one or more diagnostic events and (ii) for each diagnostic event, a definition of a corresponding list of diagnostic data generated in the packet processing circuitry and associated with the diagnostic event; and In response to occurrence of a diagnostic event, in a diagnostic data buffer, diagnostic data corresponding to a list of diagnostic events are collected from the plurality of subunits of the packet processing circuit by connecting the plurality of subunits of the packet processing circuit to a diagnostic data bus of the diagnostic data buffer.

15. The method of claim 14, wherein collecting diagnostic data comprises collecting diagnostic data in a plurality of corresponding diagnostic data lists from the packet processing circuitry during corresponding time periods that at least partially overlap with each other in response to occurrences of a plurality of diagnostic events. The method of claim 14 , wherein collecting diagnostic data comprises collecting the diagnostic data in the list across a plurality of DDC instances.

17. The method of claim 16, wherein collecting diagnostic data comprises collecting the diagnostic data from at least two different portions of the packet processing circuitry via at least two corresponding ones of the DDC instances. 18 . The method of claim 16 , wherein collecting diagnostic data comprises collecting the diagnostic data related to a same diagnostic event through at least two of the DDC instances.

19. The method of claim 14, wherein the packet processing circuitry comprises a packet processing pipeline having a plurality of pipeline stages, and wherein collecting diagnostic data comprises collecting the diagnostic data from two or more of the pipeline stages.

20. The method of claim 14, wherein collecting diagnostic data comprises collecting the diagnostic data by reading one or more counter values ​​in the packet processing circuitry.

21. The method of claim 14, wherein collecting diagnostic data comprises receiving a diagnostic trigger, and collecting diagnostic data for the corresponding list from the packet processing circuit in response to the diagnostic trigger.

22. The method of claim 14, further comprising transmitting the collected diagnostic data to a designated destination.

23. The method of claim 21, wherein sending the diagnostic data comprises sending the list to a storage location in a memory.

24. The method of claim 21, wherein sending the collected diagnostic data comprises sending the collected diagnostic data to a processor of the network-connected device, or to a host hosting the network-connected device.

25. The method of claim 21, wherein sending the collected diagnostic data comprises sending the collected diagnostic data to an address of another network-connected device coupled to the network.

26. The method of claim 21, wherein the packet processing circuitry comprises an egress queue for queuing egress packets, and wherein sending the collected diagnostic data comprises sending the collected diagnostic data to the egress queue.

27. A method for diagnosing a network device, comprising: In network equipment, packets are received, processed, and transmitted over a network using packet processing circuitry; as well as Operating diagnostic circuitry in said network device: receiving (i) diagnostic events expected to occur in the network device, and (ii) for each diagnostic event, a definition of a corresponding list of diagnostic data to be collected from the packet processing circuitry for analysis; as well as When a diagnostic event occurs, in a diagnostic data buffer, the diagnostic data list corresponding to the diagnostic event is collected from the multiple sub-units of the packet processing circuit according to the definition by connecting the multiple sub-units of the packet processing circuit to the diagnostic data bus of the diagnostic data buffer.