Hardware diagnostics based on high frequency events

Abstract

This application relates to hardware diagnostics based on high frequency events. An apparatus includes an operational circuit and a hardware diagnostics circuit (HDC). The HDC is configured to receive definitions of a plurality of trigger rules each specifying a respective trigger event from a trigger data source in the operational circuit, receive definitions of (i) a pre-trigger record set selected from a plurality of diagnostic data sources in the operational circuit, and (ii) for each trigger rule, a respective post-trigger record set including a set of one or more diagnostic data sources, and during operation of the operational circuit, record diagnostic data sources in the pre-trigger record set, record the trigger data source and repeatedly evaluate the trigger rules, and in response to triggering of a given trigger event by a given trigger rule, begin recording diagnostic data sources in the post-trigger record set for the given trigger rule.

Description

Invention Field

[0001] This invention relates generally to electronic circuits, and more particularly to in-circuit debugging of network devices and other electronic circuits. Background of the Invention

[0002] Complex electronic systems (such as network-connected devices) typically include hardware and software that facilitate online testing and diagnostics.

[0003] For example, U.S. Patent 7,730,458 describes systems and methods that facilitate diagnostic support, including application and trace components instrumented according to a Built-in Diagnostics (BID) framework. The trace components can selectively omit application-associated trace points, use some of the application-associated trace points, and / or substantially use all of the application-associated trace points. For example, this system can facilitate the instrumentation of the managed data access stack to enhance application supportability.

[0004] In another example, U.S. Patent Application Publication 2008 / 0077835 describes an automated test apparatus (ATE) capable of receiving diagnostic information from a device under test. The ATE has a built-in self-test system (BIST) and a diagnostic information collector that temporarily stores diagnostic patterns output by the BIST and provides fault indications when a fault is detected in the device under test. The ATE includes a device interface connectable to the device under test, a processing system, and processing channels. Each processing channel is connected to the device interface and the processing system and includes a test channel, a fault indication channel, and a diagnostic information channel. The test channel is interoperable with the BIST to perform a series of tests on the device under test. The fault indication channel is connected to receive fault indications from the device interface. The diagnostic information channel is operable to receive, in response to a fault indication received via the fault indication channel, at least some diagnostic patterns temporarily stored in the device under test as diagnostic information from the device interface. Invention Overview

[0005] The embodiments of the present invention described herein provide an apparatus including an operating circuit and a hardware diagnostic circuit (HDC). The HDC is configured to: receive definitions of a plurality of trigger rules, wherein each trigger rule specifies the triggering of a corresponding trigger event based on one or more trigger data sources in the operating circuit; receive definitions of: (i) a pre-trigger logging set selected from a plurality of diagnostic data sources in the operating circuit, and (ii) a corresponding post-trigger logging set for each trigger rule, including a corresponding set of one or more diagnostic data sources; and during operation of the operating circuit, repeatedly logging diagnostic data sources in the pre-trigger logging set, repeatedly logging trigger data sources and repeatedly evaluating trigger rules, and in response to the triggering of a given trigger event by a given trigger rule, initiating the logging of diagnostic data sources in the post-trigger logging set of the given trigger rule.

[0006] In some embodiments, at least one post-triggered record set differs from the pre-triggered record set. In some embodiments, the HDC is configured to record diagnostic data sources in the post-triggered record set within a defined time interval, or to record diagnostic data sources in the post-triggered record set until a defined data size is reached. In example embodiments, the defined time interval or the defined data size is specified according to triggering rules.

[0007] In the disclosed embodiments, the HDC is configured to retain only a maximum defined number of the latest data from the diagnostic data source in the pre-triggered record set. In another embodiment, the HDC is configured to record images from the diagnostic data source that are temporally coherent with respect to each other. In yet another embodiment, the HDC is configured to record the diagnostic data source in memory and, in response to a dump command, output at least a portion of the recorded pre-triggered record set and post-triggered record set.

[0008] In the disclosed embodiments, at least one triggering rule specifies a condition that the triggering data source must meet within a defined time interval. In another embodiment, at least one triggering rule specifies a statistical condition that the triggering data source must meet.

[0009] In some embodiments, the operating circuitry is configured to process communication data packets, and one or more triggering rules relate to the performance of the operating circuitry in processing data packets. In some embodiments, the operating circuitry is configured to communicate via a bus, and one or more triggering rules relate to the performance of the operating circuitry in bus communication.

[0010] According to embodiments of the present invention, a method is also provided, the method comprising: receiving, in a hardware diagnostic circuit (HDC) coupled to an operating circuit, the definitions of a plurality of trigger rules, wherein each trigger rule specifies the triggering of a corresponding trigger event based on one or more trigger data sources in the operating circuit; and additionally receiving in the HDC the definitions of: (i) a pre-trigger record set selected from the plurality of diagnostic data sources in the operating circuit, and (ii) a corresponding post-trigger record set for each trigger rule, comprising a corresponding set of one or more diagnostic data sources. During operation of the operating circuit, using the HDC, diagnostic data sources are repeatedly recorded in the pre-trigger record set, trigger data sources are repeatedly recorded, and trigger rules are repeatedly evaluated. In response to the triggering of a given trigger event by a given trigger rule, recording of diagnostic data sources in the post-trigger record set of the given trigger rule begins.

[0011] The invention will be more fully understood from the following detailed description of embodiments thereof, taken in conjunction with the accompanying drawings, in which: Brief description of the attached diagram

[0012] Figure 1 This is a block diagram schematically illustrating the structure of a network device according to an embodiment of the present invention;

[0013] Figure 2A This is a schematic illustration of an embodiment of the invention. Figure 1 A block diagram of pre-triggered data logging in network devices;

[0014] Figure 2B This is a schematic illustration of an embodiment according to the present invention. Figure 1 A block diagram of post-triggered data logging in network devices;

[0015] Figure 3 This is an embodiment of the present invention. Figure 1 A block diagram of the hardware diagnostic circuit (HDC) in a network device; and

[0016] Figure 4 This is a flowchart schematically illustrating a method for hardware diagnostics according to an embodiment of the present invention. Detailed Implementation

[0017] Overview

[0018] Generally, digital systems, and especially network devices such as network processors, network interface controllers (NICs), host channel adapters (HCAs), switches, routers, gateways, and graphics processing units (GPUs), can contain many digital sub-units with complex interdependencies. When such systems fail, or if their performance degrades, finding the root cause can be challenging.

[0019] The present invention discloses a method and apparatus for efficient and high-speed diagnostics of digital systems. In the embodiments, a hardware diagnostic circuit (HDC) is embedded in the digital system (the portion of the digital system to which the HDC is coupled will be referred to as the operating circuit).

[0020] In this embodiment, the HDC includes a data recording buffer, a recording multiplexer, and a trigger evaluation circuit. The HDC is configured to receive i) a trigger evaluation rule, ii) a pre-trigger recording rule, and iii) a post-trigger recording rule. The HDC is configured to store pre-trigger data from the operation circuit in the data recording buffer according to the pre-trigger recording rule, monitor the trigger data source, and detect trigger conditions according to the trigger evaluation rule. After detecting a trigger event, the HDC stores the post-trigger data in the data recording buffer.

[0021] According to some embodiments, the HDC is also configured to receive a trigger buffer size limit. Upon detecting a trigger event, the HDC records the trigger data until the trigger buffer size is reached, and then stops. In other embodiments, there is a time limit for the HDC to record the trigger data after receiving it.

[0022] In an embodiment, the HDC is configured to send stored recorded data from the data recording buffer to the processor for analysis and diagnostics.

[0023] Finally, according to an embodiment, the HDC includes a coherent data sampler configured to record a coherent image of data in the operating circuitry.

[0024] The disclosed technology provides powerful and effective monitoring and debugging tools for network devices and other electronic circuits. For example, in some embodiments, HDC supports highly flexible definition of trigger rules, such as rules specifying conditions that trigger data sources must meet within defined time intervals and / or statistical conditions that trigger data sources must meet.

[0025] In a typical implementation, the pre-triggered record set is common to all possible triggers, while the post-triggered record set is trigger-specific, meaning it can differ from one trigger rule to another. Therefore, the HDC is able to record a wide variety of data sources before a trigger occurs, thus covering a broad range of data sources in the operational circuitry. After a trigger event occurs, the trigger-specific definition of the post-triggered record set allows the HDC to customize the data sources to be recorded based on the specific characteristics of each trigger. This feature provides considerable flexibility in defining rules and also effectively utilizes the limited memory size of the data recording buffer.

[0026] System Description

[0027] According to embodiments of the present invention, a network device may include hardware diagnostic circuitry programmed by a processor to coherently monitor diagnostic data sources in the operating circuitry, record pre-triggered diagnostic data, evaluate data-related triggering events, and record data based on the detected triggering event if such an event occurs. The HDC can then send the recorded data to the processor for analysis.

[0028] Figure 1 This is a block diagram schematically illustrating the structure of a network device 100 according to an embodiment of the present invention. In various embodiments, for example, the network device 100 may include a network adapter (e.g., a network interface controller (NIC), host channel adapter (HCA), or data processing unit (DPU – also referred to as a “smart NIC”), a network processor, a switch, a router, a gateway, a network-enabled graphics processing unit (GPU), or any other suitable type of network device.

[0029] The network device includes operating circuitry 102 coupled to the network and configured to perform network-related operations. A user (e.g., a service engineer) may wish to diagnose the operation of the network device by observing nodes within operating circuitry 102 (e.g., to name a few non-limiting possibilities, various queue fill metrics, packet drop counts, the number of concurrent data streams). The user communicates with the network device via processor 104 (in some embodiments, processor 104 is a dedicated diagnostic processor; in other embodiments, processor 104 is a shared processor, e.g., a processor controlling the operating circuitry); in still other embodiments, processor 104 may include multiple processors.

[0030] To perform diagnostics, processor 104 is configured to write a set of trigger evaluation rules (also referred to herein as “trigger rules”) into trigger evaluation circuitry 108 and a set of data recording rules into data recording rule register 110. Each trigger recording rule specifies the triggering of a corresponding trigger event based on one or more trigger data sources in operation circuitry 102.

[0031] In an embodiment, the trigger evaluation rule may specify one or more conditions that the trigger data source must meet within a defined time interval. In an embodiment, the trigger evaluation rule may include complex evaluations, such as triggering an event when the value of a first monitored data source is greater than a preset minimum value and the value of a second source is between two preset limits (other examples of complex trigger evaluation rules will be disclosed below). In the following description, we will use the terms "detect trigger event," "determine trigger event," and "trigger trigger event" interchangeably.

[0032] In some embodiments, the operating circuitry is configured to communicate via a bus (a non-limiting example is a high-speed peripheral component interconnect or PCIe; other suitable buses may be used in alternative embodiments), and the triggering rules relate to the performance of the operating circuitry's bus communication.

[0033] In some embodiments, the trigger evaluation circuit 108 may include one or more processors. The trigger evaluation circuit is configured to receive trigger evaluation rules from the processor 104, monitor corresponding trigger evaluation data sources from the operating circuit, and detect trigger events.

[0034] In some embodiments, data logging rules may include a pre-triggered record set and a post-triggered record set. The pre-triggered record set defines the data sources within the operating circuitry that the HDC should record until a trigger event is detected. The post-triggered record set defines the data sources that the HDC should record after the trigger event is determined. In some embodiments, there may be multiple post-triggered data sources, and the data logging rules define which data source should be recorded after the trigger event is detected for each trigger event. The pre-triggered record set is typically not trigger-specific.

[0035] HDC 106 also includes a data recording multiplexer 112 and a data recording buffer 114. The data recording multiplexer 112 is configured to respond to a subset of data recording sources in a data recording rule selection operation circuit, and the data recording buffer 114 is configured to store data selected by the recording multiplexer. In an embodiment, the data recording buffer 114 is a first-in, first-out (FIFO) memory configured to discard the oldest data when new data is stored (if the buffer is full). In an embodiment, the last-triggered data recording (e.g., in time) is finite; when the last-triggered data recording is complete, the processor can issue a dump command to read the data recording buffer 114 and send the recorded data to the user (e.g., using a waveform display program).

[0036] In short, according to Figure 1 In the example embodiment shown, the user programs the HDC using a set of trigger evaluation rules, a set of pre-triggered data sources (“pre-triggered record set”), and one or more sets of post-triggered data sources (“post-triggered record set”). The HDC continuously stores the latest pre-triggered data sources and simultaneously monitors the trigger data sources and searches for trigger events (according to the trigger evaluation rules). Once the HDC detects a trigger, it begins recording post-triggered data, which can be determined based on the detected trigger event. The buffer, including pre-triggered and post-triggered data, can be read and analyzed by the processor (and the user).

[0037] As will be recognized, examples are cited. Figure 1The structure of the network device 100, including HDC 106, is shown and described above. Various suitable structures can be used in alternative embodiments. For example, in some embodiments, a user can communicate with the HDC via a network and operating circuitry instead of via processor 104. In this embodiment, there is no data recording buffer 114; instead, data is sent to the processor via a high-speed bus.

[0038] Control the size of the pre-buffer and post-buffer

[0039] In some embodiments, the processor also sends a post-trigger recording duration parameter to the HDC. Once a trigger event is detected, the HDC will fill the data recording buffer 114 with post-trigger data samples for a period equal to the recording duration parameter (also referred to as the recording time interval), and then stop. The processor will then read the data recording buffer 114, which receives both pre-trigger data records and post-trigger data records. In one embodiment, the post-trigger duration may be replaced by the buffer fill size; in other embodiments, the HDC may be configured to stop post-trigger data recording when the post-trigger data occupies a preset percentage of the data recording buffer size.

[0040] Figure 2A This is a schematic block diagram illustrating pre-triggered data recording according to an embodiment of the present invention. Figure 2A The example embodiment shown includes a data recording multiplexer 112 (reference 112). Figure 1 (As described herein) includes a pre-trigger selector 204 and a post-trigger selector 206, wherein the pre-trigger selector 204 is configured to select from (the pre-trigger data recording rules respectively) Figure 1 The operation circuit 102 selects a data source from a set of pre-triggered data sources, and the post-triggered selector 206 is configured to select a data source for the post-triggered data recording rule and the selected trigger event, respectively.

[0041] The data logging multiplexer 112 also includes a switching switch 208 configured to output a pre-triggered data source from the pre-triggered selector 204 or a post-triggered data source from the post-triggered selector 206. Figure 2A The schematic diagram illustrates the triggering of evaluation circuit 108 ( Figure 1 No triggering event has been detected and the toggle switch 208 outputs data records when the pre-triggering data source selected by the pre-trigger selector 204 is selected.

[0042] Data output from data recording multiplexer 112 is output to data recording buffer 114. In an embodiment, data recording buffer 114 includes a first-in, first-out (FIFO) memory; when the buffer's storage capacity is exhausted, the oldest data is "flushed," and instead, new data is written (in effect, new data overwrites the oldest data). In some embodiments, such as when the data recording buffer is part of shared memory, the data recording buffer is configured to store a defined amount of the latest data.

[0043] Figure 2B This is a schematic diagram illustrating a post-triggered data recording according to an embodiment of the present invention. Figure 2B Similar to Figure 2A In addition to the following, namely: in the trigger evaluation circuit 108 ( Figure 1 After a trigger event is detected, the switch 208 outputs the trigger event selected by the trigger selector 206 based on the trigger data recording rules and the detected trigger event.

[0044] Data recording buffer 114 now stores post-triggered data, which replaces the oldest pre-triggered data (except for some pre-triggered data). In some embodiments, post-triggered data recording will stop after a predefined time interval; in embodiments, different time intervals can be predefined for different trigger events. In other embodiments, post-triggered data recording stops when the post-triggered data occupies a preset percentage (e.g., 75%) of the buffer size.

[0045] When the subsequent data recording is completed, processor 104 ( Figure 1 It can read all or part of the contents of the data record buffer 114.

[0046] As will be recognized, examples are cited. Figure 2A , Figure 2B The structure of pre-triggered data recording and post-triggered data recording is shown and described above. Various suitable structures can be used in alternative embodiments. For example, in some embodiments, the data recording buffer 114 is embedded in the processor 104 (… Figure 1 In one embodiment, processor 104 may include dedicated memory for storing recorded data; in another embodiment, processor 104 stores recorded data in shared memory (e.g., in processor main memory). In yet another embodiment, data recording buffer 114 is distributed between HDC 106 and processor 104.

[0047] coherent data sampling

[0048] In embodiments, the operating circuitry can be complex and includes numerous interdependent data recording sources. For coherent analysis to be performed, data source recording must be synchronized across all data sources to maintain coherence relative to each other. In this context, the term "coherent" means that each entry in the data recording buffer is associated with an image of the operating circuitry, where different recording sources are sampled within the same clock cycle.

[0049] In some embodiments, the HDC is configured to correct for delays in interdependent recording sources within the operating circuitry. For example, the next entry written to memory may set a buffer full flag in the next clock cycle; the HDC may be configured to delay the recording of data entries written to the buffer by one clock cycle, so that they will coincide with the recording of the buffer status (including the buffer full signal).

[0050] Figure 3 This is a block diagram of HDC 106A according to an embodiment of the present invention. HDC 106A is similar to HDC 106 ( Figure 1 However, it also includes a coherent sampler 302, which is configured to operate the circuitry 102 ( Figure 1 The coherent image is sampled. The coherent sampler 302 may include a delay stage and an inter-domain clock synchronizer.

[0051] HDC 102A is coupled to something similar to processor 104 ( Figure 1 The processor is 104A; however, according to Figure 3 In the example embodiment shown, processor 104A includes an external data recording buffer 304. In this embodiment, the external data recording buffer 304 is configured to record excess data that the data recording buffer in the HDC cannot store. For example, in this embodiment, the HDC data recording buffer 114 may include 16 megabytes, while the external data recording buffer 304 may include 256 megabytes.

[0052] Figure 4 This is a schematic flowchart illustrating a method 400 for hardware diagnostics according to an embodiment of the present invention. The flowchart consists of a hardware diagnostic circuit 106 (…). Figure 1 The hardware diagnostic circuit 106 communicates with the processor 104 (and typically communicates with the user through the processor).

[0053] The flowchart begins with step 402, where the HDC obtains the trigger evaluation rule from the processor (e.g., Figure 1 The processor 104 receives the trigger evaluation rule. As described above (refer to...) Figure 1 The triggering evaluation rules can include complex evaluations (including relationships between various data points and calculations performed on the data).

[0054] Next, in step 404 of obtaining the pre-trigger recording rule, the HDC receives the operation circuitry (e.g., from the processor) from the processor. Figure 1 The HDC should record the list of data sources in the operation circuit 102) before the trigger event; then, in the step 406 of obtaining the post-trigger recording rule, the HDC receives from the processor a recording rule regarding the data to be recorded after the trigger event is detected. In an embodiment, the post-trigger rule may include a list of data sources to be recorded for each detected trigger event.

[0055] After step 406, the HDC proceeds to the Acquire-Start-Instruction step 408 and waits for an instruction to begin diagnostics (typically from the user via the processor). Upon receiving the start instruction, the HDC proceeds to the Continuous Pre-Triggering step 410, where the HDC repeatedly records pre-triggering data according to pre-triggering data recording rules and simultaneously repeatedly evaluates the triggering data source to detect triggering events.

[0056] When HDC detects a trigger event in step 410, HDC proceeds to the continuous triggering step 412, where HDC records the triggering data selected according to the triggering record rules and the detected trigger event. When the preset triggering record size (e.g., data recording buffer 114) has been reached... Figure 1 When 75% of the data has been stored and the data is triggered, the HDC enters the send completion step 414 and sends a completion indication to the processor (usually sent to the user via the processor). Then, the HDC enters the send buffer step 416, where, upon receiving a dump command from the processor, the HDC sends the contents of the data recording buffer (or a portion thereof) to the processor. After step 416, the flowchart ends.

[0057] For example, those who are familiar with it are cited through examples. Figure 4 The flowchart of method 400, shown above, is illustrated. Other suitable flowcharts may be used in alternative embodiments. In some embodiments, for example, step 408 is omitted, and the HDC begins data recording immediately upon receiving the rule. In an embodiment, the HDC sends all data along with a completion indication to the processor. In an embodiment, the HDC receives a new set of rules in a pipelined manner while sending data for records corresponding to the previous set of rules.

[0058] Example Use Case (NIC)

[0059] In some embodiments, diagnostics relate to the packet processing performance of the network device, while triggering rules relate to the packet processing performance of the operating circuitry.

[0060] According to embodiments of the present invention, this section discloses typical use cases in the performance diagnostics of a network interface controller (NIC).

[0061] In the first example, an unusually high packet loss rate was observed, and a diagnostic session was initiated.

[0062] You can set rules to trigger the evaluation, for example:

[0063] 1. Determine the trigger event when the number of dropped packets in a given port and / or a given receive buffer exceeds a preset threshold during a preset time period.

[0064] 2. Determine the trigger event when the number of dropped packets in a given port and / or a given receive buffer exceeds a preset percentage of the ingress port packet rate during a preset time period.

[0065] 3. The event is triggered only when the number of packets dropped in a given port and / or a given receive buffer during a preset time period exceeds the number of packets dropped in the previous preset time period, but the number of packets dropped in the previous preset time period is greater than a preset threshold.

[0066] In the second example, backpressure from the host or high latency in host NIC access was observed (assuming the host communicates with the NIC via the Peripheral Component Interconnect Fast (PCIe) bus). Triggering the evaluation data source can be set to include:

[0067] 1. Flow control (FC) credit from the root union (RC) that has published data requests.

[0068] 2. FC credits from RCs that have not published request data.

[0069] 3. FC credits from RCs that have not published request headers.

[0070] 4. PCIe tag.

[0071] You can set rules to trigger the evaluation, for example:

[0072] 1. Determine the trigger event when the credit of any or all trigger data sources (see above) is zero during a preset time period.

[0073] 2. A trigger event is determined when the number of zero-credit occurrences in any or all trigger data sources during a given time period exceeds the number (or percentage) of zero-credit occurrences in the previous preset time period.

[0074] As will be appreciated, the structure of the network device 100 and HDC 106 described above, as well as the method of flowchart 400, are referenced by way of example. The network device, HDC, and method according to the disclosed technology are not limited to the description above. In alternative embodiments, for example, the HDC may be distributed within the operating circuitry; the HDC data recording buffer 114 may be distributed, for example, near the recording data source. In some embodiments, trigger events may be chained; for example, the HDC may be configured to detect a first trigger event and then detect a second trigger event (sometimes more); data to be recorded may be preset before the first trigger event, between the first and second trigger events, and after the second trigger event.

[0075] Processor 104 typically includes a general-purpose processor that is programmed with software to perform the functions described herein. The software can be downloaded to the processor electronically via a network, or alternatively or additionally, the software can be provided and / or stored on a non-transitory tangible medium (such as magnetic storage, optical storage, or electronic storage).

[0076] The configuration of network device 100 including HDC 106 and the method of flowchart 400 are shown as example configurations and methods for clarity of concept only. Any other suitable configurations and flowcharts may be used in alternative embodiments.

[0077] Elements of the HDC 106 can be implemented using suitable hardware, such as in one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs), using software, using hardware, or using a combination of hardware and software elements.

[0078] Although the embodiments described herein are primarily for network device diagnostics, the methods and apparatus described herein can also be used in other applications, such as debugging and diagnostics of any digital device. In one embodiment, the HDC is embedded in a network switch having multiple ingress and egress ports, and the port for debugging is selected based on triggers and data sources from various ports (from the ingress and egress ports).

[0079] Therefore, it will be understood that the embodiments described above are by way of example, and the invention is not limited to what has been specifically shown and described above. Rather, the scope of the invention includes combinations and sub-combinations of the various features described above, as well as variations and modifications of the invention that would arise to those skilled in the art upon reading the foregoing description and that are not disclosed in the prior art. Documents incorporated herein by reference are considered part of this application, and the definitions in this specification should be considered only, unless any terms are defined in these incorporated documents in a manner that conflicts to some extent with the definitions expressly or implicitly made herein.

Claims

1. An apparatus comprising: Operating circuit; and Hardware diagnostic circuit (HDC), the hardware diagnostic circuit being configured to: The system receives the definitions of multiple trigger rules, wherein each trigger rule specifies the triggering of a corresponding trigger event based on one or more trigger data sources in the operating circuit. The following definitions are received: (i) a pre-triggered record set selected from a plurality of diagnostic data sources in the operating circuit, and (ii) a corresponding post-triggered record set for each triggering rule, comprising one or more of the corresponding sets of the diagnostic data sources; and During the operation of the operating circuit, the diagnostic data source is repeatedly recorded in the pre-trigger record set, the trigger data source is repeatedly recorded and the trigger rule is repeatedly evaluated, and in response to the triggering of a given trigger event by the given trigger rule, the recording of the diagnostic data source in the post-trigger record set of the given trigger rule begins.

2. The apparatus according to claim 1, wherein, At least one set of post-triggered records differs from the set of pre-triggered records.

3. The apparatus according to claim 1 or 2, wherein, The HDC is configured to record the diagnostic data source in the post-triggered record set at defined time intervals, or to record the diagnostic data source in the post-triggered record set until a defined data size is reached.

4. The apparatus according to claim 3, wherein, The defined time interval or defined data size is specified according to the triggering rules.

5. The apparatus according to claim 1 or 2, wherein, The HDC is configured to retain only a maximum defined number of the latest data from the diagnostic data source in the pre-triggered record set.

6. The apparatus according to claim 1 or 2, wherein, The HDC is configured to record images of the diagnostic data source that are temporally coherent with respect to each other.

7. The apparatus according to claim 1 or 2, wherein, The HDC is configured to record the diagnostic data source in memory and, in response to a dump command, output at least a portion of the recorded pre-trigger record set and post-trigger record set.

8. The apparatus according to claim 1 or 2, wherein, At least one of the triggering rules specifies the conditions that the triggering data source must meet within a defined time interval.

9. The apparatus according to claim 1 or 2, wherein, At least one of the triggering rules specifies a statistical condition that the triggering data source must meet.

10. The apparatus according to claim 1 or 2, wherein, The operating circuit is configured to process communication data packets, and one or more of the triggering rules are related to the performance of the operating circuit in processing data packets.

11. The apparatus according to claim 1 or 2, wherein, The operating circuit is configured to communicate via a bus, and one or more of the triggering rules are related to the performance of the bus communication of the operating circuit.

12. A method comprising: In the hardware diagnostic circuit (HDC) coupled to the operating circuit, the definitions of multiple trigger rules are received, wherein each trigger rule specifies the triggering of a corresponding trigger event based on one or more trigger data sources in the operating circuit; The HDC receives the following definitions: (i) a pre-triggered record set selected from a plurality of diagnostic data sources in the operating circuitry, and (ii) a corresponding post-triggered record set for each triggering rule, comprising one or more of the corresponding sets of the diagnostic data sources; and During the operation of the operating circuit, the diagnostic data source is repeatedly recorded in the pre-trigger record set using the HDC, the trigger data source is repeatedly recorded and the trigger rule is repeatedly evaluated, and in response to the triggering of a given trigger event by the given trigger rule, the recording of the diagnostic data source in the post-trigger record set of the given trigger rule is initiated.

13. The method according to claim 12, wherein, At least one set of post-triggered records differs from the set of pre-triggered records.

14. The method according to claim 12 or 13, wherein, The diagnostic data source is recorded in the post-triggered record set within a defined time interval, or the diagnostic data source is recorded in the post-triggered record set until a defined data size is reached.

15. The method according to claim 14, wherein, The defined time interval or defined data size is specified according to the triggering rules.

16. The method according to claim 12 or 13, wherein, The diagnostic data source is recorded by retaining only a maximum defined number of the latest data from the diagnostic data source in the pre-triggered record set.

17. The method according to claim 12 or 13, wherein, Recording the diagnostic data source includes recording images from the diagnostic data source that are temporally coherent with respect to each other.

18. The method according to claim 12 or 13, wherein, Recording the diagnostic data source includes recording the diagnostic data source in memory and, in response to a dump command, outputting at least a portion of the recorded pre-trigger record set and post-trigger record set.

19. The method according to claim 12 or 13, wherein, At least one of the triggering rules specifies the conditions that the triggering data source must meet within a defined time interval.

20. The method according to claim 12 or 13, wherein, At least one of the triggering rules specifies a statistical condition that the triggering data source must meet.

21. The method according to claim 12 or 13, wherein, One or more of the triggering rules are related to the performance of the operating circuit in processing communication data packets.

22. The method according to claim 12 or 13, wherein, One or more of the triggering rules are related to the performance of the operating circuit in communicating via the bus.

Images