Coprocessor-based logging for time travel debugging

Abstract

The coprocessor stores at least a portion of an execution trace based on code execution at the main processor. The coprocessor includes control logic that configures the coprocessor to listen to a bus interconnecting the main processor and the coprocessor and receive (multiple) cache coherence protocol (CCP) messages from the bus (i.e., (multiple) CCP messages sent by the main processor on the bus based on the main processor having consumed data of the memory unit). Based on receiving the (multiple) CCP messages, the coprocessor initiates storage of the consumed data of the memory unit into the execution trace.

Description

Technical Field

[0001] The present disclosure relates to coprocessor-based logging for time travel debugging. Background Art

[0002] Tracking and correcting unexpected software behavior in software code is a core activity during software development, which is often referred to as "debugging" the code. Unexpected software behavior can include many things, such as execution crashes, runtime exceptions, slow execution performance, wrong data results, data corruption, etc. Unexpected software behavior may be triggered by a wide variety of factors such as data input, user input, race conditions (for example, when accessing shared resources), etc. Given the diversity of triggers, unexpected software behavior may be rare and seemingly random, and extremely difficult to reproduce. As such, it can be very time-consuming and difficult for developers to identify a given unexpected software behavior. Once the unexpected software behavior is identified, it can be time-consuming and laborious to determine its (multiple) root causes again.

[0003] One approach that developers use to debug code is to use a "real-time" debugger. Generally speaking, a real-time debugger is attached to the execution of a real-time process and enables the developer to monitor and direct the forward execution of the process. For example, a real-time debugger may enable the developer to set breakpoints that pause the execution of the program when the program reaches a specific instruction, set watchpoints that pause the execution of the program when the program accesses a specific memory address, single-step through instructions through lines of code as the code executes, etc.

[0004] An emerging form of diagnostic tool enables "historical" debugging (also known as "time travel" or "reverse" debugging), in which the execution of at least a portion of a program's thread(s) is recorded / traced to one or more trace files (i.e., traces). Using some tracing techniques, a trace may contain very high fidelity "bit-accurate" historical trace data, which enables virtual "replay" of the recorded portion(s) of the traced thread(s) with extremely high fidelity (even down to the granularity of individual instructions (e.g., machine code instructions, intermediate language code instructions, etc.). Thus, using bit-accurate trace data, a "time travel" debugger enables developers to monitor and direct not only forward simulation of the traced code (e.g., via breakpoints, watchpoints, single-stepping instructions, etc.), but also detect and direct reverse simulation of the traced code (e.g., via reverse breakpoints, reverse watchpoints, reverse single-stepping instructions, etc.). Thus, developers can monitor and direct the execution of any portion of a program prior to tracing.

[0005] As mentioned, historical debugging techniques record the execution of a program with extremely high fidelity (potentially bit-accurate). Due to the extremely high number of instructions that modern processors can execute per second, this generates a large amount of trace data. Recording this data can have a significant performance impact on the program being traced. Summary of the invention

[0006] At least some embodiments described herein mitigate some of the performance impacts on the performance impact of the program being traced by using a coprocessor to store at least a portion of an execution trace based on code execution at a main processor. Specifically, during code execution, the main processor sends messages such as cache consistency protocol (CCP) messages onto a bus. These messages are related to the main processor's consumption of data and may include cache inflows. The coprocessor receives these messages from the bus and, based on these messages, records at least a portion of the execution trace on behalf of the main processor.

[0007] In some embodiments, a computer system includes: a main processor associated with a memory cache; a coprocessor; and a bus interconnecting the main processor and the coprocessor. The computer system is configured to store at least a portion of an execution trace using the coprocessor. Storing at least a portion of the execution trace may include: the main processor executing one or more executable instructions of an executable entity. Based on the main processor executing the one or more executable instructions, the main processor sends one or more CCP messages on the bus. The one or more CCP messages correspond to a memory unit. Moreover, based on the main processor executing the one or more executable instructions, the main processor consumes data of the memory unit. The coprocessor receives the one or more CCP messages from the bus, and based on the coprocessor receiving the one or more CCP messages, the coprocessor initiates storing the consumed data of the memory unit into the execution trace.

[0008] In other embodiments, a method is implemented in a computer system including a main processor associated with a memory cache, a coprocessor, and a bus interconnecting the main processor and the coprocessor. The method uses the coprocessor to store at least a portion of an execution trace. The method includes: executing one or more executable instructions of an executable entity at the main processor. The method also includes: based on the main processor executing the one or more executable instructions, sending one or more CCP messages on the bus by the main processor, the one or more CCP messages corresponding to the memory unit; and consuming data of the memory unit by the main processor. The method also includes: receiving one or more CCP messages from the bus by the coprocessor; and based on the coprocessor receiving the one or more CCP messages, initiating, by the coprocessor, storing the data of the consumed memory unit into the execution trace.

[0009] In other embodiments, the coprocessor stores at least a portion of an execution trace based on code execution at the main processor. The coprocessor includes control logic that configures the coprocessor to listen to a bus interconnecting the main processor and the coprocessor and receive one or more CCP messages from the bus. Based on the main processor consuming data of the memory unit, the one or more CCP messages are sent on the bus by the main processor. Based on receiving the one or more CCP messages, the coprocessor initiates storing the consumed data of the memory unit into the execution trace.

[0010] This Summary is provided to introduce a selection of concepts in a simplified form that are further described in the Detailed Description below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to describe the manner in which the above and other advantages and features of the present invention can be obtained, a more particular description of the invention briefly described above will be provided by reference to specific embodiments of the invention, which are illustrated in the accompanying drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:

[0012] Figure 1 An example computing environment that facilitates using a coprocessor to store at least a portion of an execution trace is illustrated;

[0013] Figure 2 An example computing environment including multiple layers of cache is illustrated;

[0014] Figure 3 illustrates an example of a time travel debug trace; and

[0015] Figure 4 A flow diagram of an example method of using a coprocessor to store at least a portion of an execution trace is illustrated. DETAILED DESCRIPTION

[0016] At least some embodiments described herein mitigate some of the performance impacts on the performance impact of the program being traced by using a coprocessor to store at least a portion of an execution trace based on code execution at a main processor. Specifically, during code execution, the main processor sends messages such as cache consistency protocol (CCP) messages onto a bus. These messages are related to the main processor's consumption of data and may include cache inflows. The coprocessor receives these messages from the bus and, based on these messages, records at least a portion of the execution trace on behalf of the main processor.

[0017] In order to achieve the above, Figure 1 An example computing environment 100 is illustrated that facilitates the use of a coprocessor to store at least a portion of an execution trace. As depicted, the computing environment 100 may include or utilize a special purpose computer system or a general purpose computer system 101 that may include computer hardware such as, for example, one or more main processors 102 that are generally responsible for executing application code (e.g., of application 112), a system memory 103 that is generally responsible for temporary data storage, one or more computer-readable storage media 104 for storing application code and / or data, at least one coprocessor that is generally responsible for recording at least a portion of executing application code (e.g., of application 112) at the one or more main processors 102, and one or more external communication channels 114 (e.g., a network interface card or even just one or more wires / pins). Each of the main processors 102, the system memory 103, and the coprocessor 105 are communicatively coupled using one or more communication buses 106.

[0018] like Figure 1 As shown, each main processor 102 (hereinafter also more succinctly referred to as processor 102) may include (among other things) one or more processing units 107 (e.g., processor cores) and one or more caches 108. Each processing unit 107 loads and executes machine code instructions (e.g., of application 112) via cache 108. During execution of these machine code instructions at one or more execution units 107b, these instructions may use internal processor registers 107a as temporary storage locations and may read and write to various locations in system memory 103 via cache 108.

[0019] In an embodiment, the operation of the processor 102 is orchestrated at least in part by the control logic 109. As indicated by the ellipsis within the control logic 109, the control logic 109 can do many things. However, for simplicity of illustration, the control logic 109 only shows the logic components (i.e., CCP logic 109a and logging logic 109b) used herein to describe how the processor 102 can help facilitate the use of the coprocessor 105 to store at least a portion of the execution trace. As will be appreciated by one of ordinary skill in the art, the control logic 109 may include digital logic (e.g., a fixed arrangement of transistors, one or more programmed field programmable gate arrays, etc.) and / or stored executable instructions (e.g., processor microcode) that may be executed by one or more components of the processor 102.

[0020] Returning to cache 108, cache 108 temporarily caches portions of data stored in system memory 103; for example, cache 108 may include a "code" portion that caches portions of system memory 103 that store application code (e.g., of application 112) and a "data" portion that caches portions of system memory 103 that store application runtime data. If processing unit 107 requires data (e.g., code or application runtime data) that is not already stored in cache 108, processing unit 107 may initiate a "cache miss," resulting in the required data being retrieved from a backing storage device (e.g., system memory 103, another cache, etc.). For example, in Figure 1 , the backing store for cache 108 may be system memory 103, so cache misses may be serviced by system memory 103. At times, data may be "evicted" from cache 108 back to its backing store (e.g., system memory 103), such as when the data is no longer needed in cache 108 or when more urgent data is needed in cache.

[0021] To further explain these concepts, Figure 2 An example environment 200 illustrating multiple layers of caching is shown. Figure 2 In the embodiment, there are two processors 201a and 201b (eg, each processor is connected to Figure 1 ) and system memory 202 (e.g., corresponding to different processors 102 of Figure 1 In the example environment 200, each processor 201 includes four physical processing units (i.e., units A1 to A4 of processor 201a and units B1 to B4 of processor 201b, which may correspond to system memory 103). Figure 1The example environment 200 also includes a three-level cache hierarchy within each processor 201. The environment 200 is only an example cache layout and is not limited to the cache hierarchy in which the embodiments herein may operate. In the environment 200, each processing unit is associated with its own dedicated L1 cache (e.g., L1 cache "L1-A1" in the processor 201a of unit A1, L1 cache "L1-A2" in the processor 201a of unit A2, etc.). Each processor 201 also includes two L2 caches (e.g., L2 cache "L2-A1" in the processor 201a used as a backup storage device for L1 caches L1-A1 and L1-A2, L2 cache "L1-A2" in the processor 201a used as a backup storage device for L1 caches L1-A3 and L1-A4, etc.). Finally, each processor 201 also includes a single L3 cache (e.g., L3 cache "L3-A" in processor 201a that serves as a backing store for L2 caches L2-A1 and L2-A2 and L3 cache "L3-B" in processor 201b that serves as a backing store for L2 caches L2-B1 and L2-B2). As shown, system memory 202 serves as a backing store for L3 caches L3-A and L3-B. In this arrangement and depending on the cache implementation, a cache miss in an L1 cache may be serviced by its corresponding L2 cache, its corresponding L3 cache, and / or system memory 202; a cache miss in an L2 cache may be serviced by its corresponding L3 cache and / or system memory 202; and a cache miss in an L3 cache may be serviced by system memory 202.

[0022] like Figure 2 As shown, when multiple cache layers are used, the processing unit(s) may interact directly with the nearest layer (e.g., L1), although this is not always the case. In most cases, data flows between the layers (e.g., the L3 cache interacts with the system memory 103 and provides data to the L2 cache, which in turn provides data to the L1 cache). When a processing unit performs a write, the caches coordinate to ensure that the cache that affects the data shared between the processing unit(s) 102a no longer owns it. This coordination is performed using a cache coherence protocol (CCP).

[0023] Thus, the caches in environment 200 may be considered "shared" caches. For example, each L2 and L3 cache serves multiple processing units within a given processor 201 and is therefore shared by the processing units. Because the individual L1 caches may be coordinated with each other (i.e., via CCP) to ensure consistency (i.e., so that each cached memory location is viewed consistently across all L1 caches), the L1 caches within a given processor 201 may be considered shared overall even though each L1 cache corresponds to a single processing unit. The L2 caches within each processor 201 may similarly be coordinated via CCP. Additionally, if processor 201 supports hyperthreading, each individual L1 cache may be considered shared by two or more logical processing units and is therefore "shared" even at the individual level.

[0024] As mentioned, when multiple caches 108 are present (e.g. Figure 2 108 ), the processor 102 operates the cache 108 according to one or more CCPs. In general, the CCP defines how to maintain consistency between the caches when the processing units 107 of the one or more processors 102 read and write data in the caches and how to ensure that the processing units always read valid data from a given location in the cache. The CCP is usually related to and enables the memory model defined by the instruction set architecture (ISA) of the processor. Popular ISA examples include the architectures of the x86 and x86_64 families from INTEL and the ARM architecture from ARM HOLDINGS.

[0025] Examples of common CCPs include the MSI protocol (i.e., Modified, Shared, and Invalid), the MESI protocol (i.e., Modified, Exclusive, Shared, and Invalid), and the MOESI protocol (i.e., Modified, Exclusive, Shared, and Invalid). Each of these protocols defines the state of individual locations (e.g., lines) in the shared cache 108. A "Modified" cache location contains data that has been modified in the cache 108 and is therefore inconsistent with the corresponding data in a backing store (e.g., system memory 103 or another cache). When a location with a "Modified" state is evicted from the cache 108, the common CCP requires the cache to ensure that its data is written back to the backing store or another cache takes over this responsibility. A "Shared" cache location contains data that has not been modified from the data in the backing store, exists in a read-only state, and is shared by the (multiple) processing units 107. The cache 108 can evict this data without writing it to the backing store. An "Invalid" cache location does not contain valid data, can be considered empty, and can be used to store data from a cache miss. An "exclusive" cache location contains data that matches a backing store and is used only by a single processing unit 107. The "exclusive" cache location can be changed to a "shared" state at any time (i.e., in response to a read request) or to a "modified" state when the "exclusive" cache location is written. An "own" cache location is shared by two or more processing units 107, but one of the processing units has the exclusive right to make changes to it. When the process makes a change, the notified processing unit notifies the other processing units because the notified processing unit may need to be invalidated or updated based on the CCP implementation.

[0026] Reference again Figure 1 , storage medium 104 may store computer executable instructions and / or data structures representing executable software components, such as application 112; accordingly, during execution of the software at processor(s) 102, one or more portions of these computer executable instructions and / or data structures may be loaded into system memory 103 (i.e., shown as application 112'). Additionally, storage medium 104 may store one or more execution traces 113, such as a trace representing execution of application 112 at processor 102. Storage medium 104 is shown broken to indicate that it may be optional. For example, application 112 may alternatively be received via external communication channel 114, and / or trace 113 may alternatively be transmitted via external communication channel 114.

[0027] During execution of application 112, processor 102 and coprocessor 105 cooperate to record an execution trace representing one or more portions of the execution of the application. The execution trace may be at least temporarily stored in system memory 103 (i.e., trace 113'). From there it may be persisted to storage medium 104 (i.e., trace(s) 113) and / or sent to another computer system via communication channel 114. Alternatively, the execution trace may be sent to another computer system directly through communication channel 114 bypassing system memory 103 entirely.

[0028] In an embodiment, coprocessor 105 records at least a portion of the execution trace based on CCP messages sent by processor 102 via bus 106. Thus, Figure 1 The bus 106 in represents any number (i.e., one or more) of buses capable of communicating at least one subset of CCP messages between the processor 102 and the coprocessor 105. For example, Figure 1 The bus 106 in may implement at least a subset of the Advanced Microcontroller Bus Architecture (AMBA) bus architecture standard, which may support a variety of bus technologies / types, such as Coherent Hub Interface (CHI), Advanced eXtensible Interface (AXI), AXI Coherence Extensions (ACE), Advanced High-Performance Bus (AHB), Advanced Peripheral Bus (APB), and / or Advanced Trace Bus (ATB). However, as will be appreciated by those of ordinary skill in the art, the embodiments herein may also operate within other bus architectures (both existing and yet to be developed).

[0029] In an embodiment, the processor 102 and the coprocessor 105 cooperate to record an execution trace into a trace 113 / 113' that represents a high-fidelity record of a previous execution of one or more threads of an application 112 at the processor 102, and wherein the trace data enables later simulation (i.e., "replay") of any portion of the traced (multiple) threads (i.e., even down to the granularity of individual machine code instructions in the ISA of the processor 102). As used herein, a trace 113 is a trace that includes sufficient data to enable replay of the code of an application 112 previously executed at the processor 102 via simulation so that the emulator can simulate the execution of the code in substantially the same manner as it was previously executed. A variety of approaches may be used to record and store such a trace 113.

[0030] In an embodiment, the tracing approach used by coprocessor 105 and processor 102 is based on the recognition that processor instructions generally fall into one of three categories: (1) instructions identified as "non-deterministic" because they do not produce predictable outputs because their outputs are not completely determined by data in general purpose registers (e.g., register 107a) or caches (e.g., cache 108); (2) deterministic instructions whose inputs do not depend on memory values ​​(e.g., they depend only on processor register values ​​or values ​​defined in the code itself); and (3) deterministic instructions whose inputs depend on reading values ​​from memory. Thus, in some embodiments, storing sufficient state data to reproduce the execution of instructions can be achieved by solving the following problems: (1) how to record non-deterministic instructions that produce outputs that are not completely determined by their inputs; (2) how to reproduce the values ​​of the input registers of instructions that depend on registers; and (3) how to reproduce the values ​​of the input memory of instructions that depend on memory reads.

[0031] To address the challenge of reproducing the execution of non-deterministic instructions that produce outputs that are not completely determined by their inputs, embodiments may record the consequences of the execution of such instructions (e.g., the outputs of the instructions) into the trace 113. As used herein, "non-deterministic" instructions may include some less common instructions that (i) produce non-deterministic outputs each time they are executed (e.g., RDTSC on an INTEL processor, which writes the number of processor cycles since the last processor reset into a register); (ii) may produce deterministic outputs, but depend on inputs that are not tracked during trace recording (e.g., debug registers, timers, etc.); and / or (iii) produce processor-specific information (e.g., CPUID on an INTEL processor, which writes processor-specific data into registers). Storing the consequences of the execution of such instructions may include, for example, storing register values ​​and / or memory values ​​that are changed by the execution of the instructions. In some architectures, such as the architecture from INTEL, processor features such as those found in Virtual Machine Extensions (VMX) may be used to capture instructions to record their consequences into the trace 113.

[0032] Solving the challenge of how to reproduce the values ​​of input registers for deterministic instructions (e.g., whose inputs depend only on processor register values) is straightforward, since they are the output of executing the previous instruction(s). Thus, trace 113 can represent the execution of an entire series of processor instructions by storing data that can be used to reproduce the register values ​​at the start of the series.

[0033] To address how to reproduce the values ​​of input memory for deterministic instructions whose inputs depend on memory values, an embodiment may record the memory values ​​consumed by these instructions (i.e., reads) into trace 113, regardless of how the values ​​read by the instructions are written to memory. In other words, trace 113 may represent the values ​​read by memory, but not necessarily the values ​​written to memory. For example, although the value may be written to memory by the current thread, another thread (including the kernel, for example, as part of processing an interrupt), or a hardware device, it is only the values ​​read by the thread's instructions that are needed to fully replay the instructions of the thread that performed the read. This is because it is the values ​​read by the thread (not necessarily all the values ​​written to memory) that determine how the thread executes.

[0034] A hardware-based approach to recording these readings is based on the recognition that a processor (e.g., processor 102) forms a semi-closed system or a quasi-closed system. For example, once a portion of a thread's data (i.e., code data and runtime application data) is loaded into cache 108, processor 102 can operate on its own as a semi-closed system or a quasi-closed system for time bursts without any input. Specifically, once cache 108 is loaded with data, one or more of processing units 107 use the runtime data stored in the data portion of cache 108 and use registers 107a to execute instructions from the code portion of cache 108. When a processing unit 107 needs some information to flow in (e.g., because it is executing, will execute, or may execute instructions that access code or runtime data that is not in cache 108), a "cache miss" occurs and the information is brought into cache 108 from system memory 103. Processing unit 107 may then continue execution using the new information in cache 108 until new information is again brought into cache 108 (e.g., due to another cache miss or an uncached read). Thus, embodiments may log sufficient data into trace 113 to be able to reproduce the inflows of information into cache 108 as the traced code executes. Additionally, embodiments may also log sufficient data into trace 113 to be able to reproduce any uncached reads or uncacheable reads. In embodiments, when multiple layers of cache are used, a hardware-based tracing method may record inflows at a particular "logged" cache layer. Thus, for example, in Figure 2 In the example environment 200 of , if logging is being performed on the L2 layer, then the logged inflow may be serviced by the L3 layer and / or the system memory 202 .

[0035] In addition, additional optimizations may be made to cache-based tracking. For example, one optimization is to track and record only the cache lines consumed by each processing unit 107, rather than simply recording cache inflows. As will be appreciated by one of ordinary skill in the art, this can result in significantly smaller tracking files than simply recording cache inflows. As used herein, a processing unit 107 has "consumed" a cache line when the processing unit 107 learns its current value. This may be because the processing unit 107 is the processing unit that wrote the current value of the cache line or because the processing unit performed a read on the cache line (which may result in an inflow into the cache line if there is currently no system memory 103). In various embodiments, these optimizations may involve extensions to the cache 108 that enable the processor 102 to identify one or more processing units 107 that consume the current value of each cache line. For example, each cache line may be extended using one or more tracking bits. Depending on the number of tracking bits and the logic implementation using these tracking bits, these tracking bits can be used to indicate, for example, whether any processing unit 107 has consumed a cache line (e.g., a single flag bit), which specific (multiple) one or more processing units 107 consumed the cache line (e.g., a tracking bit for each processing unit 107), the index of a single processing unit that has consumed a cache line (e.g., by using multiple tracking bits to store an integer index), etc.

[0036] Figure 3 The diagram shows the corresponding Figure 1 An example of tracking 300 can be created based on one or more of the aforementioned tracking techniques. Figure 3 In the example of , the trace 300 includes one or more trace data streams 301. Figure 3, three trace data streams 301 (i.e., trace data streams 301a to 301c) are shown. In an embodiment, each trace data stream 301 represents the execution of a different thread of code execution from application 112. For example, trace data stream 301a may represent the execution of a first thread of application 112, trace data stream 301b may represent the second thread of application 112, and trace data stream 301c may represent the execution of a third thread of application 112. As shown, each trace data stream 301 includes multiple data packets 302 (i.e., data packet 302a for trace data stream 301a, data packet 302b for trace data stream 301b, and data packet 302c for trace data stream 301c). In other embodiments, each trace data stream 301 may represent only a subset of the thread execution, such that multiple trace data streams 301 are required to fully represent the execution of the thread. In other embodiments, trace data streams 301 may represent the execution of multiple threads (e.g., multiple threads executing at a single processing unit 107). Because the specific data recorded in each data packet 302 may be different, they are shown as having different sizes. In view of the foregoing discussion of tracing techniques, it should be appreciated that the data packet 302 may at least represent the input (e.g., register values, memory values, cache line data, etc.) of one or more executable instructions executed as part of the first thread of the application 112.

[0037] As shown, the trace data stream 301 may also include one or more key frames 303 (e.g., key frames 303a through 303e), each of which represents sufficient information, such as a snapshot of register and / or memory values, to enable a previous execution of a thread containing the key frame to be replayed starting from the point of the key frame 303 onward. In addition, the trace data stream 301 may include one or more sequencing events, which are Figure 3 1 through 9. Although in an embodiment, each trace data stream 301 generally represents the execution of a corresponding single thread, the sequencing events represent the occurrence of events that are sequenced across threads (and therefore across trace data streams 301). For example, these sequencing events may correspond to events where threads interact, such as through shared memory, via function calls, etc. Although for simplicity, the events in the trace data stream 301 rotate through the order of the threads in a round-robin manner, it should be appreciated that they will generally occur in a less predictable manner.

[0038] In an embodiment, trace 113 may also include the actual code that was executed. Figure 3, each data packet 302 is shown as including a non-shaded data input portion 304 (i.e., data input portion 304a for data packet 302a, data input portion 304b for data packet 302b, and data input portion 304c for data packet 302c) and a shaded code portion 305 (i.e., code portion 305a for data packet 302a, code portion 305b for data packet 302b, and code portion 305c for data packet 302c). In an embodiment, the code portion 305 in the packet 302 may include executable instructions that are executed based on the corresponding data input. However, in other embodiments, the trace 113 may omit the actual code that is executed, and instead rely on separate access to the code of the application 112. In these other embodiments, each data packet may, for example, be assigned an address or offset to (multiple) appropriate executable instructions. As shown, the trace 113 may include any number of additional data streams 306 (i.e., data streams 306a to 306n) that may store any type of additional trace data. The additional trace data may include, for example, index data, such as occasional memory snapshots, reverse lookup data structures for quickly locating memory addresses / values ​​in the trace data stream 301, and the like.

[0039] As mentioned, the hardware-based tracing approach may record traces 113 based at least in part on recording the fallout of non-deterministic instructions, the influx of data into cache 108, and any uncached / uncacheable reads. While it is possible that processor 102 is capable of performing this task entirely, embodiments herein utilize coprocessor 105 to perform at least a portion of the tracing work, which includes recording the influx of data into cache 108.

[0040] As will be appreciated by one of ordinary skill in the art, using the coprocessor 105 in this manner can provide several advantages. For example, from a processor design and manufacturing perspective, it should be appreciated that adding full trace circuitry / logic to the processor 102 can be difficult and expensive. For example, given the already very complex design of modern processors, adding such circuitry / logic can be difficult. Additionally, adding such circuitry / logic can result in a larger processor die size, which can hurt yield during manufacturing. From a performance perspective, any resources used by the processor 102 to perform logging tasks can be resources that can be used to execute application code. Furthermore, logging can often interact with the system memory 103 (e.g., write trace 113'), and memory interactions are one of the most expensive operations that a processor can perform (e.g., in terms of processor clock cycles). Therefore, logging by the processor 102 can result in a real significant impact on application performance. In addition, since logging by the processor 102 can result in increased utilization of processor resources when logging, any logging activity performed by the processor 102 will generate additional heat and increase the power consumption of the processor 102. Each of these disadvantages of full logging by processor 102 may be avoided or at least mitigated using coprocessor 105 by moving the logging logic to a separate die and by providing dedicated circuitry / logic to perform logging.

[0041] While the disclosure herein focuses primarily on using the coprocessor 105 to record data consumption in the cache 108 (e.g., writes to cache lines, reads from cache lines, flows into cache lines), it should be appreciated that the coprocessor 105 may be used to record nearly any type of tracking data on behalf of the processor 102. This may include, for example, initial state (i.e., when logging first begins), memory snapshots, key frames, checkpoints, uncached / uncacheable reads, etc.

[0042] Reference again Figure 1 , the coprocessor 105 may include control logic 110. Similar to the control logic 109 in the processor 102, the control logic 110 in the coprocessor 105 may include CCP logic 110a and logging logic 110b (although it should be explained that the functions of CCP logic and logging logic differ between the processor and the coprocessor). As shown, the coprocessor may also include a cache 111, which may be used, for example, as a temporary buffer to store data before writing the data to the trace 113 or sending the data over the communication channel 114.

[0043] In general, CCP logic 109a at processor 102 sends CCP messages on bus 106, and CCP logic 109a at coprocessor 105 listens to those CCP messages on bus 106. In this way, coprocessor 105 can be at least one passive direct memory access (DMA) participant. In an embodiment, processor 102 sends at least CCP messages related to data consumption to bus 106 (e.g., data flows from system memory 103 into at least one layer of cache 108, or a cache line is read from / written to the cache line by processing unit 107). These CCP messages vary depending on the CCP protocol, but they are generally messages for the processor to request a cache line at a specific address, messages for the processor to promote a cache line from shared to exclusive, etc. (and responses to these messages). When coprocessor 105 receives (multiple) CCP messages indicating that there is data consumption, logging logic 11b at coprocessor 105 can initiate logging of the consumption (e.g., to trace 113 or to communication channel 114). This may include, for example, logging the value(s) of one or more memory cells as part of a cache flow-in, logging the value(s) of one or more memory cells as part of a cache line read from or written to by processing unit 107. Coprocessor 105 may obtain the memory cell value(s) 106 from bus 106 (e.g., when they are communicated to cache 108 as part of a cache miss), from system memory 103, or even from cache 108.

[0044] In general, the logging logic 109 b at the processor 102 also performs one or more logging tasks that have not been offloaded to the coprocessor 105. This may include, for example, logging the fallout of non-deterministic instructions, logging key frames, logging memory snapshots, logging checkpoints (e.g., a checksum / hash of the values ​​of one or more registers), etc. In an embodiment, the logging logic 109 b may log this information directly (i.e., by the processor 102), although it may alternatively request the coprocessor 105 (e.g., via a request on the bus 106) to log any of the information.

[0045] In an embodiment, the logging logic 109b at the processor 102 may record its tracing data for a given thread into one or more tracing data streams, while the logging logic 110b at the coprocessor 105 may record its tracing data for the same thread into one or more other tracing data streams. If this is the case, one or both of the logging logic 109b and / or the logging logic 109c may record data timing information (e.g., instruction counts or other timer information) data streams that may be used to stitch these data streams together to create a single data stream for the thread. Of course, the logging logic 109b and the logging logic 110b may alternatively record tracing data into the same data stream for a given thread.

[0046] It should be appreciated that system memory 103 may be modified by something other than processor 102 (i.e., some other DMA device). In an embodiment, CCP logic 110a at coprocessor 105 may detect such memory modifications and log them using logging logic 110b. For example, logging logic 110b may log the address or address range affected by the DMA operation and the time it occurred and / or the new memory cell value. In an embodiment, it may be desirable to log the address (and / or address range), but prohibit logging the changed values. This is because if these values ​​are actually consumed by code executed at processor 102 and can be logged by coprocessor 105 at that time, then these values ​​will appear to flow in. Nevertheless, by recording when the DMA operation occurs and which memory address or addresses are affected, a debugger that consumes trace 113 can at least visualize this information. In other embodiments, it may be desirable to log the changed values, but prohibit logging the address information. If logging logic 110b logs the values ​​that DMA changes as they occur (possibly with compression), and then logs CCP transactions (without values) as those values ​​are consumed, then reduced trace size may be achieved (ie, compared to logging addresses rather than values).

[0047] In an embodiment, CCP logic 110a may actively participate in the processor's CCP. Thus, it may not be a passive DMA participant (i.e., read-only), but rather an active DMA participant (i.e., read / write). If this is the case, additional scenarios are enabled, such as coprocessor 105 participating in an initial memory snapshot, or coprocessor 105 actually satisfying a cache inflow request. These concepts are described in more detail below.

[0048] Figure 4A flowchart of an example method 400 for storing at least a portion of an execution trace using a coprocessor is illustrated. Discussion of method 400 relates to method actions (i.e., actions 401 to 409). Although these method actions may be discussed in a particular order or illustrated in a flowchart as occurring in a particular order, no particular ordering is required unless otherwise specified or because an action depends on another action being completed before the action is being performed.

[0049] exist Figures 1 to 3 in the context of and with reference to Figure 4 The method 400 is described in the flowchart of FIG. Figure 1 , method 400 can be implemented in a computer system (e.g., computer system 101) that includes one or more main processors (e.g., one or more processors 102) associated with a memory cache (e.g., cache 108); one or more coprocessors (e.g., coprocessor 105); and one or more buses (e.g., bus 106) interconnecting the main processors with the coprocessors.

[0050] like Figure 4 As shown, method 400 may include actions performed by main processor 102 and actions performed by coprocessor 105. In an embodiment, the actions performed by main processor(s) 102 may be at least partially orchestrated by control logic 109, and the actions performed by coprocessor(s) 105 may be at least partially orchestrated by control logic 110. However, it should be appreciated that method 400 may alternatively be viewed as two related but distinct methods, i.e., one method (i.e., method 400a, including actions 401 to 405) is performed entirely at main processor(s) 102, while the other method (i.e., method 400b, including actions 406 to 409) is performed entirely at coprocessor(s) 105. In an embodiment, method 400 operates based at least in part on main processor 102 executing one or more executable instructions of an executable entity (e.g., application 112).

[0051] As shown, method 400 includes an action 401 of starting at the main processor to log the execution of a thread at the main processor. For example, in conjunction with the main processor executing one or more executable instructions of one or more threads of application 112, logging logic 109b can start logging of at least one of the threads. It is worth noting that the logging can be started when the execution of the thread is started, or at some time after the thread is executed. As shown by the arrow between actions 401 and 406, method 400 may include a corresponding action 406 of starting at the coprocessor to log the execution of the thread at the main processor. For example, logging logic 109b at processor 102 can signal logging logic 110b at coprocessor 105 that logging of threads and / or processing units 107 has been enabled / started. Therefore, logging logic 110b at coprocessor 105 can start logging of its own thread / processing unit.

[0052] As shown, based on the logging of the execution of the thread at the main processor having begun in act 406, the coprocessor 105 may listen to the bus for CCP messages in act 407. For example, the coprocessor 105 may listen to the bus 106. In an embodiment, act 407 may include the coprocessor 105 listening on a single bus or on multiple buses. In an embodiment, act 406 may include the coprocessor 105 "listening" to an existing bus signal that is otherwise point-to-point with another DMA participant on the bus 106. In some implementations of this embodiment, the coprocessor 105 may be implemented as a "receive-only tap" (i.e., without indicating its presence on the bus 106). In other embodiments, act 406 may include the coprocessor 105 operating as a "middleman" that intercepts CCP messages and forwards them to other DMA participants.

[0053] Figure 4It is also shown that after logging the execution of the thread begins at the main processor 102 in action 401, the main processor 102 can start logging the initial state in action 402 (i.e., using the logging logic 109b). It is worth noting that the logging logic 109b can record any such initial state information (or any other trace data related thereto) by itself, or can instead request the coprocessor 105 to record the initial state information. The initial state can include any information that can be used to start the thread simulation from the point where logging is started in action 401 later. If logging is started at substantially the same time as the execution of the thread is started (i.e., in action 401), this may mean that the logging logic 109b starts recording a key frame, which includes the state of one or more registers 107a of the processing unit 107 that is executing the thread. Therefore, in action 402, the processor 102 can start the storage of the initial state of one or more registers at the beginning of the trace.

[0054] If logging is started after the thread has executed (i.e., in action 401), this may mean that the logging logic 109b records a memory snapshot (such as the values ​​stored in the cache 108) in addition to recording the register state. As an alternative to recording the memory snapshot, the main processor 102 may flush the cache 108 so that any reads that occur after logging is enabled will appear to flow into the cache 108 (and therefore may be logged by the coprocessor 105). Thus, in conjunction with action 402, in addition to recording the register state, the main processor 102 may flush the memory cache 108 at the start of the trace.

[0055] As mentioned, the CCP logic 110a may actively participate in the processor's CCP. If this is the case, there may be additional options for snapshotting the initial state of the cache 108. For example, the main processor 102 may give the coprocessor 105 the value of the cache 108 for the coprocessor 105 to log. It is worth noting that logging (by the main processor 102 or the coprocessor 105) a snapshot of the cache 108 may require a large number of processor clock cycles, which means that the main processor 102 may need to stop the execution of application code (for a potentially large number) until the snapshot is completed. Therefore, in action 402, the coprocessor 105 may participate in the CCP at the beginning of the trace to receive the initial state of the memory cache 108, and the main processor 102 may wait for the coprocessor 105 to store the memory cache 108 into the execution trace 113 at the beginning of the trace.

[0056] Alternatively, if the CCP logic 110a is actively participating in the CCP, the CCP logic 109a at the host processor 102 may initiate a transfer of the initial cache state by giving at least a subset of the lines of the cache 108 to the coprocessor 105 in a shared (or equivalent) CCP state. As used herein, when a cache line is in a "shared" CCP state, it is generally illegal for any entity (including the host processor 102) to modify the cache line without first notifying other entities (e.g., the coprocessor 105) that also have the cache line in a shared state. Thus, in an embodiment, after giving the cache line to the coprocessor 105 in a shared CCP state, the host processor 102 may resume application code execution without waiting for the coprocessor 105 to store a complete snapshot. The coprocessor 105 may then log the cache line by obtaining the value of the cache line from the system memory 103, as it has the resources to do so, and then release its shared state on any logged cache lines. If the main processor 102 needs to modify one of the cache lines before the coprocessor 105 logs it, the coprocessor 105 is informed by virtue of the shared cache line before the modification takes effect so that the coprocessor 105 can ensure that the cache line is logged before the modification takes effect. If for some reason the cache line is not shareable, the main processor 102 may evict the cache line, give the value of the cache line directly to the coprocessor 105, or perform any appropriate operation to make the cache line shareable. Thus, in act 402, the coprocessor 105 may participate in the CCP to receive the initial state of the memory cache 108 at the beginning of the trace, and the main processor 102 may give at least one cache line in the memory cache 108 to the coprocessor 105 in the shared CCP state at the beginning of the trace, and the coprocessor 105 obtains the value of the at least one cache line from the memory supporting the memory cache 108.

[0057] Figure 4 It is also shown that the main processor 102 may perform one or more of several actions (in series or in parallel) after initiating logging of the initial state in action 402. For example, the main processor 102 may initiate logging of cache consumption (i.e., action 403), may initiate logging of one or more non-deterministic instructions (i.e., action 404), and / or may initiate logging of other trace data (i.e., action 405).

[0058] Referring to act 403, initiating a cache-consuming store may include sending a CCP message on the bus (at 403a) and consuming data from the memory cell (act 403b). Figure 4, actions 403a and 403b are connected via a double-ended arrow. This means that unless otherwise specified, this description (and claims) does not enforce the ordering between actions 403a and 403b. Therefore, depending on the implementation, actions 403a and 403b can be performed in parallel or in series (in either order).

[0059] In some embodiments, action 403a may include: based on the main processor executing the one or more executable instructions, the main processor sending one or more CCP messages on the bus, the one or more CCP messages corresponding to the memory unit. In some embodiments, action 403b may include: based on the main processor executing the one or more executable instructions, the main processor consuming the data of the memory unit. For example, as part of executing the tracked thread of application 112, processor 102 may execute machine code instructions of application 112, the machine code instructions using the memory unit value (i.e., from system memory 103) as input.

[0060] If the memory location value is already in cache 108 (i.e., co-located with system memory 103), execution of the instruction may cause the host processor 102 to consume the value and set a tracking bit on the appropriate cache line to indicate that it was consumed. Thus, in act 403b, the host processor consuming the data of the memory location may include providing the memory cache of the existing data of the memory location to the processing unit 107. This provision of data may be orchestrated via one or more CCP messages, and the host processor 102 may send at least a portion of these CCP messages on the bus 106 (e.g., using CCP logic 109a) in act 403a. In an embodiment, these CCP messages may include CCP messages specifically intended for the coprocessor 105, as sending CCP messages to the bus may not be typical in this case.

[0061] If the memory cell value is not already in cache 108, execution of the instruction may cause a cache miss on the memory cell, which cache miss initiates a flow of the value of the memory cell from system memory 103 into cache 108. Thus. In action 403b, the host processor 102 consuming the data of the memory cell may include the memory cache 108 of the data flowing into the memory cell. The cache miss may be orchestrated by one or more CCP messages, and the host processor 102 may send at least a portion of these CCP messages on bus 106 in action 403a (e.g., using CCP logic 109a). In an embodiment, these CCP messages may be conventional CCP messages for cache misses, such as CCP messages communicated to a memory controller of system memory 103. However, in other embodiments, these CCP messages may include CCP messages specifically intended for coprocessor 105.

[0062] As already described, the coprocessor 105 listens for CCP messages on the bus 106 (i.e., action 407). Thus, as indicated by the arrow between actions 403a and 408, the method 400 may include a corresponding action 408 of receiving, at the coprocessor, a CCP message sent due to consumption. In some implementations, action 408 may include: the coprocessor receiving one or more CCP messages from the bus. For example, as mentioned, the CCP logic 110a may "listen" for the CCP message sent on the bus 106 in action 403a, may intercept the CCP message sent on the bus 106 in action 403a, or otherwise receive the CCP message sent in action 403a from the bus 106.

[0063] Figure 4It is also shown that after receiving one or more CCP messages from the bus in action 408, the coprocessor 105 may perform an action 409 of initiating logging of consumption to the execution trace. In some embodiments, action 409 includes: based on the coprocessor receiving the one or more CCP messages, the coprocessor initiating storage of the consumed data of the memory unit into the execution trace. For example, the logging logic 110b at the coprocessor 105 may insert one or more data packets 302 into one or more trace data streams 301 representing the consumption. For example, if in action 403b, the memory cache provides existing data for the memory unit, then action 409 may include: generating a data packet 302 representing the provided data or some proxy thereof. Since in this instance, the data may have already been logged to the trace 113 (e.g., as part of a previous influx), the proxy for the provided data may be a reference to the data, such as a reference to the data already stored in one or more trace data streams 301. If on the other hand, in action 403b, the memory caches the influx data for the memory unit, then action 409 may include: generating a data packet 302 representing the influx data.

[0064] As mentioned, the coprocessor 105 may send the trace data to the trace 113' in the system memory 103, or it may send the trace data via the communication channel 114. Thus, in act 409, the coprocessor initiating storage of the consumed memory unit data into the execution trace may include at least one of: (i) the coprocessor writing a portion of the trace data to the memory, or (ii) the coprocessor sending a portion of the trace data via the communication channel.

[0065] like Figure 1 As shown, the coprocessor 105 may include compression logic 110c. In general, the compression logic 110c may compress any trace data before storing (e.g., to system memory 103) or sending (e.g., via communication channel 114). In an embodiment, the compression logic 110c compresses not only the trace data generated by the coprocessor 105, but also the trace data generated by the processor 102 (i.e., any data generated by the logging logic 109b). In an embodiment, the compression logic 110c performs lossless compression, such as using a general lossless compression algorithm (e.g., gzip, bzip2, LZMA, etc.). Additionally or alternatively, the compression logic 110c may perform domain-specific compression (i.e., compressing memory addresses and data values ​​based on the knowledge that they are such). In view of this description of the compression logic 110c, it should be appreciated that action 409 may include: the coprocessor 105 compresses one or more portions of the execution trace.

[0066] Returning to action 402, after initiating logging of the initial state, the main processor 102 may additionally or alternatively initiate logging of (multiple) non-deterministic instructions in action 404. For example, the logging logic 109b may determine the collateral consequences (i.e., outputs) of one or more executed instructions whose outputs do not depend solely on their inputs, and generate one or more data packets representing the execution of these instructions (e.g., representing their outputs). Therefore, method 400 may include: the main processor initiating storage of the results of executing non-deterministic instructions. Action 404 is shown as a broken line, indicating that it may be optional. If action 404 is not performed, it is possible to reconstruct one of the traced executions of the application 112 (e.g., based on the consumption data recorded in action 409), however, the resulting trace 113 will not be completely "bit accurate" (i.e., there may be some existing execution state that cannot be fully reconstructed based on the trace 113).

[0067] Returning to act 402, after initiating logging of the initial state, the main processor 102 may additionally or alternatively initiate logging of other tracking data in act 405. For example, in addition to logging non-deterministic instructions (i.e., act 404), the logging logic 109b at the processor 102 may also initiate storage of various other types of tracking data, such as occasional keyframes, checkpoint data, memory writes, or any other type of data useful for reconstructing execution. Similar to act 404, act 405 is also shown in broken lines, indicating that it may be optional.

[0068] It should be appreciated, in light of the disclosure herein, that the main processor 102 initiating storage of one or more other portions of trace data (e.g., representations of non-deterministic instructions, keyframes, checkpoints, writes, etc.) into the execution trace in acts 404 and / or 405 may include one or both of the following: (i) the main processor 102 directly stores the one or more other portions of trace data into the execution trace 113; or (ii) the main processor 102 instructs the coprocessor 105 to store the one or more other portions of trace data into the execution trace 113. Additionally, if the main processor stores the one or more other portions of trace data directly into the execution trace, then in act 409, the main processor may store the data into a different trace data stream 301 than the trace data stream into which the coprocessor stores the data of the consumed memory unit.

[0069] As combined Figure 2As described, the cache 108 may include multiple cache levels (e.g., L1, L2, L3, etc.). In an embodiment, the method 400 may generally be performed based on the activity of a single cache level. For example, if the cache 108 includes L1 and L2 cache levels, the method 400 may be performed to use the coprocessor 105 to log consumption of the L2 cache level (e.g., generally log inflows to the cache 108). Therefore, in the method 400, the memory cache may include multiple cache levels (e.g., L1 and L2 cache levels), wherein the memory cache stores data of consumed memory cells at a first cache level (e.g., L2 cache level), and the one or more CCP messages are one or more first CCP messages related to the activity of the first cache level (e.g., L2 cache level). However, by doing so, the resulting trace 113 may lack information about processor activity that does not affect the cache level. For example, referring to Figure 2 , processing units A1 and A2 may operate on their respective L1 caches (i.e., L1-A1 and L1-A2), and this activity may not be captured by logging at the L2 cache level (e.g., L2-A1). In an embodiment, the main processor 102 and / or the coprocessor 105 are configured to capture additional trace data to facilitate reconstruction of this activity. There are various approaches that can be used alone or in combination.

[0070] In some embodiments, one approach configures the CCP logic 109a at the main processor 102 to also send CCP messages related to activity at one or more other cache levels onto the bus 106. Thus, the CCP logic 110a at the coprocessor 105 may receive these CCP messages, and, as appropriate, the logging logic 110b may log trace data based on these CCP messages. Thus, in method 400, the main processor may also send one or more second CCP messages related to activity at a second cache level on the bus; the coprocessor may receive the one or more second CCP messages from the bus; and based on the coprocessor receiving the one or more second CCP messages, the coprocessor may initiate storing trace data related to activity at the second cache level into the execution trace. It should be appreciated that sending these second CCP messages on the bus 106 will generate additional traffic on the bus 106, which typically has a limited amount of available bandwidth. As such, sending these second CCP messages on the bus 106 may impair overall system performance. Thus, in an embodiment, the host processor sends CCP messages related to activity of the second cache level on the bus only when execution trace logging is enabled (eg, in act 401), and / or uses a separate bus for these messages.

[0071] In other embodiments, one approach configures logging logic 109b / 110b at the main processor 102 and / or coprocessor 105 to log code flow traces, which can be used to reconstruct activity at one or more cache levels that are not logged. It should be appreciated by those of ordinary skill in the art that code flow traces typically capture the results of branch / code flow instructions (e.g., machine code instructions for implementing loops, "if" statements, switches, etc.). In an embodiment, the logging logic 109b / 110b at the main processor 102 and / or coprocessor 105 may log only a subset of these code flow instructions, such as indirect jumps with unpredictable returns and conditional branches after a memory barrier. When logging a subset of code flow information rather than all code flow information, the logging logic 109b / 110b at the main processor 102 and / or coprocessor 105 may ignore such code flow information because the conditional jump is not near a memory barrier and a predictable return. In view of the foregoing, it should be appreciated that in method 400, at least one of the main processor or the coprocessor may initiate storage of at least a portion of a code flow trace into an execution trace. This may include, for example, the main processor directly storing at least a subset of the code flow trace (which may include omitting storage of at least one code flow trace event type). Alternatively, this may include the main processor sending one or more messages related to code flow on a bus (which may include omitting sending messages related to at least one code flow trace event type); the coprocessor receiving the messages from the bus; and, based on the coprocessor receiving the messages, the coprocessor initiating storage of at least a subset of the code flow trace. In either case, initiating storage of at least a portion of the code flow trace into the execution trace may initiate storage of one or more of the following: (i) an indirect jump with an unpredictable return; or (ii) a conditional branch following a memory barrier.

[0072] In other embodiments, one approach configures the logging logic 109b / 110b at the main processor 102 and / or coprocessor 105 to log one or more memory writes. It will be appreciated by those skilled in the art that if you own the writes of a program, you own all of its outputs, and this information can be used to reconstruct activity at one or more non-logging cache levels. For example, the logging logic 109b at the main processor 102 can log the writes directly, or the main processor 102 can send messages on the bus 106 related to storing these writes (even if they are not actually submitted to the system memory 103). Therefore, the coprocessor 105 can receive these messages, and the logging logic 110b at the coprocessor 105 can log them. In an embodiment, the coprocessor 105 can "fold" the writes (e.g., by logging them only when there is a cache eviction). For example, the coprocessor 105 can store writes to a memory cell in its cache 111, thereby keeping the cache 111 updated when it observes a write to the memory cell. Then, when there is an actual eviction from the cache 108, the log logic 110b can log the value stored in its own cache 111. Alternatively, the main processor 102 can notify the coprocessor 105 that it is evicting a dirty cache line (or moving it to a shared state). The coprocessor 105 can then log the current data value. This reduces bus traffic because the bus traffic can be one message where there are typically multiple writes. In view of the above, it should be appreciated that in method 400, at least one of the main processor or the coprocessor can initiate storage of at least one write caused by the execution of the executable entity into the execution trace. For example, this can include the main processor directly storing the write. Alternatively, this can include the main processor sending one or more messages related to the write on the bus; the coprocessor receiving (multiple) of these messages from the bus; and, based on the coprocessor receiving the (multiple) messages, the coprocessor initiates storage of the write or a merged representation of the set of writes (i.e., the last value in the series).

[0073] In other embodiments, one approach configures logging logic 109b / 110b at the main processor 102 and / or coprocessor 105 to log one or more checkpoints, which can be used to reconstruct activity at one or more cache levels that are not logged. For example, the logging logic 109b at the main processor 102 may occasionally generate a checksum / hash of the value(s) of one or more registers 107a and / or other processor state. The logging logic 109b can then store the checkpoint directly to the trace 113, or request the logging logic 110b at the coprocessor 105 to do so. Later, during trace replay, if there is ambiguity about which execution branch of a finite set of execution branches was taken, the checkpoint can be used to determine which execution branch was actually taken during the trace recording (e.g., by generating similar checkpoints for each of these branches based on the simulated processor state and comparing these checkpoints with the logged checkpoints). In view of the above, it should be appreciated that in method 400, at least one of the main processor or the coprocessor can initiate storage of at least one checkpoint based on (e.g., checksum / hash) one or more register values ​​at the main processor during execution of the executable entity. For example, this can include the main processor directly storing the checkpoint. Alternatively, this can include the main processor sending (multiple) messages related to the checkpoint on a bus; the coprocessor receiving (multiple) of these messages from the bus; and based on the coprocessor receiving the one or more messages, the coprocessor initiating storage of the checkpoint.

[0074] Regardless of the approach used (e.g., exposing additional CCP data, logging code flow traces, logging writes, logging checkpoints, etc.), the logging logic 109b / 110b at the main processor 102 and / or coprocessor 105 can be configured to log other timing information, which can also be used to help resolve ambiguities in the trace data. Example timing information includes, for example, instruction counts or other timing information. In view of the above, it should be appreciated that in method 400, at least one of the main processor or the coprocessor can initiate storage of timing information. For example, this can include the main processor directly storing the timing information. Alternatively, this can include the main processor sending one or more messages related to the timing information on a bus; the coprocessor receiving (multiple) of these messages from the bus; and based on the coprocessor receiving the (multiple) messages, the coprocessor initiating storage of the timing information.

[0075] As mentioned previously, the CCP logic 110a may actively participate in the processor's CCP (i.e., read / write). If so, this may be accomplished by the coprocessor 105 to implement other functions. For example, the coprocessor 105 itself may act as a "side" cache for the processor 102. For example, as explained above, the coprocessor 105 may use its cache 111 to log writes as they are evicted by updating the value written to a given memory cell. Based on activity at the main processor 102, the cache 108 (e.g., via a CPP message on the bus 106) requests the value of the memory cell, while the coprocessor 105 still has the value in its cache 111. If so, the coprocessor 105 may provide the value to the cache 108, rather than requiring the system memory 103 to satisfy the request. Additionally, if this is the case, the coprocessor 105 may avoid logging the inflow into the cache 108, since it gives the value to the cache 108 (and therefore already knows the value). Thus, in method 400, the coprocessor may participate in the CCP to complete a cache read request on behalf of the host processor.

[0076] As also mentioned, the coprocessor 105 can detect DMA modifications by other entities and log them. For example, the logging logic 110b can record the addresses or address ranges affected by the DMA operation and / or the new memory cell values. Thus, in method 400, the coprocessor can initiate storage of at least one of the values ​​of the memory cells affected by the DMA event or the identities of the memory cells affected by the DMA event into the execution trace.

[0077] Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts described above or the order of acts described above. Rather, the described features and acts are disclosed as example forms of implementing the claims.

[0078] Embodiments within the scope of the present invention may include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available media that can be accessed by a general-purpose computer system or a special-purpose computer system. Computer-readable media that store computer-executable instructions and / or data structures are computer storage media. Computer-readable media that carry computer-executable instructions and / or data structures are transmission media. Thus, by way of example and not limitation, embodiments of the present invention may include at least two distinct computer-readable media: computer storage media and transmission media.

[0079] Computer storage media are physical storage media that store computer executable instructions and / or data structures. Physical storage media may include, for example, system memory 103, storage media 104, and on-processor storage (e.g., storing at least a portion of control logic 109 and / or control logic 110). Physical storage media include computer hardware such as RAM, ROM, EEPROM, solid-state drive ("SSD"), flash memory, phase change memory ("PCM"), optical disk storage, magnetic disk storage or other magnetic storage device, or (multiple) any other hardware storage device that can be used to store program code in the form of computer executable instructions or data structures that can be accessed and executed by a general-purpose computer system or a special-purpose computer system to implement the functions of the disclosed invention.

[0080] Transmission media may include networks and / or data links (e.g., communication channels 114) that may be used to carry program code in the form of computer executable instructions or data structures and that may be accessed by a general-purpose computer system or a special-purpose computer system. A "network" is defined as one or more data links that enable electronic data to be transmitted between computer systems and / or modules and / or other electronic devices. When information is transmitted or provided to a computer system via a network or other communication connection (hardwired, wireless, or a combination of hardwired or wireless), the computer system may view the connection as a transmission medium. The above combinations should also be included within the scope of computer-readable media.

[0081] Further, upon reaching various computer system components, program code in the form of computer executable instructions or data structures can be automatically transferred from transmission media to computer storage media (or vice versa). For example, computer executable instructions or data structures received over a network or data link can be cached in RAM within a network interface module (e.g., "NIC") and ultimately transferred to computer system RAM and / or low-volatility computer storage media at the computer system. Thus, it should be understood that computer storage media can be included in computer system components that also (or even primarily) utilize transmission media.

[0082] Computer executable instructions include, for example, instructions and data that cause a general purpose computer system, a special purpose computer system, or a special purpose processing device to perform a specific function or group of functions when executed on one or more processors. Computer executable instructions may be, for example, binary files, intermediate format instructions such as assembly language, processor microcode, or even source code.

[0083] Without departing from its spirit or essential characteristics, the present invention may be specifically implemented in other specific forms. The described embodiments should be considered illustrative and not restrictive in all respects. Therefore, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes falling within the equivalent meaning and scope of the claims should be included within their scope. When introducing an element in the appended claims, the articles "one", "an", "the" and "said" are intended to mean one or more elements in the element. The terms "comprise", "comprising" and "having" are intended to be inclusive and mean that there may be additional elements in addition to the listed elements.

Claims

1. A computer system comprising: a main processor associated with a memory cache; Coprocessor; as well as A bus interconnecting the main processor and the coprocessor, The computer system is configured to store at least a portion of an execution trace using the coprocessor based on performing at least: The main processor executes one or more executable instructions of the executable entity; Based on the main processor executing the one or more executable instructions, the host processor sending one or more cache coherence protocol (CCP) messages on the bus, the one or more CCP messages corresponding to data inflow to a memory unit due to a cache miss associated with the memory cache; The coprocessor receives the one or more CCP messages from the bus; as well as Based on the coprocessor receiving the one or more CCP messages, the coprocessor: using the one or more CCP messages to determine that the host processor caused the data for the memory unit to flow into the memory cache; as well as Based on a determination that the master processor caused the data for the memory unit to flow into the memory cache, storing the data for the memory unit into the execution trace is initiated.

2. The computer system according to claim 1, wherein: The influx of data to the memory unit due to a cache miss associated with the memory cache is associated with the master processor consuming data to the memory unit.

3. The computer system according to claim 1, wherein: The host processor initiates storing one or more other portions of trace data into the execution trace, the one or more other portions of trace data comprising one or more of: The results of executing non-deterministic instructions; and The initial state of one or more registers when tracing begins.

4. The computer system according to claim 3, wherein: The host processor initiating storage of the one or more other portions of trace data into the execution trace comprises one of the following: the host processor storing the one or more other portions of trace data directly into the execution trace; and The host processor instructs the coprocessor to store the one or more other portions of trace data in the execution trace.

5. The computer system according to claim 4, wherein: The main processor stores the one or more other portions of the trace data directly into the execution trace, and wherein the main processor stores the one or more other portions of the trace data in a trace data stream that is different from the trace data stream into which the coprocessor stores the data for the memory unit.

6. The computer system according to claim 1, wherein: The coprocessor initiating storing the data for the memory unit into the execution trace comprises at least one of the following: The coprocessor writes a portion of the trace data to a memory; and The coprocessor sends the portion of the tracking data over a communication channel.

7. The computer system according to claim 1, wherein: The coprocessor compresses one or more portions of the execution trace.

8. The computer system according to claim 1, wherein: The memory cache comprises a plurality of cache levels, and wherein, The memory cache stores the data for the memory location in a first cache level; and The one or more CCP messages are one or more first CCP messages related to activity of the first cache level.

9. The computer system according to claim 8, wherein: The host processor also sends one or more second CCP messages on the bus related to activity at a second cache level; The coprocessor receives the one or more second CCP messages from the bus; as well as Based on the coprocessor receiving the one or more second CCP messages, the coprocessor initiates storing trace data related to the activity of the second cache level in the execution trace.

10. The computer system according to claim 9, wherein: The master processor sends the one or more second CCP messages related to the activity of the second cache level on the bus only when execution trace logging is enabled.

11. The computer system according to claim 1, wherein: At least one of the main processor and the coprocessor initiates storing at least a portion of a code flow trace in the execution trace.

12. The computer system of claim 11, wherein initiating storage of the at least a portion of the code flow trace into the execution trace comprises initiating storage of one or more of: (i) an indirect jump with an unpredictable return; and (ii) a conditional branch following a memory barrier.

13. The computer system according to claim 1, wherein: At least one of the main processor and the coprocessor initiates storing one or more of the following into the execution trace: one or more writes caused by execution of the executable entity; a checkpoint based on one or more register values ​​at the host processor during execution of the executable entity; and Timing information during execution of the executable entity.

14. The computer system according to claim 1, wherein: The coprocessor engages the CCP to receive an initial state of the memory cache at the start of tracing.

15. The computer system of claim 14, wherein: The main processor gives at least one cache line in the memory cache to the coprocessor in a shared CCP state when the tracing starts, and the coprocessor obtains a value for the at least one cache line from a memory supporting the memory cache.

16. The computer system of claim 14, wherein: The main processor waits for the coprocessor to store the memory cache into the execution trace at the start of the trace.

17. The computer system of claim 1, wherein: The coprocessor participates in the CCP to complete cache read requests on behalf of the primary processor.

18. The computer system of claim 1, wherein: The coprocessor initiates storing in the execution trace at least one of: a value of at least one memory cell affected by a direct memory access (DMA) event; and An identity of the at least one memory cell affected by the DMA event.

19. A method implemented at a computer system, the computer system comprising (i) a main processor associated with a memory cache; (ii) a coprocessor; and (iii) a bus interconnecting the main processor and the coprocessor, the method for storing at least a portion of an execution trace using the coprocessor, the method comprising: executing, at the host processor, one or more executable instructions of the executable entity; Based on the master processor executing the one or more executable instructions, sending, by the master processor, one or more cache coherence protocol (CCP) messages on the bus, the one or more CCP messages corresponding to data inflow to a memory unit due to a cache miss associated with the memory cache; receiving, by the coprocessor, the one or more CCP messages from the bus; and Based on the coprocessor receiving the one or more CCP messages, the coprocessor: using the one or more CCP messages to determine that the master processor caused the data for the memory unit to flow into the memory cache; and Based on a determination that the master processor caused the data for the memory unit to flow into the memory cache, storing the data for the memory unit into the execution trace is initiated.

20. A coprocessor that stores at least a portion of an execution trace based on code execution at a main processor, the coprocessor comprising a hardware storage device that stores control logic that configures the coprocessor to perform at least: monitoring a bus that interconnects the main processor and the coprocessor; receiving one or more cache coherence protocol (CCP) messages from the bus, the one or more CCP messages having been sent on the bus by the host processor associated with a memory cache and corresponding to an influx of data for a memory cell due to a cache miss associated with the memory cache; as well as Based on receiving the one or more CCP messages, using the one or more CCP messages to determine that the host processor caused the data for the memory unit to flow into the memory cache; as well as Based on a determination that the master processor caused the data for the memory unit to flow into the memory cache, storing the data for the memory unit into an execution trace is initiated.

Images