METHOD AND DEVICE FOR IMPROVING THE SYSTEM OVERHEAD FOR API TRACE CAPTURE

Abstract

A device, a method, and a storage medium are disclosed. The method comprises the following steps: creating (501) an asset vault in an initial phase of an application session, wherein the asset vault contains a set of assets captured by an application programming interface (API) that serves as a recording mechanism for use by an application; initiating (502) a capture process within the application session; recording (503) API commands in a recording file during the capture process, wherein the recording file contains a reference to the asset vault; and generating (504) one or more images or frames of rendered content using the recording file.

Description

TECHNICAL AREA

[0001] The disclosure relates generally to tools for tracing and troubleshooting an application programming interface (API) for software applications that use accelerators such as graphics processing units (GPUs). In particular, the subject matter disclosed herein relates to improvements in reducing system overhead during the acquisition of graphics API traces, especially by optimizing the storage and reuse of asset data to improve the efficiency and accuracy of trace acquisition. SUMMARY

[0002] API trace capture software tools, especially for graphics applications that use accelerators like GPUs, are frequently used to monitor and troubleshoot an application's instruction stream. These tools record an application's interactions with the GPU, allowing developers to replay the captured trace for debugging purposes. This typically requires storing large amounts of asset data, such as textures, shaders, and buffers, necessary to replicate the application's state during replay. While capturing these assets along with the instruction streams enables an accurate replay of the trace, it can increase memory and CPU overhead, particularly when capturing multiple traces from the same application session.

[0003] Some API trace capture tools store all asset data directly in each trace file, regardless of whether the assets are reused across multiple captures. This approach is convenient for trace replay because all the data needed to reproduce the application state is included in each capture file. However, this approach can lead to significant data redundancy, as the same assets are repeatedly saved with each capture. Attempts to reduce trace overhead have historically focused on compression techniques or selective data capture, but these approaches still struggle with redundancy and may not be able to efficiently handle multiple traces of the same workload.

[0004] One problem with the approach described above is that it requires duplicating asset data across multiple captures, resulting in high memory consumption, increased file sizes, and CPU overhead. This redundancy is particularly problematic when capturing repeated sessions of the same application, as large, immutable assets are stored each time, consuming unnecessary resources and slowing down the capture process. Furthermore, capturing asset data at the beginning of each trace can introduce significant delays, impacting the accuracy of the profiling data for the first few frames of each capture.

[0005] To overcome these problems, this paper describes systems and procedures for decoupling asset capture during API tracing from instruction stream capture. This is achieved by storing assets in a shared external repository, known as an "asset vault," which multiple traces can access. When a new trace is captured, the system checks if the assets already exist in the asset vault. If so, the trace file contains references to the stored assets instead of duplicating the data. When assets change or new assets are created, a separate "delta asset" file captures these updates and patches the asset vault as needed to maintain trace accuracy. This approach reduces the need to re-save unchanged assets and minimizes memory and CPU overhead.

[0006] The approaches described above improve upon previous methods by significantly reducing trace file size, memory usage, and CPU utilization during acquisition. By reusing asset data stored in the asset vault across multiple traces, the system minimizes redundancy and increases acquisition efficiency. Furthermore, by separating the asset acquisition phase from the instruction stream acquisition, this method reduces the impact on system performance and enables more accurate profiling and trace recording. These improvements allow developers to perform highly accurate API tracing and debugging with fewer resources, thereby enhancing the usability and scalability of API tracing tools.

[0007] According to one aspect of the disclosure, a method comprises the following steps: creating, in an initial phase of an application session, an asset vault containing a set of assets captured by an API recording mechanism for use by an application; initiating a capture process within the application session; recording API commands in a recording file during the capture process, the recording file containing a reference to the asset vault; and generating one or more images or frames of rendered content using the recording file.

[0008] According to another aspect of the disclosure, a device comprises memory configured to store an asset vault containing a set of assets captured by an API recording mechanism for use by an application, the asset vault being generated during an initial phase of an application session. The device further comprises a processor configured to initiate a capture process within the application session; a capture module configured to record API commands during the capture process to a recording file, the recording file containing a reference to the asset vault; and a rendering module configured to generate one or more images or frames of rendered content using the recording file.

[0009] According to another aspect of the disclosure, a non-transitory, machine-readable storage medium is provided that stores instructions. When executed by a processor, the instructions cause a computing device, in an initial phase of an application session, to create an asset vault containing a set of assets captured by an API recording mechanism for use by an application; to initiate a capture process within the application session; to record API instructions during the capture process in a recording file, the recording file containing a reference to the asset vault; and to generate one or more images or frames of rendered content using the recording file. BRIEF DESCRIPTION OF THE DRAWING

[0010] In the following section, the aspects of the subject matter disclosed herein are described with reference to exemplary embodiments shown in the figures, wherein Fig. 1 shows two recording files generated by a recording service according to one embodiment; Fig. 2A is a block diagram illustrating an asset vault file according to one embodiment; Fig. 2B is a block diagram illustrating how the asset data stored in an asset vault file can replace the asset data within a record file according to one embodiment; Fig. 3 is a block diagram illustrating a workflow of the asset and command capture process according to one embodiment; Fig. Figure 4 illustrates the relationship between GPU memory usage and recording trace overhead during an application session according to one embodiment; Fig. Figure 5 is a flowchart illustrating an API trace capture process according to one embodiment; and Fig. Figure 6 is a block diagram of an electronic device in a network environment according to one embodiment. DETAILED DESCRIPTION

[0011] The following detailed description sets forth numerous specific details to enable a comprehensive understanding of the disclosure. However, those skilled in the art will understand that the disclosed aspects can also be practiced without these specific details. In other cases, known methods, procedures, components, and circuits have not been described in detail so as not to obscure the subject matter disclosed here.

[0012] The reference in this description to “an embodiment” means that a particular feature, structure, or property described in connection with the embodiment may be included in at least one embodiment disclosed herein. Therefore, the expressions “in an embodiment” or “according to an embodiment” (or other expressions of similar meaning) that appear at various points in this description do not necessarily all refer to the same embodiment. Furthermore, the particular features, structures, or properties in one or more embodiments may be combined in any suitable manner. In this context, the word “exemplary” as used here means “serving as an example, instance, or illustration.”Each embodiment described herein as "exemplary" is not to be construed as necessarily preferable or advantageous over other embodiments. Furthermore, the particular features, structures, or properties of one or more embodiments may be combined in any suitable manner. Similarly, hyphenated terms (e.g., "two-dimensional," "predetermined," "pixel-specific," etc.) may occasionally be used interchangeably with a corresponding non-hyphenated version (e.g., "two-dimensional," "predetermined," "pixel-specific," etc.), and an uppercase entry (e.g., "counter clock," "row select," "PIXOUT," etc.) may be used interchangeably with a corresponding non-uppercase version (e.g., "counter clock," "row select," "pixout," etc.). Such occasional interchangeable uses are not to be considered incompatible.

[0013] Furthermore, a singular term may, depending on the context of the discussion contained herein, include the corresponding plural forms, and a plural term may include the corresponding singular form. It should also be noted that various figures shown and discussed herein (including component diagrams) are for illustrative purposes only and are not drawn to scale. For example, the dimensions of some elements may be exaggerated compared to others for the sake of clarity. In addition, where deemed appropriate, reference numbers have been repeated in the figures to identify corresponding and / or analogous elements.

[0014] The terminology used herein serves only to describe some exemplary embodiments and is not intended to limit the claimed subject matter. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, it is understood that the terms "comprises" and / or "has" when used in this description specify the presence of the indicated features, integers, steps, processes, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, processes, elements, components, and / or groups thereof.

[0015] It is understood that when an element or layer is described as being "on" another element or layer, "connected" to another element or layer, or "coupled" to another element or layer, it may be located directly on top of, connected to, or coupled to that other element or layer, or there may be intervening elements or layers. Conversely, when an element is described as being "directly on," "directly connected to," or "directly coupled to" another element or layer, there are no intervening elements or layers. The same numbers throughout refer to the same elements. As used here, the term "and / or" includes all combinations of one or more of the listed elements.

[0016] The terms “first,” “second,” etc., as used herein, serve as designations for the nouns preceding them and do not imply any order (e.g., spatial, temporal, logical, etc.) unless expressly defined as such. Furthermore, the same reference numerals may be used in two or more figures to designate parts, components, blocks, circuits, units, or modules with the same or similar functionality. However, such use is solely for the sake of simplification of presentation and facilitation of discussion; it does not imply that the construction or architectural details of such components or units are the same in all embodiments, or that such jointly referenced parts / modules are the only way to implement some of the embodiments disclosed herein.

[0017] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as generally understood by a person skilled in the art in the field to which the subject matter belongs. It is further understood that terms as defined in standard dictionaries should be interpreted in such a way as to correspond to their meaning in the context of the relevant field and that they should not be interpreted in an idealized or overly formal sense, unless expressly defined herein.

[0018] As used herein, the term “module” refers to any combination of software, firmware, and / or hardware configured to provide the functionality described herein in conjunction with a module. For example, software may be embodied as a software package, code, and / or instruction set or commands, and the term “hardware,” as used in any implementation described herein, may include, for example, individually or in any combination, an assembly, hard-wired circuitry, programmable circuitry, state machine circuitry, and / or firmware that stores commands executed by programmable circuitry. The modules may be implemented collectively or individually as circuitry that is part of a larger system, such as, but not limited to, an integrated circuit (IC), a system-on-a-chip (SoC), an assembly, etc.

[0019] "API" here refers to a set of routines, protocols, and tools that allow an application to communicate with hardware components such as a GPU. Some examples of "API" are Vulkan, DirectX, and OpenGL.

[0020] "API trace" here refers to a data file that records the commands and data sent to an accelerator via an API, allowing developers to reconstruct the sequence of operations as they were originally executed. Examples of API traces include API recording files created during debugging or performance analysis of graphics applications.

[0021] "Collection service" here refers to a software component that intercepts and records API commands, asset data, and system state information during application runtime. Examples of "collection services" include tools that generate trace files and manage the asset vault and delta data for efficient data collection.

[0022] "Asset vault" here refers to a central repository where the original assets (such as textures, shaders, and models) required by an application are stored. Examples of asset vaults are external files containing shared assets that multiple record files can access. The asset vault can be stored on a user device (e.g., a mobile phone) or externally and accessed remotely.

[0023] "Delta data" here refers to incremental asset updates recorded after the initial asset capture, capturing changes to assets during application runtime. Examples of delta data include texture patches, model changes, and shader parameter updates, all recorded in a trace file.

[0024] "Recording file" here refers to the output file generated by the capture service, which contains a trace of API commands, references to the asset vault, and delta data. Examples of "recording files" include single monolithic files containing all assets, as well as replayable trace files created during a graphics session that record both command streams and asset references.

[0025] "Playback" here refers to the process of using an API trace file to reproduce the application's original operations on the GPU. Examples of "playback" include executing captured command streams to debug rendering issues or to analyze GPU performance in a controlled environment.

[0026] There are tools that allow graphics drivers and similar systems to capture the instruction sequences sent to a GPU by an application. This captured data enables a deterministic reconstruction of the application's behavior and serves as a "black box" recorder for debugging purposes. When graphics API instruction streams, such as those in a Vulkan API environment, are captured by, for example, a GFXReconstruct tracing tool, a large number of assets, such as models, textures, and scene data, are typically loaded into the application's allocated memory at the start of the application. In Vulkan, for example, these assets might be created and bound to memory using instruction pairs like VkCreateBuffer and VkBindBufferMemory, often followed by memory writes (e.g., memcpy).

[0027] Many command tracing utilities store these assets in a preamble section at the beginning of a recording file. While this is convenient for playback, it can be inefficient during the capture process itself, as storing these assets can consume significant memory bandwidth and system resources.

[0028] To address these inefficiencies, this disclosure provides a system for managing the capture of asset files that enables reuse across multiple captures. This approach increases efficiency by reducing the redundant storage of unchanged assets and command streams across captures.

[0029] One aspect of the disclosure describes a routine (e.g., an algorithm) to reduce memory bandwidth and CPU overhead when capturing multiple traces of a single workload. Since most assets, such as shaders, models, textures, and attribute buffers, remain unchanged between captures of the same workload, they can be stored in an external repository (e.g., a file, a vault or secure location, or a database) and referenced by hash in subsequent captures if the assets already exist. This approach avoids unnecessary duplicates and saves system resources.

[0030] If assets are modified between captures, a "delta asset file" (or "delta file") can be created to record the changes. This delta file can patch the original assets as needed to ensure that subsequent captures remain accurate and are not based on outdated data. By managing asset storage and modification in this way, the system ensures data integrity across all captures while minimizing resource consumption.

[0031] Various implementations can be achieved through a computer system comprising a CPU, a memory bus, and an accelerator (such as a GPU, neural processing unit (NPU), or digital signal processor (DSP)) that can operate with an API enabling instruction exchange between the CPU and the accelerator. The system captures these instructions via an API interception tracing software tool.

[0032] An API recording trace can be a data file containing the complete set of data and commands required to replay the recorded session over a defined period. This trace captures all necessary assets, including images, textures, shaders, vertices, and metadata, used to reconstruct the desired output. Typically, these assets need to be loaded early in the application's lifecycle so that commands issued later in the session can reference and utilize them.

[0033] In addition to capturing core assets, the system records control commands and asset updates, which are stored in a database referred to as the asset file, asset vault, or asset data. Changes to assets can be stored in a delta file or as delta data, which appends updates to or patches the original snapshot of the asset file. This allows the trace to evolve by incorporating only incremental changes without duplicating the entire database.

[0034] API captures can start at the beginning of the application session, with the asset file being reinitialized from scratch for each run. However, to minimize capture overhead, it is advantageous to create the initial asset file before the primary trace capture begins. This initial asset file can be generated manually, either by the user selecting a specific start frame or time, or automatically using predefined heuristics. For example, one heuristic could evaluate the asset file's growth rate and assume the load is complete when the growth rate stabilizes. Another heuristic could monitor GPU utilization and conclude the load is complete when GPU activity reaches a certain threshold.

[0035] Once the original asset file has been created, any additional asset data can be stored as delta data in delta asset files, which can temporarily patch the original asset file. For compression and simplification, these delta data files can later be inserted into the main asset file, thus consolidating the incremental data.

[0036] During playback, API recordings made after the asset file is created can use a correlation mechanism to verify whether the referenced assets are present in the asset file. This correlation mechanism can use metadata descriptions, asset identity structures, or binary hashes as keys. In the event of a metadata hash collision, the system can use a secondary binary hash to precisely distinguish assets. If an asset referenced in the trace is not present in the asset file, it can be stored in a new delta asset file, which is linked to the original asset file and all previous delta asset files in a daisy-chain structure.

[0037] In some cases, completely separate executions of an application can reference an existing asset file, provided certain conditions are met. For example, the application may need to exhibit deterministic behavior with respect to threading, asset creation, and storage. Furthermore, a mechanism must be in place to efficiently correlate assets across different executions without requiring a full binary comparison of the asset data. This approach may require prior knowledge of the application's structure to ensure accurate and efficient asset correlation.

[0038] Fig. Figure 1 shows two recording files generated by a recording service according to one embodiment.

[0039] With reference to Fig. For each trace of an application session, separate capture files 101 and 102 are generated. Each capture file contains a header that defines version and device information, followed by an asset data block, 103 or 104, and then a sequence of frames starting at a specific frame (e.g., frame X in the case of capture file 101 or frame Y in the case of capture file 102). When capture begins with the first frame (X = 1 or Y = 1), the asset data is typically embedded directly into the command streams of the frames, so a separate asset data block is not required.However, if the capture starts after the first frame (X > 1 or Y > 1), an Asset Data Block 103 and an Asset Data Block 104 are added to store the asset creation commands that precede frame X in capture file 101 or frame Y in capture file 102, so that all necessary assets are ready for playback before the command sequence begins.

[0040] Fig. Figure 1 also highlights a problem that arises when multiple traces of the same application session are captured. Each record file, 101 and 102, records a complete set of asset data in asset data blocks 103 and 104, respectively, for the application, regardless of whether these assets remain unchanged across the record files. This duplication of asset data across record files results in significantly larger files and increased storage and processing requirements, as the same assets are redundantly stored in each record file.

[0041] To solve the problem of asset duplication across multiple captures, one embodiment of the disclosure provides a method for isolating these assets in a dedicated archive file, referred to as an "asset vault", "asset file" or "asset file vault".

[0042] Fig. Figure 2A is a block diagram illustrating an asset vault file according to one embodiment.

[0043] With reference to Fig. 2A of the asset vault file includes a preamble section containing metadata such as version and device information. This preamble enables the asset vault file 201 to be compatible with different capture sessions by providing the necessary system and version context for the stored assets. Below the preamble, the asset vault file 201 contains an asset data block that stores the actual assets required by the application, such as models, textures, and shaders. The label "init" within the asset data block indicates that this data contains initial assets that do not change frequently during different captures from the same application session.

[0044] By storing assets in a central asset vault file 201, this approach allows subsequent capture files to reference the shared assets instead of duplicating them. This reduces the size of the capture files and minimizes memory and CPU usage during the capture process. If an asset already exists in the asset vault file 201, it can be reused for multiple captures, thus avoiding redundant storage and conserving system resources.

[0045] Fig. Figure 2B is a block diagram illustrating how the asset data stored in an asset vault file can replace the asset data within a record file according to one embodiment.

[0046] With reference to Fig. In this approach, 2B, the asset vault file 211, acts as a central storage location for assets, so that multiple record files can reference shared asset data instead of duplicating it.

[0047] In this embodiment, the asset data 213 replaces the asset data 214, which would normally be included in each record file. As in Fig. As shown in Figure 2B, the recording file 212 contains a preamble 215 with a reference to the asset vault file 211, allowing the recording file 212 to access the necessary assets without directly storing them. This asset vault file 211 can be created at the time the first recording file is created and provides an initial set of assets that future recording files can reference as needed. This design offers performance comparable to traditional capture methods because the asset data 213 is generated upfront and readily available for replay.

[0048] During playback of a recording file, the asset vault file 211 can be loaded based on the reference in the recording file preamble 215, instead of retrieving assets from the recording file 212 itself. This configuration allows subsequent recording files in the same application session to reference the same asset vault file 211, thereby reducing redundancies and memory requirements.

[0049] To handle situations where assets may change between captures, a delta data block (or delta file) can be included in all captures made after the creation of the original asset vault file 211.

[0050] According to one embodiment, the delta data block can ensure data integrity by recording updates to assets that may have been modified, replaced, or added after the initial creation of the asset vault file 211. During playback, the system can load the asset vault file 211 from the recording file preamble 215 and then apply the delta data to ensure that all assets are up to date.

[0051] The delta data block can capture various types of asset changes, including new initializations of graphics API objects, memory allocations, or updates to existing objects or memory. Furthermore, the capture service can create the asset vault file 211 independently of the capture files, ideally at an earlier point in time or frame, thus enabling a "cache assets" operation that allows assets to be preloaded.

[0052] In this configuration, even the initial acquisition can be treated similarly to subsequent acquisitions by including a delta data block, thus decoupling the asset vault from the main command stream data acquisition. This separation of asset storage from command stream data improves efficiency and eliminates the problem of redundant asset storage across multiple acquisitions.

[0053] Fig. Figure 3 is a block diagram illustrating the flow of the asset and command capture process according to one embodiment.

[0054] With reference to Fig. Frame 3 of the graphics asset timeline begins with application startup. During this time, the application initializes and loads the necessary asset data, such as textures, shaders, and models, into memory to prepare them for rendering. This initial load includes all the core resources the application requires throughout its execution. As the application runs, some assets in memory may change, resulting in delta changes between frames. These delta changes represent updates to the asset data, which the system records for potential use in later captures.

[0055] The timeline of the data entry and transcription service 302 at the bottom of Fig. Figure 3 shows how the capture process is managed. In an early phase, designated Frame A, the capture service performs a cache asset operation. During this operation, the capture service proactively saves the original set of assets to a separate file called the asset vault, which serves as a central repository for all core assets required by the application. This step ensures that the asset vault contains the complete state of the graphics assets before detailed frame capture begins, thus reducing the system load during the capture process itself. Once created, the asset vault can be referenced by multiple capture files, minimizing redundancy and allowing these files to share common data without having to re-save it.

[0056] The first capture session, starting at frame X, begins by recording a preamble reference in the capture file that points to the existing asset vault created at frame A. This reference allows the capture file to access the centralized asset vault for asset data, rather than duplicating all assets within the file itself. During this initial capture session, which lasts from frame X to frame X + N, the capture service records the graphics API command stream. This stream comprises the sequence of rendering commands that the application sends to the GPU to generate each frame. The capture service also records any delta data between frame A and frame X, capturing asset changes made after the initial asset caching.This delta data ensures that all asset updates made since the initial asset capture are accurately reflected.

[0057] As the application continues to run, additional delta changes to the assets in memory can accumulate as the application modifies its data. At a later point, designated Frame Y, a second capture session begins. Like the first session, this second capture references the original asset vault created at Frame A by including a preamble reference that points back to it. This approach allows the second capture to reuse the same core assets without duplicating them in the file, thus reducing file size and system overhead. Before the second capture begins, the capture service records all new delta data, capturing the changes made to the assets between the end of the first capture session and Frame Y.This new delta data block is stored in the recording file for the second session to ensure that all relevant asset updates are retained and correctly applied during playback.

[0058] Accordingly, in this process, the asset vault serves as a common repository for all original assets, reducing the need for data duplication in the capture files. Each capture session contains delta data blocks that include only the incremental changes to the assets, keeping asset information up-to-date while minimizing storage requirements. Each capture file also contains a preamble reference that points back to the asset vault, creating a consistent reference point for asset data and further reducing the need for duplicate storage.

[0059] This process increases efficiency by separating the storage of static asset data from the dynamic command stream, allowing developers to perform multiple capture sessions without redundant asset data. By centralizing the original assets and tracking only incremental changes, this approach reduces the size of capture files, lowers system overhead, and provides accurate, powerful traces for debugging and profiling across multiple sessions.

[0060] Fig. Figure 4 illustrates the relationship between GPU memory usage and recording trace overhead during an application session according to one embodiment. More specifically, it shows Fig. 4, how storage usage and overhead fluctuate based on loading and unloading assets and activating capture events.

[0061] Referring to Fig. 4. At the start of the session, while the application is initializing, GPU memory usage spikes sharply as initial assets (such as textures, models, and shaders) are loaded. This initial load corresponds to an upward trend in the memory usage line, indicating increasing memory consumption until a stable state is reached. At this point, the application has loaded most of the required assets, and GPU memory usage stabilizes.

[0062] The recording trace overhead, represented by the memory usage line, shows high initial memory usage. This high memory usage results from the intensive memory and CPU activity required to capture all initial assets and data from the GPU when the first capture begins. This overhead includes the cost of transferring the asset data to memory and organizing it within the recording file.

[0063] During application execution, the capture trace overhead may decrease after an initial spike, but increase again during certain capture events, such as gameplay or in-game events where new assets or updates are loaded and captured. Only incremental changes or delta data are captured during each of these events, keeping the overhead relatively low.

[0064] At certain points, indicated by a drop in the memory usage line, the application unloads assets, for example, when switching to a different level or scene. This drop in memory usage reduces the demands on the GPU's memory resources. The recording trace overhead remains stable during these intervals because it only needs to record the current application state without reloading large asset files.

[0065] Towards the end of the session, the application loads additional assets, which increases GPU memory usage and recording trace overhead.

[0066] By centralizing the original assets in the asset vault and capturing only incremental updates, the system reduces storage and overhead load, enabling developers to efficiently perform captures even in memory-intensive applications.

[0067] Fig. Figure 5 is a flowchart illustrating an API trace capture process according to one embodiment.

[0068] The in Fig. The 5 steps shown can be performed by a data acquisition service operating in a computing device, such as a computer or an electronic device equipped with a GPU.

[0069] With reference to Fig. In step 501, an asset vault is created. The asset vault can be generated in an initial phase of an application session. The asset vault can contain a number of assets captured by an API recording mechanism for use by an application, such as textures, shaders, models, and metadata.

[0070] Step 502 initiates a data capture process. This process can be initiated within the application session, which can be set manually by a user, automatically based on predefined criteria, or semi-automatically based on a combination of these.

[0071] Step 503 involves recording API commands. These API commands can be recorded to a recording file during the ingestion process. The recording file can contain a reference to the asset vault, allowing it to access the assets stored there. Additionally, any changes to assets that occur during the ingestion process can be detected and saved as delta data in a separate file.

[0072] In step 504, one or more images or frames of the rendered content are generated. These images or frames can be generated using the recording file, which may contain API commands and references to the asset vault.

[0073] Accordingly, according to one embodiment, the optimization of the asset database per capture / partial capture can be achieved by tracking the use of each asset during a playback and removing assets that are not accessed.

[0074] Furthermore, according to one embodiment, comparing multiple asset databases from several acquisitions can help create a more robust and smaller asset database by removing all assets that are not identical from one run to the next. This can also improve the size and portability of the trace file.

[0075] Furthermore, according to one embodiment, recording files (e.g., a single monolithic file containing all assets) can be regenerated from an asset database, which can improve the portability of the trace file and allow the trace file to be played back without additional files.

[0076] Fig. Figure 6 is a block diagram of an electronic device in a network environment according to one embodiment.

[0077] With reference to Fig.6. An electronic device 601 in a network environment 600 can communicate with an electronic device 602 via a first network 698 (e.g., a short-range wireless communication network) or with an electronic device 604 or a server 608 via a second network 699 (e.g., a long-range wireless communication network). The electronic device 601 can communicate with the electronic device 604 via the server 608. The electronic device 601 may include a processor 620, a memory 630, an input device 650, a sound output device 655, a display device 660, an audio module 670, a sensor module 676, an interface 677, a haptic module 679, a camera module 680, a power management module 688, a battery 689, a communication module 690, a SIM card (Subscriber Identification Module) 696 or an antenna module 697.In one embodiment, at least one of the components (e.g., the display device 660 or the camera module 680) can be omitted from the electronic device 601, or one or more other components can be added to the electronic device 601. Some of the components can be implemented as a single integrated circuit (IC). For example, the sensor module 676 (e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) can be embedded in the display device 660 (e.g., a display).

[0078] The electronic device 601 in a network environment 600 can perform procedures for capturing and replaying API traces, particularly for instructions sent to an accelerator such as a GPU. The processor 620 can provide instructions for executing the capture process, managing both the original asset data stored in an asset vault and any delta data recorded during the runtime of an application. The processor 620 can also coordinate the replay process by using the data in the capture file to issue instructions to the GPU or other accelerators, thus reproducing the original graphics output of the application.

[0079] The Memory 630 can store both the asset vault and the delta data files. By offloading large, redundant asset data from each recording file to a shared asset vault, the solutions proposed here reduce the storage and data storage requirements for multiple capture sessions, freeing up memory resources for other tasks within the device. The Memory 630 is also used to store metadata, asset hashes, and other correlation data, enabling efficient asset referencing and querying during playback.

[0080] The 690 Communications Module enables the device to transfer recording files, asset vaults, or delta data between devices, for example, to a 608 server for troubleshooting or analysis in a remote environment. This feature allows developers to capture traces on one device and replay them on another, facilitating cross-device testing and troubleshooting. If the 601 device has a GPU, NPU, or DSP, these accelerators can directly benefit from the streamlined capture process, as the reduced trace overhead allows these specialized processors to be used more efficiently for graphics and AI (artificial intelligence) tasks.

[0081] The processor 620 can execute software (e.g. a program 640) to control at least one other component (e.g. a hardware or software component) of the electronic devices 601 coupled to the processor 620, and can perform various data processing or calculations.

[0082] As at least part of its data processing or computational functions, the processor 620 can load an instruction or data received from another component (e.g., the sensor module 676 or the communication module 690) into the volatile memory 632, process the instruction or the data stored in the volatile memory 632, and store the resulting data in the non-volatile memory 634. The processor 620 can comprise a main processor 621 (e.g., a CPU or an application processor (AP)) and an auxiliary processor 623 (e.g., a GPU, an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)), which can operate independently of or in conjunction with the main processor 621. Additionally or alternatively, the auxiliary processor 623 can be designed to consume less power than the main processor 621 or to perform a specific function.The auxiliary processor 623 can be implemented separately from the main processor 621 or as part of it.

[0083] The auxiliary processor 623 can control at least some of the functions or states associated with at least one component (e.g., the display device 660, the sensor module 676, or the communication module 690) among the components of the electronic device 601, instead of the main processor 621 while the main processor 621 is in an inactive state (e.g., sleep state), or together with the main processor 621 while the main processor 621 is in an active state (e.g., executing an application). The auxiliary processor 623 (e.g., an image signal processor or a communication processor) can be implemented as part of another component (e.g., the camera module 680 or the communication module 690) that is functionally related to the auxiliary processor 623.

[0084] The memory 630 can store various data used by at least one component (e.g., the processor 620 or the sensor module 676) of the electronic device 601. The various data can include, for example, software (e.g., the program 640) and input or output data for an associated instruction. The memory 630 can comprise volatile memory 632 or non-volatile memory 634. The non-volatile memory 634 can comprise internal memory 636 and / or external memory 638.

[0085] The program 640 can be stored as software in memory 630 and may include, for example, an operating system (OS) 642, middleware 644, or an application 646.

[0086] The input device 650 can receive a command or data to be used by another component (e.g., the processor 620) of the electronic device 601, from outside the electronic device 601 (e.g., from a user). The input device 650 can, for example, include a microphone, a mouse, or a keyboard.

[0087] The sound output device 655 can output sound signals to the outside of the electronic device 601. The sound output device 655 can, for example, include a loudspeaker or a receiver. The loudspeaker can be used for general purposes, such as playing multimedia content or recording, and the receiver can be used to receive an incoming call. The receiver can be implemented separately from the loudspeaker or as part of it.

[0088] The display device 660 can visually transmit information to the outside world (e.g., a user) of the electronic device 601. The display device 660 can, for example, include a display, a hologram device, or a projector, and a control circuit for controlling such a display, hologram device, or projector. The display device 660 can include a touch circuit for detecting a touch or a sensor circuit (e.g., a pressure sensor) for measuring the force exerted by the touch.

[0089] The audio module 670 can convert sound into an electrical signal and vice versa. The audio module 670 can receive sound via the input device 650 or output sound directly (e.g., wired) or wirelessly via the audio output device 655 or headphones to an external electronic device 602, or via the electronic device 601.

[0090] The sensor module 676 can detect an operating state (e.g., power supply or temperature) of the electronic device 601 or an environmental state (e.g., a user's state) outside the electronic device 601 and then generate an electrical signal or a data value corresponding to the detected state. The sensor module 676 can, for example, include a gesture sensor, a gyroscope, a barometric pressure sensor, a magnetometer, an accelerometer, a grip sensor, a proximity sensor, a color sensor, an infrared sensor (IR sensor), a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0091] The 677 interface can support one or more specific protocols used to connect the electronic device 601 directly (e.g., via cable) or wirelessly to the external electronic device 602. The 677 interface can include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0092] A connector 678 can include a plug through which the electronic device 601 can be physically connected to the external electronic device 602. For example, the connector 678 can include an HDMI plug, a USB plug, an SD card plug, or an audio connector (e.g., a headphone jack).

[0093] The haptic module 679 can convert an electrical signal into a mechanical stimulus (e.g., a vibration or movement) or an electrical stimulus that can be perceived by a user through touch or kinesthetic sensation. The haptic module 679 can, for example, include a motor, a piezoelectric element, or an electrical stimulator.

[0094] The camera module 680 can capture still or moving images. The camera module 680 can include one or more lenses, image sensors, image signal processors, or flash units. The power management module 688 can manage the power supplied to the electronic device 601. The power management module 688 can, for example, be implemented as at least one part of a power management integrated circuit (PMIC).

[0095] The battery 689 can supply power to at least one component of the electronic device 601. The battery 689 can, for example, comprise a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

[0096] The communication module 690 can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 601 and the external electronic device (e.g., the electronic device 602, the electronic device 604, or the server 608) and the execution of communication over the established communication channel. The communication module 690 can include one or more communication processors that can be operated independently of the processor 620 (e.g., the AP) and that support direct (e.g., wired) or wireless communication. The communication module 690 can include a wireless communication module 692 (e.g., a cellular module, a short-range radio module, or a GNSS (Global Navigation Satellite System) communication module) or a wired communication module 694 (e.g., a cellular module, a cellular module, or a GNSS communication module).a LAN (Local Area Network) communication module or a PLC (Power Line Communication) module). A corresponding communication module of these types can communicate over the first network 698 (e.g., a short-range communication network such as BLUETOOTH™, Wireless Fidelity (Wi-Fi) Direct, or an Infrared Data Association (IrDA) standard) or the second network 699 (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., LAN or Wide Area Network (WAN))). These different types of communication modules can be implemented as a single component (e.g., a single IC) or as multiple separate components (e.g., multiple ICs). The wireless communication module 692 can connect the electronic device 601 to a communication network, such as the first network 698 or the second network 699, using subscriber information (e.g.,Identify and authenticate the International Mobile Subscriber Identifier (IMSI), which is stored in the Subscriber Identification Module 696.

[0097] The antenna module 697 can send or receive a signal or power to or from outside (e.g., the external electronic device) the electronic device 601. The antenna module 697 can comprise one or more antennas, and at least one antenna can be selected from these by, for example, the communication module 690 (e.g., the wireless communication module 692) that is suitable for a communication scheme used in the communication network, such as the first network 698 or the second network 699. The signal or power can then be sent or received between the communication module 690 and the external electronic device via the selected at least one antenna.

[0098] Commands or data can be transmitted or received between the electronic device 601 and the external electronic device 604 via the server 608, which is connected to the second network 699. Each of the electronic devices 602 and 604 can be of the same type as the electronic device 601 or of a different type. All or some of the operations to be performed on the electronic device 601 can be performed on one or more of the external electronic devices 602, 604, or 608.For example, if the electronic device 601 is to perform a function or service automatically or in response to a request from a user or another device, the electronic device 601 may, instead of or in addition to performing the function or service itself, request one or more external electronic devices to perform at least part of the function or service. The one or more external electronic devices receiving the request may perform at least part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 601. The electronic device 601 may provide the result, with or without further processing of the result, as at least part of a response to the request.For this purpose, for example, cloud computing, distributed computing or client-server computing technology can be used.

[0099] Embodiments of the subject matter and the processes described herein can be implemented in digital electronic circuits or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or combinations thereof. Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a computer storage medium for execution by, or control of the operation of, a data processing device. Additionally or alternatively, the program instructions can be encoded on an artificially generated, propagated signal, e.g.,A computer storage medium is a machine-generated electrical, optical, or electromagnetic signal produced to encode information for transmission to a suitable receiving device for execution by a data processing device. A computer storage medium may be or include a computer-readable storage device, a computer-readable storage substrate, a storage array, or a random-accessible or serial-accessible storage device, or a combination thereof. Furthermore, while not a propagated signal itself, a computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium may also be or include one or more separate physical components or media (e.g., multiple CDs, floppy disks, or other storage devices).Additionally, the operations described in this description can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources.

[0100] Although this description may contain many specific implementation details, these details should not be interpreted as limitations on the scope of a claimed subject matter, but rather as descriptions of features specific to particular embodiments. Certain features described in this description in connection with separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in connection with a single embodiment may also be implemented separately in several embodiments or in any suitable subcombination.Furthermore, features, although described above as acting in certain combinations and even originally claimed as such, can in some cases be removed from the claimed combination, and the claimed combination may be directed towards a subcombination or a variation of a subcombination.

[0101] Similarly, the operations depicted in the drawings in a specific order should not be interpreted as requiring that these operations be performed in the depicted order or sequentially, or that all depicted operations must be performed to achieve the desired results. Multitasking and parallel processing may be advantageous under certain circumstances. Furthermore, the separation of different system components in the embodiments described above should not be interpreted as requiring such separation in all embodiments. It should be understood that the described program components and systems can be integrated into a single software product as well as implemented in multiple software products.

Claims

[1] Method comprising the following: Creating (501) in an initial phase of an application session an asset vault (211) containing a set of assets captured by a recording mechanism of an application programming interface, API, for use by an application (646); Initiate (502) a capture process within the application session; Recording (503) API commands to a recording file (212) during the capture process, wherein the recording file (212) contains a reference to the asset vault (211); and Generating (504) one or more images or frames of rendered content using the recording file (212). [2] The method of claim 1, further comprising: Detecting changes to assets during the data capture process; and Saving the changes to a delta data file separate from the asset vault, and where optional: The procedure further comprises replaying the recording file (212) by referencing the asset vault (211) and applying the delta data file to reproduce the captured assets expected for correct output, and / or The feature includes saving changes to the delta data file, detecting new or modified graphics API objects and recording them in the delta data file, and / or The procedure further includes the creation of additional delta data files for subsequent acquisitions of the application session, with each delta data file referencing the asset vault (211) to avoid duplication of unchanged assets. [3] Method according to claim 1 or 2, wherein the creation of the asset vault (211) further comprises the creation of a first asset vault comprising a first set of assets and the creation of a second asset vault comprising a second set of assets. [4] Method according to any one of claims 1 to 3, further comprising checking whether referenced assets in the recording file (212) are present in the asset vault (211). [5] Device (601) comprising: a memory (630) configured to store an asset vault (211) containing a group of assets captured by an application programming interface, API, a recording mechanism for use by an application (646), wherein the asset vault (211) is created in an initial phase of an application session; a processor (620) configured to initiate a capture process within the application session; a capture module configured to record API commands in a recording file (212) during the capture process, wherein the recording file (212) contains a reference to the asset vault (211); and a rendering module configured to produce one or more images or frames of rendered content using the recording file (212). [6] Device (601) according to claim 5, further comprising: an asset recognition module configured to detect changes to the assets during the capture process; and a delta data module configured to store changes in a delta data file separate from the asset vault, where optional: the device (601) further comprises a playback module configured to play back the recording file (212) by referencing the asset vault (211) and applying the delta data file to reproduce the captured assets expected for correct output, and / or The delta data module is further configured to detect new or modified graphics API objects and record them in the delta data file, and / or The device (601) further comprises a delta data module configured to create additional delta data files for subsequent acquisitions of the application session, each delta data file referencing the asset vault (211) to avoid duplication of unchanged assets. [7] Device (601) according to claim 5 or 6, wherein the creation of the asset memory (211) further comprises the creation of a first asset memory comprising a first set of assets and the creation of a second asset memory comprising a second set of assets. [8] Device (601) according to any one of claims 5 to 7, further comprising: a correlation module configured to check whether the assets referenced in the recording file (212) are present in the asset vault (211). [9] Non-transitory computer-readable storage medium which stores instructions which, when executed by a computer, cause the computer to execute a method according to any one of claims 1 to 4.

Images