Method, apparatus and device for acquiring running state information of an application

Abstract

Embodiments of the present disclosure provide methods, apparatuses and devices for obtaining running state information of an application. An example method includes obtaining a configuration file, the configuration file including configuration data associated with at least one hardware collector, the at least one hardware collector being configured to collect running state information of the application; generating, based on the configuration file, code for obtaining the running state information of the application, the code being inserted into code of the application; and in response to the generated code being executed during a running of the application, obtaining the running state information of the application collected by the at least one hardware collector. In this way, the running state information of the application can be obtained and recorded in a flexible and efficient manner based on a customized configuration file, while the impact of state information tracking on the performance of the application can be reduced by utilizing a mechanism combined with hardware.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more specifically, to the field of data tracking technology. Embodiments of this disclosure relate to methods, apparatus, electronic devices, computer-readable storage media, and computer program products for obtaining runtime status information of an application. Background Technology

[0002] During the execution of high-performance applications, a low-load program state tracing tool is needed to help high-performance applications collect state information during the program's operation. Developers can combine this state information to help analyze application problems, performance bottlenecks, etc.

[0003] Hardware data collectors play a crucial role in this process. Leveraging their direct interaction with hardware, they can efficiently acquire critical state data of the underlying hardware during program execution. For example, they can accurately monitor the real-time utilization of each processor core. By collecting data on the load of different cores at various stages of application execution, developers can clearly determine whether there are uneven resource allocation issues when the application is processing tasks in parallel. Regarding memory usage, hardware data collectors can track memory read / write frequencies, cache hit rates, and other information in real time. This data provides crucial clues for analyzing whether applications have memory leaks or performance degradation due to frequent memory fragmentation. However, leveraging the advantages of hardware data collectors to collect state information under low load presents a challenge for program state tracing tools. Summary of the Invention

[0004] The embodiments of this disclosure provide a flexible, efficient, and low-load tracking scheme for operational status information.

[0005] According to a first aspect of this disclosure, a method for obtaining runtime status information of an application is provided. The method includes: obtaining a configuration file including configuration data associated with at least one hardware collector for collecting runtime status information of the application; generating code for obtaining the runtime status information of the application based on the configuration file, the code being inserted into the application's code; and obtaining the runtime status information of the application collected by the at least one hardware collector in response to the generated code being executed during application runtime. In this manner, runtime status information of the application can be obtained and recorded flexibly and efficiently based on a custom configuration file, while leveraging a hardware-integrated mechanism to reduce the performance impact of status information tracking on the application.

[0006] In some embodiments of the first aspect, the configuration data may include: identification information of at least one hardware collector selected from a plurality of hardware collectors; and the type of data collected for the at least one hardware collector. This approach makes the configuration data more specific and clear, facilitating more accurate selection and configuration of hardware collectors and determination of the type of data collected, thereby improving the relevance and accuracy of obtaining application runtime status information.

[0007] In some embodiments of the first aspect, generating code for obtaining application runtime status information may include: generating code for obtaining runtime status information corresponding to the data type from at least one hardware collector based on the identifier of at least one hardware collector and the data type being collected. This approach enables more targeted generation of code for obtaining specific types of runtime status information from specific hardware collectors, improving the accuracy and effectiveness of code generation and ensuring that the obtained runtime status information better meets requirements.

[0008] In some embodiments of the first aspect, obtaining application runtime information may include filtering runtime information collected by at least one hardware collector based on the type of data being collected. This approach removes unnecessary runtime information, retaining only information that meets the requirements of the data type being collected, thereby improving the quality and effectiveness of the obtained runtime information.

[0009] In some embodiments of the first aspect, the method may further include: obtaining an updated configuration file, the updated configuration file including updated data types of data being collected; and adjusting the filtering rules for runtime status information based on the updated data types of data being collected. This approach improves the system's adaptability to changes and ensures that the acquired runtime status information always meets the latest requirements.

[0010] In some embodiments of the first aspect, the multiple hardware acquisition units may include at least one of the following: a performance monitoring unit (PMU); a collaborative processing element (SPE); or a call stack. Based on this approach, more comprehensive application runtime information can be collected.

[0011] In some embodiments of the first aspect, the configuration data may include a log data format, which can define the storage format of runtime status information acquired during application runtime. This approach standardizes the storage of runtime status information, facilitating subsequent management and use of the stored information, and improving the standardization and manageability of data storage.

[0012] In some embodiments of the first aspect, generating code for obtaining application runtime status information may include: generating code for serializing application runtime status information based on log data format. This approach facilitates the storage, transmission, and subsequent analysis of runtime status information, improving the efficiency and standardization of runtime status information processing.

[0013] In some embodiments of the first aspect, the generated code may include application programming interface (API) code, and the method may further include providing a runtime library for the API code, the runtime library including an implementation of the API code. This approach enables the generated code to run more stably and efficiently, improving the reliability and performance of the code execution.

[0014] In some embodiments of the first aspect, acquiring application runtime information may include: starting a first background thread; acquiring application runtime information from the recording area of ​​at least one hardware collector through the application's process, and writing the acquired runtime information to a first storage medium; and reading runtime information from the first storage medium through the first background thread, and writing the runtime information to a second storage medium, wherein the read / write speed / performance of the first storage medium is higher than that of the second storage medium. Based on this approach, a separate thread can be used to record and store runtime information, reducing the impact on application runtime performance, and the performance characteristics of different storage media can be reasonably utilized to improve the efficiency of acquiring and storing runtime information.

[0015] In some embodiments of the first aspect, writing runtime information into the first storage medium may include writing the runtime information into the first storage medium using a lock-free circular buffer. This approach avoids the performance overhead caused by lock contention, ensuring that runtime information can be written to the first storage medium quickly and stably, thereby improving data writing performance.

[0016] In some embodiments of the first aspect, the method may further include: automatically refreshing the recording area of ​​at least one hardware acquisition device via a first background thread based on a preset time interval. This approach enables timely cleaning of the recording area, ensuring that the hardware acquisition device can continuously and effectively collect operational status information, preventing data backlog in the recording area from affecting the acquisition results, and improving the timeliness and accuracy of the collected data.

[0017] In some embodiments of the first aspect, the method may further include: receiving an instruction for updating a flow control policy of a first background thread, the flow control policy being used to control the rate at which runtime status information is written to a second storage medium; and updating the flow control policy of the first background thread based on the instruction. This approach improves the system's ability to control data write speed, avoiding any impact on system performance due to excessively fast or slow write speeds.

[0018] In some embodiments of the first aspect, the method may further include: updating a flow control strategy in response to the required speed for writing operational status information to the second storage medium exceeding a threshold, by reducing the amount of operational status information to be written to the second storage medium. Based on this approach, the amount of data written can be dynamically adjusted according to actual conditions while ensuring system performance, avoiding system performance degradation due to excessive data writing pressure, and improving system stability and adaptability.

[0019] In some embodiments of the first aspect, the first storage medium may include a processor cache, and the second storage medium may include memory. Based on this approach, the high-performance characteristics of the cache can be utilized to improve data retrieval speed, and the larger storage capacity of memory can be used for further data storage, thereby optimizing the system's storage architecture and performance.

[0020] In some embodiments of the first aspect, the method may further include: in response to the runtime status information stored in the second storage medium reaching a capacity threshold, starting a second background thread; and migrating the runtime status information from the second storage medium to a third storage medium, wherein the third storage medium is a persistent storage device, via the second background thread. Based on this approach, storage resources can be effectively managed, avoiding data loss or system performance degradation due to insufficient capacity of the second storage medium, and ensuring the long-term storage and accessibility of runtime status information.

[0021] In some embodiments of the first aspect, the runtime status information can be stored as a log file in a third storage medium, and the method may further include: reading the log file from the third storage medium; parsing the data format of the log file; organizing the parsed log file; and converting the organized log file into a text format. Based on this approach, the stored runtime status information can be presented in a more understandable and usable text format, facilitating subsequent analysis, processing, and display of the information, and improving the usability and analyzability of the runtime status information.

[0022] According to a second aspect of this disclosure, an electronic device is provided. The electronic device includes a processing unit and a memory, the processing unit executing instructions in the memory to cause the electronic device to perform a method, the method comprising: acquiring a configuration file including configuration data associated with at least one hardware collector, the at least one hardware collector being used to collect runtime status information of an application; generating code for acquiring the runtime status information of the application based on the configuration file, the code being inserted into the application's code; and acquiring the runtime status information of the application collected by the at least one hardware collector in response to the generated code being executed during application runtime.

[0023] According to a third aspect of this disclosure, an apparatus for acquiring runtime status information of an application is provided. The apparatus includes: a configuration file acquisition unit configured to acquire a configuration file including configuration data associated with at least one hardware collector, the at least one hardware collector being used to acquire runtime status information of the application; a code generation unit configured to generate code for acquiring runtime status information of the application based on the configuration file, the code being inserted into the application's code; and an information acquisition unit configured to acquire the runtime status information of the application collected by the at least one hardware collector in response to the generated code being executed during application runtime.

[0024] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores one or more computer instructions thereon, wherein one or more computer instructions, when executed by a processor, cause the processor to perform the method described according to a first aspect of this disclosure.

[0025] According to a fifth aspect of this disclosure, a computer program product is provided, including machine-executable instructions that, when executed by a device, cause the device to perform the method described according to a first aspect of this disclosure. Attached Figure Description

[0026] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0027] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0028] Figure 2 A schematic flowchart illustrating a process for obtaining runtime status information of an application according to an embodiment of the present disclosure is shown.

[0029] Figure 3 A schematic diagram of a multi-threaded operating state according to an embodiment of the present disclosure is shown;

[0030] Figure 4 A schematic flowchart of the code development phase according to an embodiment of the present disclosure is shown;

[0031] Figure 5 A schematic flowchart illustrating the business operation phases according to an embodiment of the present disclosure is shown;

[0032] Figure 6 A schematic diagram illustrating a process for modifying flow control according to an embodiment of the present disclosure is shown;

[0033] Figure 7 A schematic diagram illustrating the log parsing process according to an embodiment of the present disclosure is shown;

[0034] Figure 8 A schematic block diagram of an apparatus for obtaining application runtime status information according to embodiments of the present disclosure is shown; and

[0035] Figure 9 A schematic block diagram of an example device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0036] The technical solutions of this disclosure will now be described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments.

[0037] The technical solutions of the embodiments of this disclosure will now be described with reference to the accompanying drawings. In the description of the embodiments of this disclosure, unless otherwise stated, " / " signifies "and / or," for example, A / B can mean A or B, or A and B. "And / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Furthermore, in the description of the embodiments of this disclosure, "plural" or "multiple" refers to two or more than two.

[0038] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0039] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. As used in the specification and appended claims of this disclosure, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this disclosure, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0040] References to "one embodiment" or "some embodiments" as used in this specification mean that one or more embodiments of this disclosure include a particular feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "one embodiment," "some embodiments," "another embodiment," "other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0041] The numbers or values ​​used in this specification are illustrative and are intended only to facilitate understanding of the techniques of the embodiments of this disclosure, and are by no means intended to limit the scope of this disclosure.

[0042] Overview

[0043] During the execution of high-performance applications, a low-load program state tracing tool is needed to help these applications collect state information during runtime. Developers can then use this state information to analyze application problems and performance bottlenecks. The inventors noted that high-performance applications (e.g., applications on network interface cards) run at high speeds, generating a large amount of state information data during collection. Efficiently storing this state information data while minimizing its impact on application performance presents a significant challenge for program state tracing tools.

[0044] Several state tracing tools have adopted a full-stack event tracing architecture that covers both kernel and user space. By setting up numerous tracing points in both kernel and user space, they comprehensively capture various events from underlying hardware drivers to upper-level applications, building a complete view of the system's runtime state and providing comprehensive data support for in-depth analysis of system behavior. These tools include the Long Term Trace Toolkit NextGeneration (LTTng), SystemTap, and Dynamic Tracing (DTrace). Taking LTTng as an example, it also builds an asynchronous event collection system that can be flexibly configured according to user needs. Users can customize parameters such as the types and frequencies of events to be traced based on specific application scenarios and analysis objectives. During event collection, an asynchronous mechanism is used to separate event capture and storage from the normal operation of the system, avoiding excessive impact on system performance and ensuring stable event tracing even under high load. However, although LTTng's design goal is to minimize performance impact, it still lacks the ability to integrate hardware to reduce performance overhead during high loads or large-scale event generation.

[0045] When tracking application runtime status information, hardware data acquisition devices, such as Performance Monitoring Units (PMUs) and Special Processing Elements (SPEs), offer significant advantages in data acquisition, processing capabilities, and collaboration with software. Taking the PMU as an example, located at the hardware level, it can directly monitor and collect data from underlying hardware events, such as CPU clock cycles, instruction execution status, and cache access. Compared to software acquisition, which requires instruction calls and system calls to obtain information, the PMU can acquire this data in real time without software intervention, avoiding the additional instruction execution overhead and context switching overhead that can occur with software acquisition. Furthermore, the PMU operates independently of the CPU's normal operational logic, possessing its own independent hardware circuitry and control logic. During data acquisition, it minimally interferes with the CPU's normal application execution flow. It can perform performance data acquisition in parallel with little or no impact on CPU execution efficiency, unlike software acquisition which may affect application performance due to CPU resource contention.

[0046] Other solutions (such as the DPDK trace library) embed trace points in the packet processing path, delving into each stage of the packet processing flow to accurately record the complete path of a packet from entering the application to leaving. The DPDK trace library, by setting callback functions, triggers data sampling when specific events occur, collecting only critical, performance-related data. This avoids collecting too much useless information that could lead to performance degradation, providing developers with effective performance analysis data in a lightweight manner. However, because it provides relatively little collected information, it often requires secondary development by users, resulting in low tuning efficiency.

[0047] Given that existing methods for obtaining application runtime status information suffer from problems such as lack of hardware integration capabilities, high load, and insufficient types of information provided, embodiments of this disclosure offer a low-load, highly configurable solution for obtaining application runtime status information. By leveraging custom configuration capabilities and hardware integration capabilities to track application runtime status information, compared to existing technologies, it can record and save application runtime status information in a flexible and efficient manner, while reducing the performance impact of tracking runtime status information.

[0048] Example Environment

[0049] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. It should be understood that... Figure 1 The environment 100 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the implementation described in this disclosure. Figure 1 As shown, the example environment 100 may include an application layer 110, a system layer 120, and a hardware layer 130. In the example environment 100, the application layer 110, system layer 120, and hardware layer 130 are interconnected and work collaboratively. The application layer 110 is the layer that directly interacts with the user, primarily responsible for providing various specific application functions. The system layer 120 is located between the application layer 110 and the hardware layer 130, primarily responsible for managing and coordinating resources within the example environment 100, providing a stable and efficient operating environment for the application layer 110. The hardware layer 130 is the foundation of the example environment 100, providing physical support for the application layer 110 and the system layer 120.

[0050] In some implementations, the application layer 110 may include business applications 112 (hereinafter referred to as "applications") and log tracing tools 114. Business applications 112 may be high-performance applications requiring performance tuning, including but not limited to application software on internet network interface cards (NICs), database management software, and video encoding / decoding software. Taking application software on internet NICs as an example, after performance tuning, it can cope with problems such as data packet loss and increased latency in high-concurrency scenarios (e.g., promotional activities on large e-commerce platforms, online live streaming, etc.). Furthermore, it can reduce system resource consumption (e.g., CPU, memory, etc.) and improve system stability and reliability.

[0051] Log tracing tool 114 is the primary tool for developers and operations personnel to interact with business applications 112 via logs. In some implementations, log tracing tool 114 can generate a tracing interface 112-2 and a runtime tracing library 112-4 based on a user-provided configuration file. The tracing interface 112-2 is a set of predefined functions or methods, typically provided to developers as an application programming interface (API). It bridges the communication gap between the application and the tracing system, allowing developers to insert tracing code at critical points in the application to obtain various information about the application's runtime. The runtime tracing library 112-4 is a software library containing various tracing functions and tools. It is tightly integrated with the application's runtime environment, providing underlying support and implementation for the tracing interface 112-2. The runtime tracing library typically includes modules for data acquisition, data processing, data storage, and data transmission, enabling effective processing and management of the data collected by the tracing interface 112-2. In some implementations, log tracing tool 114 can also parse and organize the collected logs.

[0052] The inventors noted that a processor cache (referred to herein as L0 memory) also exists in hardware layer 130, with a speed between registers and general memory, and can be invoked based on specific drivers. In some implementations, system layer 120 may include L0 driver 122 and perf (performance) event logging area 124. Hardware layer 130 may include L0 memory 132, hardware collector 134, and DDR memory 136. The runtime tracing library 112-4 of application layer 110 may include code implementation of L0 driver 122 to control L0 memory 132. Using L0 memory 132 as a primary storage medium for log tracing based on L0 driver 122 can accelerate read and write speeds and reduce system overhead. Memory (e.g., double-rate SDRAM, DDR memory), which is slower but has a larger capacity than L0 memory, can be used to temporarily store log information.

[0053] In some implementations, the perf tool can be used to collect various performance events in the system. The perf tool is a performance analysis tool that helps developers and system administrators gain a deeper understanding of system performance bottlenecks. When using the perf tool to monitor various performance events in the system, the data collected and associated with the hardware collector 134 is stored in the perf event log area 124. The runtime tracing library 112-4 may include corresponding code implementations to collect system information from the perf event log area 124.

[0054] The hardware acquisition unit 134 may include a performance monitoring unit (PMU), a coprocessor element (SPE), a processor (PROC), a call stack, etc. The PMU internally contains multiple hardware counters that can count specific events. For example, a CPU clock cycle counter records the number of clock cycles elapsed since CPU startup, thereby measuring CPU operating time. In addition, the PMU also contains an instruction execution counter, which counts the number of instructions executed, helping to understand the CPU's computational load. The PMU can monitor various hardware events, such as cache hits and misses, branch prediction successes and failures, etc. When these events occur, the PMU's monitoring logic triggers corresponding operations, recording the event information into specific registers or buffers.

[0055] System Processing Entities (SPEs) typically have their own independent processing capabilities and data acquisition mechanisms. When performing specific tasks, they record their own operational status information, such as the progress of computational tasks and intermediate results of data processing. SPEs interact with the main processor through communication mechanisms such as buses, sending the collected system status information to the main processor or shared memory area for unified processing and analysis by the main processor.

[0056] The registers inside the PROC store critical information such as the currently executing instruction, operands, and program counter. By reading the values ​​of these registers, the current working state and execution progress of the PROC can be obtained. In addition, when an interrupt or exception occurs in the system, the PROC saves relevant status information to specific registers or memory areas. This information includes the interrupt source and the program context at the time of the interrupt, which helps to understand the system's running state at a specific moment.

[0057] During a function call, the processor creates a stack frame for each function call on the stack. The stack frame contains information such as the function's parameters, local variables, and return address. By analyzing the stack frame structure, the call relationships and execution path can be understood. When an exception occurs, the processor records the current call stack information for analysis during exception handling. This information is very helpful in locating program errors and performance issues.

[0058] In the above process, code is automatically generated using a user-provided configuration file, utilizing the processor's cache (e.g., L0 memory) as the primary storage medium, and combined with a hardware collector for log tracing. Compared to traditional solutions that lack hardware integration capabilities or have low configurability, the embodiments of this disclosure achieve low load, high configurability, and high collection efficiency.

[0059] Collect running status information

[0060] The following is for reference Figures 2 to 9 Embodiments of this disclosure are further described below.

[0061] Figure 2 A schematic flowchart of a process 200 for obtaining runtime status information of an application according to an embodiment of the present disclosure is shown. It should be understood that method 200 may also include additional actions not shown and / or the actions shown may be omitted, and the scope of the present disclosure is not limited in this respect.

[0062] like Figure 2 As shown in box 210, method 200 obtains a configuration file, which includes configuration data associated with at least one hardware collector used to collect application runtime information. The configuration file can be provided by the user and the configuration data can be customized according to actual needs. For example, for web applications, users may be more concerned with information such as server-side CPU usage, memory usage, and network traffic, as well as client-side page load time and resource loading status. For database applications, users may be more concerned with information such as cache hit rate and SQL statement execution time. In some embodiments, the configuration data may also include log data formatting for later recording and parsing.

[0063] Given the advantages of hardware data acquisition devices in data acquisition, processing capabilities, and software integration, configuration data can include identification information for at least one hardware data acquisition device selected from a plurality of hardware data acquisition devices to determine which hardware data acquisition devices should be used to collect data. Furthermore, the configuration data may also include the data type for at least one hardware data acquisition device to further clarify acquisition requirements and improve the quality of collected data. In some embodiments, once the application is running, the acquisition requirements can be modified based on a new configuration file that includes different data acquisition data types.

[0064] In box 220, method 200 generates code for acquiring application runtime status information based on a configuration file, and this code is inserted into the application's code. The code may include tracing interface code and its concrete implementation (e.g., a runtime library). Users can insert the tracing interface code into appropriate locations in the application's source code to collect runtime status information. In some embodiments, the generated code may also include L0 driver code to implement calls to L0 memory. In some embodiments, the generated code may also include serialization code. By serializing the collected runtime status information, storage resource consumption can be reduced, thereby improving data acquisition efficiency.

[0065] In box 230, method 200, in response to the generated code being executed during application runtime, acquires application runtime status information collected by at least one hardware collector. The runtime status information can be obtained from the recording area of ​​the hardware collector and dumped to DDR memory via L0 memory. To further reduce the load, in some embodiments, the acquisition of runtime status information can be performed by starting a resident background thread (referred to as a first background thread). The first background thread can have a preset flow control strategy and can be modified by the user during application runtime to avoid the impact of write speed on system performance.

[0066] A lock-free circular buffer is a buffer structure that enables efficient data reading and writing in a multi-threaded or multi-process environment without the need for lock mechanisms. A circular buffer is a fixed-size buffer that logically forms a ring. When data is written to or read from the end of the buffer, it automatically returns to the beginning to continue the operation. Lock-free means that in a multi-threaded or multi-process environment, read and write operations do not rely on traditional locks (such as mutexes or semaphores) to ensure data consistency and concurrency safety. Optionally, runtime state information can be written to L0 memory based on the lock-free circular buffer approach to avoid the performance overhead of lock contention and improve data writing performance.

[0067] If the runtime status information exceeds the memory capacity threshold, a temporary thread (which may be called a second background thread) can be started to dump the runtime status information to persistent storage media (e.g., hard disk). In some embodiments, the runtime status information in persistent storage media can also be parsed offline, independently of the application's main process, to further reduce the load.

[0068] Figure 3 A schematic diagram of a multi-threaded operating state 300 according to an embodiment of the present disclosure is shown. Figure 3As shown, runtime state 300 may include business thread 310, resident background thread 320, and temporary thread 330. After the user inserts code generated based on the configuration file and starts the application, business thread 310 and resident background thread 320 start simultaneously. In business thread 310, runtime state information can be collected from the perf event logging area 342 and the call stack 314, and log assembly 312 can be performed. Then, the assembled log can be serialized based on the serialization code in the inserted code 316. The serialized runtime log can be written to the lock-free circular buffer 344 in L0 memory 340.

[0069] In the resident background thread 320, the event loop 328 can be entered, which automatically receives logs 322 and temporarily stores them in DDR memory 350, automatically retrieves the latest perf event records 326 from the perf event record area 342, and determines whether log dumping is needed 324. If it is determined that log dumping is needed 324, a temporary thread 330 is started to write logs to disk 332, so as to migrate the logs in DDR memory 350 to the hard disk 360 for persistent storage.

[0070] Figure 4 A schematic flowchart of a code development phase 400 according to an embodiment of the present disclosure is shown. Figure 4 As shown, for developers, after completing the detailed design of the application code 402, they can simultaneously write the business logic code 404 and prepare a configuration file 406, including the hardware collector and the data types to be collected, and provide it to the log tracing tool. Upon receiving the configuration file, the log tracing tool can parse the log format 408 to facilitate subsequent management and use of the stored information. Based on the developer-defined configuration file 406, the log tracing tool can generate interface code 410 and serialization code 412, and further generate a log interface header file 414. After obtaining the generated code, developers can choose appropriate locations to insert the log interface 416 to collect runtime status information at the corresponding times. Developers can compile the business logic code with the inserted log tracing code into an executable file 418, thereby enabling automatic log collection during application runtime.

[0071] Figure 5 A schematic flowchart of a service operation phase 500 according to an embodiment of the present disclosure is shown. Figure 5As shown, after starting the program 502, a business thread 504 and a background persistent thread 518 can be created. During operation, the business thread can trigger a logging point 506 based on the interface code inserted by the developer, and then filter the collected status information at the log level 508 based on the collection type in the configuration file, retaining only the information required by the developer. Next, the business thread can assemble 510 and serialize 512 the log content before sending it to L0 memory for processing by the background thread, thereby reducing load and improving data processing efficiency. After sending the log information, the process can return to the business flow 516.

[0072] On the other hand, the background persistent thread can first create a memory channel 520, then enter the event loop 522 to begin the log collection process. After the business thread writes the logs to L0 memory, the background persistent thread can receive log events 524 and begin log reception 526, retrieving logs from L0 memory. Then, the background persistent thread can determine whether its flow control strategy will cause traffic overload 528. If it will cause overload, flow control processing 530 is performed to adjust the speed of log migration from L0 memory. If it will not cause overload, log information is written to DDR memory for temporary log storage 532. Then, the background persistent thread can determine whether log dumping is necessary 534 based on the log capacity in DDR memory. If the logs exceed the capacity threshold, log dumping 536 is performed, migrating the log data from DDR memory to the hard drive for persistent storage. If the logs do not exceed the capacity threshold, it returns to continue receiving log events 524.

[0073] Figure 6 A schematic diagram of a process 600 for changing flow control according to an embodiment of the present disclosure is shown. Figure 6 As shown, in step S601, user 602 can modify the flow control policy when log writes are full and provide the updated flow control policy to log tracing tool 604. If no measures are taken when log writes are full, newly generated log data will not be able to be written, which may lead to data loss, which is extremely detrimental to subsequent data analysis. In step S602, log tracing tool 604 can send the received new flow control policy to background resident thread 606, and then in step 603, background resident thread 606 can update itself based on the new flow control policy to alleviate the log write full situation. It is worth noting that the above process of updating the flow control policy can be completed online without stopping the application.

[0074] Optionally, modifications to the flow control policy may include lowering the logging level, temporarily ignoring some non-critical information logs and debug logs to reduce the amount of logs generated. Optionally, modifications to the flow control policy may also include traffic shaping, limiting the number of logs written per second. For example, a token bucket algorithm or a leaky bucket algorithm can be used to implement flow control, ensuring that the log writing rate remains within an acceptable range.

[0075] In addition to adjusting flow control strategies online, users can also modify filtering rules offline in step S604. Specifically, user 602 can upload a configuration file containing the new data collection type to the log tracing tool 604. Then, in step S605, the log tracing tool 604 can generate tracing code containing new filtering rules based on the new configuration file. In step S606, when the new tracing code is run, the background resident thread 606 can update the filtering rules corresponding to the new configuration file uploaded by the user to meet the user's needs.

[0076] Figure 7 A schematic block diagram of a log parsing process 700 according to an embodiment of the present disclosure is shown. Figure 7 As shown, in step S701, user 702 can issue a command to log tracing tool 704 to read the log file. Then, log tracing tool 704 can read the binary log file from hard disk 706 in step S702, loading the binary data into memory to prepare for subsequent parsing. In step S703, log tracing tool 704 can parse the log format. Log tracing tool 704 can first identify the log file format by the file extension, header information, or other identifiers, such as a custom binary format, a standard log format (e.g., JSON, XML), or other specific formats. Then, according to the determined format, it parses the binary data according to the corresponding rules. For custom binary formats, it may be necessary to parse the data according to a predefined structure or protocol; for standard formats, a corresponding parsing library can be used. The parsed data can easily extract the required log fields, such as timestamps, log levels, and log content. These fields will be used for subsequent processing and output.

[0077] In step S704, the log tracing tool 704 can organize the log stream to make the log information clearer and more orderly, facilitating subsequent analysis and viewing. For example, the log information can be sorted according to the log timestamp to ensure that the logs are arranged in chronological order. Another example is that related log information can be aggregated to statistically analyze error logs of the same type. Then, in step S705, the log tracing tool 704 can output the log information in text format for user 702 to view and analyze.

[0078] Exemplary devices and implementations

[0079] Figure 8 A schematic block diagram of an apparatus 800 for acquiring application runtime status information according to an embodiment of the present disclosure is shown. The apparatus 800 may be implemented in example environment 100. The apparatus 800 includes a configuration file acquisition unit 810, a code generation unit 820, and an information acquisition unit 830. The configuration file acquisition unit 810 is configured to acquire a configuration file including configuration data associated with at least one hardware collector for acquiring application runtime status information; the code generation unit 820 is configured to generate code for acquiring application runtime status information based on the configuration file, and the code is inserted into the application's code; the information acquisition unit 830 is configured to acquire the application runtime status information acquired by the at least one hardware collector in response to the generated code being executed during application runtime.

[0080] In some embodiments, the configuration data may include: identification information of at least one hardware acquisition device selected from a plurality of hardware acquisition devices; and the acquisition data type for at least one hardware acquisition device.

[0081] In some embodiments, the code generation unit 820 may also be configured to generate code for obtaining running status information corresponding to the data type of the data acquisition from at least one hardware acquisition device, based on the identifier of at least one hardware acquisition device and the data acquisition data type.

[0082] In some embodiments, the information acquisition unit 830 may also be configured to filter the operating status information acquired by at least one hardware acquisition device based on the type of data to be acquired.

[0083] In some embodiments, the apparatus 800 may further include a filtering rule update unit configured to acquire an updated configuration file, the updated configuration file including an updated data type of data being collected; and to adjust the filtering rules for the running status information based on the updated data type of data being collected.

[0084] In some embodiments, the plurality of hardware acquisition units may include at least one of the following: a performance monitoring unit (PMU); a cooperative processing element (SPE); or a call stack.

[0085] In some embodiments, configuration data may include a log data format, which may define the storage format for runtime status information acquired during application runtime.

[0086] In some embodiments, the code generation unit 820 can also be configured to generate code for serializing application runtime status information based on log data format.

[0087] In some embodiments, the generated code may include application programming interface (API) code, and the apparatus 800 may further include a runtime library providing unit configured to provide a runtime library for the API code, the runtime library including an implementation of the API code.

[0088] In some embodiments, the information acquisition unit 830 may also be configured to start a first background thread; acquire the application's running status information from the recording area of ​​at least one hardware collector through the application's process, and write the acquired running status information into a first storage medium; and read the running status information from the first storage medium through the first background thread, and write the running status information into a second storage medium, wherein the read / write speed / performance of the first storage medium is higher than that of the second storage medium.

[0089] In some embodiments, the information acquisition unit 830 may also be configured to write running status information into the first storage medium based on a lock-free circular buffer.

[0090] In some embodiments, the device 800 may further include a recording area refresh unit, configured to automatically refresh the recording area of ​​at least one hardware acquisition device based on a preset time interval via a first background thread.

[0091] In some embodiments, the apparatus 800 may further include a flow control update unit configured to receive an instruction for updating a flow control policy of a first background thread, the flow control policy being used to control the rate at which runtime status information is written to a second storage medium; and to update the flow control policy of the first background thread based on the instruction.

[0092] In some embodiments, the flow control update unit may also be configured to update the flow control policy by reducing the amount of running status information to be written to the second storage medium in response to the required rate of writing running status information to the second storage medium exceeding a threshold.

[0093] In some embodiments, the first storage medium may include a processor cache (e.g., L0 memory), and the second storage medium may include memory.

[0094] In some embodiments, the apparatus 800 may further include a dump unit configured to initiate a second background thread in response to the operation status information stored in the second storage medium reaching a capacity threshold; and to migrate the operation status information from the second storage medium to a third storage medium, wherein the third storage medium is a persistent storage device, via the second background thread.

[0095] In some embodiments, the running status information may be stored as a log file in a third storage medium, and the apparatus 800 may further include a log parsing unit configured to read the log file from the third storage medium; parse the data format of the log file; organize the parsed log file; and convert the organized log file into a text format.

[0096] Figure 9 A schematic block diagram of an example device 900 that can be used to implement embodiments of the present disclosure is shown. As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 902 or loaded from a storage unit 906 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 may also be stored in the RAM 903. The computing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0097] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as a touchscreen, keyboard, mouse, etc.; output unit 907, such as various types of displays (e.g., interactive displays, such as touchscreens), speakers, etc.; storage unit 908, such as a disk, optical disk, etc.; and communication unit 909, such as a network interface card (NIC), modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0098] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform method 200 by any other suitable means (e.g., by means of firmware).

[0099] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0100] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0101] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0102] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0103] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0104] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0105] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0106] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0107] Various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for obtaining application runtime status information, comprising: Obtain a configuration file, which includes configuration data associated with at least one hardware collector used to collect application running status information; Based on the configuration file, code for obtaining the running status information of the application is generated, and the code is inserted into the application's code. as well as In response to the generated code being executed during the operation of the application, the application's runtime status information is obtained.

2. The method according to claim 1, wherein the configuration data includes: Identification information of the at least one hardware collector selected from a plurality of hardware collectors; as well as The type of data acquired by the at least one hardware acquisition device.

3. The method according to claim 2, wherein generating the code for obtaining the running status information of the application includes: Based on the identifier of the at least one hardware collector and the data type of the data being collected, code is generated for obtaining running status information corresponding to the data type of the data being collected from the at least one hardware collector.

4. The method according to claim 3, wherein obtaining the application's running status information includes: The operating status information collected by the at least one hardware collector is filtered based on the data type of the collected data.

5. The method according to claim 4, further comprising: Obtain the updated configuration file, which includes the updated data types of the collected data; as well as Based on the updated data type of the collected data, the filtering rules for the running status information are adjusted.

6. The method of claim 2, wherein the plurality of hardware acquisition devices comprises at least one of the following: Performance monitoring unit (PMU); Synergistic Processing Element (SPE); or Call stack.

7. The method according to claim 1, wherein the configuration data includes a log data format, the log data format defining the storage format of runtime status information acquired during the operation of the application.

8. The method of claim 7, wherein generating the code for obtaining the running status information of the application comprises: Based on the log data format, code is generated for serializing the running status information of the application.

9. The method of claim 1, wherein the generated code includes application programming interface (API) code, and the method further comprises: A runtime library is provided for the API code, the runtime library including the implementation of the API code.

10. The method according to claim 1, wherein obtaining the application's running status information includes: Start the first background thread; The application's running status information is obtained from the recording area of ​​the at least one hardware collector through the application's process, and the obtained running status information is written into the first storage medium. as well as The first background thread reads the running status information from the first storage medium and writes the running status information into the second storage medium. The read / write speed / performance of the first storage medium is higher than that of the second storage medium.

11. The method of claim 10, wherein writing the operating status information into the first storage medium comprises: The running status information is written to the first storage medium using a lock-free circular buffer.

12. The method of claim 10, further comprising: The first background thread automatically refreshes the recording area of ​​the at least one hardware collector based on a preset time interval.

13. The method of claim 10, further comprising: Receive an instruction for updating the flow control policy of the first background thread, the flow control policy being used to control the speed at which the running status information is written to the second storage medium; as well as Based on the instruction, update the flow control policy of the first background thread.

14. The method of claim 13, further comprising: In response to the speed required to write the operational status information to the second storage medium exceeding a threshold, the flow control policy is updated by reducing the amount of operational status information to be written to the second storage medium.

15. The method of claim 10, wherein the first storage medium includes a processor cache, and the second storage medium includes memory.

16. The method of claim 10, further comprising: In response to the running status information stored in the second storage medium reaching a capacity threshold, a second background thread is started; as well as The running status information is migrated from the second storage medium to the third storage medium, which is a persistent storage device, through the second background thread.

17. The method of claim 16, wherein the running status information is stored as a log file in the third storage medium, and the method further comprises: Read the log file from the third storage medium; The data format of the log file is parsed; The parsed log files are then organized; as well as Convert the sorted log file into text format.

18. An apparatus for acquiring application runtime status information, comprising: The configuration file acquisition unit is configured to acquire a configuration file, which includes configuration data associated with at least one hardware collector, the at least one hardware collector being used to collect application running status information. The code generation unit is configured to generate code for obtaining the running status information of the application based on the configuration file, and the code is inserted into the application's code; as well as The information acquisition unit is configured to acquire the application's runtime status information collected by the at least one hardware collector in response to the generated code being executed during the application's operation.

19. An electronic device comprising: processor; as well as A memory storing one or more computer instructions, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 17.

20. A computer-readable storage medium having stored thereon one or more computer instructions, wherein the one or more computer instructions are executed by a processor to cause the processor to perform the method according to any one of claims 1 to 17.

21. A computer program product comprising machine-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1 to 17.

Images