Performance Monitoring Method, Device, Storage Medium, and Program Product

Abstract

Embodiments of the present application provide a performance monitoring method, device, storage medium, and program product. In these embodiments, a CPU runs first machine code of a scheduler that schedules data processing tasks. During the execution of the first machine code of the scheduler, machine code corresponding to a performance monitoring program is inserted into the first machine code, thereby implementing dynamic machine code instrumentation. This dynamic machine code instrumentation method eliminates the need to modify the scheduler script and can reduce the intrusiveness of the performance monitoring program on the scheduler.

Description

Technical Field

[0001] The present application relates to the field of performance monitoring technology, and in particular to a performance monitoring method, device, storage medium, and program product. Background Art

[0002] With the rapid development of artificial intelligence (AI) fields such as large language models (LLMs), the parameter size of AI (Artificial Intelligence) models is increasing, posing a challenge to computing resources. Parallel processing units (PPUs), such as graphics processing units (GPUs), often have large parallel processing capabilities and are widely used to perform AI model training or inference tasks.

[0003] However, parallel processing units (GPUs) such as GPUs face performance bottlenecks in computing power, significantly limiting the research and evolution of algorithmic models. Therefore, there is an urgent need to improve computing resource utilization amidst the overall computing resource shortage. By deeply understanding the performance of each task executed by a parallel processing unit, computing resources such as GPUs can be more rationally allocated to each task, ensuring that each parallel processing unit is fully utilized. This requires probing the performance data of each task on a parallel processing unit such as a GPU. However, existing performance probing methods are intrusive to user code. Summary of the Invention

[0004] Embodiments of the present application provide a performance monitoring method, device, storage medium, and program product to reduce the intrusiveness of performance detection on a scheduler of a data processing task.

[0005] An embodiment of the present application provides a performance monitoring method applicable to a central processing unit of an electronic device, the electronic device further comprising: at least one parallel processing unit; the central processing unit running a first machine code of a scheduler, the scheduler being configured to assign a data processing task to a target parallel processing unit among the at least one parallel processing unit;

[0006] The method comprises:

[0007] Obtaining a second machine code of a performance monitoring program; the performance monitoring program includes: an identifier of a performance analysis tool interface provided by the parallel processing unit; the performance analysis tool interface is used to collect performance data of an application program executing the data processing task in the target parallel processing unit;

[0008] During execution of the first machine code, the second machine code is inserted into the first machine code, so as to utilize the second machine code to obtain performance data of the application in the target parallel processing unit according to the identifier of the performance analysis tool interface.

[0009] An embodiment of the present application further provides an electronic device, comprising: a memory and a processor; wherein the memory is used to store computer programs; the processor comprises a central processing unit (CPU) and at least one parallel processing unit (PPU); the CPU runs a first machine code of a scheduler, wherein the scheduler is used to assign a data processing task to a target PPU among the at least one PPU;

[0010] The central processing unit is coupled to the memory and is configured to execute the computer program to perform the steps in the aforementioned performance monitoring method.

[0011] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, causes the one or more processors to execute the steps in the aforementioned performance monitoring method.

[0012] An embodiment of the present application further provides a computer program product, comprising a computer program, which, when executed by one or more processors, causes the one or more processors to execute the steps in the aforementioned performance monitoring method.

[0013] In an embodiment of the present application, a CPU runs a first machine code of a scheduler that schedules data processing tasks. During the execution of the first machine code of the scheduler, machine code corresponding to a performance monitoring program is inserted into the first machine code, thereby implementing dynamic machine code instrumentation. This dynamic machine code instrumentation method does not require modifying the scheduler script and can reduce the intrusiveness of the performance monitoring program on the scheduler. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0015] Figure 1 A flow chart of a performance monitoring method provided in an embodiment of the present application;

[0016] Figure 2 A schematic diagram of the internal structure of an electronic device provided in an embodiment of the present application;

[0017] Figure 3 A schematic diagram of the inter-process interaction process of the performance monitoring method provided in an embodiment of the present application;

[0018] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0020] It should be noted that when the embodiments of this application involve user information, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0021] The following is an explanation of the terms or concepts involved in the embodiments of the present application.

[0022] Artificial Intelligence (AI): AI is a technology that aims to automate tasks and make decisions by simulating human intelligent behavior through computers.

[0023] Extended Berkeley Packet Filter (eBPF): eBPF is a Linux (an operating system) kernel technology that allows users to monitor and dynamically modify the behavior of the operating system kernel in real time through secure and flexible programs at runtime without modifying the kernel source code.

[0024] Python: An interpreted object-oriented programming language, it is a commonly used programming language in current AI scenarios. For example, PyTorch and Tensorflow both provide Python-based programming interfaces by default.

[0025] TensorFlow: An open-source AI framework that provides a flexible and efficient set of tools and libraries for building, training, and deploying machine learning (ML) and AI models.

[0026] PyTorch: An open-source AI framework, widely used for ML and AI model development in academic research and industrial production due to its advantages in dynamic graph computing, ease of use, and flexibility.

[0027] Graphics Processing Unit (GPU): A type of hardware specialized for parallel computing that accelerates graphics rendering and complex computing tasks. It is widely used in areas such as gaming, AI, and scientific computing.

[0028] Compute Unified Device Architecture (CUDA): A parallel computing platform and programming model that allows developers to use GPUs to accelerate the execution of compute-intensive applications.

[0029] CUDA Profiling Tools Interface (CUPTI): A CUDA development tool that collects and analyzes performance data of CUDA applications to help developers optimize and debug.

[0030] Dynamic Binary Instrumentation (DBI): A technology that modifies the binary code of a program in real time and inserts analysis code while the program is running. It is used for performance analysis, debugging, and security checking without modifying the source code.

[0031] Nsight: A GPU performance monitoring tool. Nsight is a CUDA-level performance analysis tool. Nsight provides device-level performance analysis metrics that reveal in-depth details of GPU workloads. For example, Nsight can display GPU utilization, identify idle periods or overload conditions, analyze kernel scheduling and execution, including whether grid dimensions are properly set and whether stream concurrency fully utilizes GPU resources. It can also calculate the GPU instruction execution rate to assess performance bottlenecks and potential optimization areas.

[0032] However, Nsight requires modifying the startup method of applications that perform data processing tasks (such as AI tasks), which is somewhat invasive to the applications.

[0033] In some embodiments of the present application, a CPU runs first machine code of a scheduler that schedules data processing tasks. During the execution of the first machine code of the scheduler, machine code corresponding to a performance monitoring program is inserted into the first machine code, thereby implementing dynamic machine code instrumentation. This dynamic machine code instrumentation method eliminates the need to modify the scheduler script and reduces the intrusiveness of the performance monitoring program on the scheduler.

[0034] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0035] It should be noted that the same reference numerals represent the same objects or steps in the following drawings and embodiments. Therefore, once an object or a step is defined in one drawing or embodiment, it does not need to be further discussed in the subsequent drawings and embodiments.

[0036] Figure 1 Schematic diagram of the performance monitoring method provided in the embodiment of this application. Figure 1 As shown, the performance monitoring method includes:

[0037] 101. Obtain a second machine code of a performance monitoring program; the performance monitoring program includes: an identifier of a performance analysis tool interface provided by a parallel processing unit; the performance analysis tool interface is used to collect performance data of a data processing task in a target parallel processing unit.

[0038] 102. During the execution of the first machine code, insert the second machine code into the first machine code, so as to use the second machine code to obtain performance data of the application executing the data processing task in the target parallel processing unit according to the identifier of the performance analysis tool interface.

[0039] The performance monitoring method provided in the embodiment of the present application is mainly executed by the central processing unit (CPU) on the electronic device. Figure 2 As shown, the electronic device may include a CPU 10 and a parallel processing unit 20. The parallel processing unit 20 is a processor with parallel processing capabilities, and may be a GPU, a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC) chip. There may be one or more parallel processing units 20. "Multiple" means two or more (including two).

[0040] Due to its powerful computing power, the parallel processing unit 20 is often used to perform data processing tasks, such as AI model training tasks and / or AI model reasoning tasks. Among them, the AI model refers to a mathematical model constructed through machine learning methods, which can analyze data patterns and realize intelligent functions such as prediction and decision-making. The AI model training task refers to model training of the AI model. The AI model reasoning task refers to reasoning or prediction using the trained AI model. The driver and scheduler of the parallel processing unit 20 run on the CPU 10. The driver of the parallel processing unit 20 is used to drive the parallel processing unit 20 to operate. The scheduler is used to assign data processing tasks to the parallel processing unit 20. The machine code of the scheduler, such as binary code, runs on the CPU 10.

[0041] In this embodiment, if Figure 3 As shown, the insertion logic of the performance monitoring program can be initialized. For example, the Frida insertion logic can be initialized. Frida is a powerful dynamic code injection and instrumentation framework that is widely used in security research, reverse engineering, software testing and other fields. The initialization process of the insertion logic of the performance monitoring program includes loading the libraries that the performance monitoring program depends on, selecting the target process to be inserted (in the embodiment of the present application, the target process is the process that executes the scheduler), writing the performance monitoring program, and compiling the performance monitoring program.

[0042] The parallel processing unit 20 is provided with a performance analysis tool interface, through which performance data of data processing tasks on the parallel processing unit can be collected. For example, the performance analysis tool interface provided by some GPUs is CUPTI. Through the performance analysis tool interface (such as CUPTI, etc.), the kernel execution time, memory bandwidth usage, and number of instructions of the application performing the data processing task in the parallel processing unit can be obtained. Through the performance analysis tool interface (such as CUPTI, etc.), various performance events and counter data can also be collected from the parallel processing unit, which helps to identify performance bottlenecks. The performance analysis tool interface (such as CUPTI, etc.) can also be used to trace the application performing the data processing task, such as tracking the start time and completion time of a function in the application.

[0043] In some embodiments of the present application, to obtain performance data of an application executing a data processing task on a parallel processing unit, an identifier for a performance analysis tool interface (such as CUPTI) provided by the parallel processing unit can be set in the performance monitoring program. This allows the performance monitoring program to call the performance analysis tool interface and, through it, obtain performance data of the application executing the data processing task on the parallel processing unit.

[0044] In some embodiments, the performance analysis tool interface provided by the parallel processing unit can be pre-compiled into a library function. Among them, the library function refers to a pre-written function that can be reused by multiple programs. In this way, when the user writes the performance monitoring program, the library function corresponding to the performance analysis tool interface can be directly called in the performance monitoring program. Specifically, the function name of the library function corresponding to the performance analysis tool interface can be added in the performance monitoring program. Specifically, a programming interface can be provided, and the user can customize the insertion position of the performance monitoring program in the scheduler of the data processing task according to the needs, and add the function name of the library function corresponding to the performance analysis tool interface in the performance monitoring program to realize the user-defined performance monitoring program. In this way, the performance monitoring program can directly call the library function corresponding to the performance analysis tool interface during runtime to obtain the application program that performs the data processing task and the performance data in the parallel processing unit, which can reduce the development cost of the performance monitoring program.

[0045] In order to be able to call the library functions corresponding to the performance analysis tool interface, such as Figure 3 As shown, library functions corresponding to performance analysis tool interfaces, such as the CUPTI library, can be initialized. Specifically, the performance monitoring program can subscribe to CUPTI events, register a subscriber ID for the performance monitoring program, and define how the performance monitoring program handles notifications from performance analysis tool interfaces (such as CUPTI), thereby initializing the performance analysis tool interface.

[0046] In some embodiments, the performance monitoring program may be an eBPF program. The eBPF program allows users to monitor and dynamically modify the behavior of the operating system kernel in real time at runtime through a secure and flexible program, without modifying the kernel source code. This reduces the intrusion of the performance monitoring program into the scheduler. Because the eBPF program runs in kernel space, while the scheduler for data processing tasks runs in user space, in order to make the eBPF program pluggable into the user-space scheduler, the user-space extended Berkeley Packet Filter (uBFP) technology can be used to run the eBPF program in user space, enabling monitoring and analysis of the user-space scheduler.

[0047] In order to use uBPF technology, such as Figure 3 As shown, the uBPF runtime can be initialized. Specifically, the eBPF program's dependent libraries can be loaded, and a context object can be created for uBPF, which will manage the state of the entire runtime. Furthermore, the compiled eBPF program can be read and a built-in verifier can be used to check whether the eBPF program is safe and conforms to expected behavior. If the eBPF program is safe and conforms to expected behavior, the eBPF program is loaded into the uBPF virtual machine, completing the uBPF runtime initialization.

[0048] Using uBPF technology to run eBPF programs in user space allows them to monitor the user-space scheduler. This maintains the advantage of eBPF programs not requiring modifications to the scheduler, reducing the intrusiveness of performance monitoring programs on the scheduler. Furthermore, even if the performance monitoring program experiences an anomaly, it will not affect the scheduler.

[0049] In an embodiment of the present application, a dynamic binary instrumentation mechanism is introduced to reduce the intrusiveness of the performance monitoring program on the scheduler, enabling dynamic insertion of the performance monitoring program. The performance monitoring program is pre-compiled into machine code, i.e., binary code. In an embodiment of the present application, for ease of description and distinction, the machine code of the scheduler run by the CPU is defined as the first machine code, and the machine code of the performance monitoring program is defined as the second machine code.

[0050] In this embodiment, the target parallel processing unit refers to a parallel processing unit that performs data processing tasks, and the parallel processing unit runs an application that performs data processing tasks, such as an AI model training program, an AI model inference program, or an application that performs other data processing tasks.

[0051] In this embodiment, to reduce the intrusiveness of the performance monitoring program on the data processing task scheduler, in step 101, a second machine code of the performance monitoring program is obtained. In step 102, during the execution of the first machine code of the scheduler, the second machine code is inserted into the first machine code, thereby implementing dynamic machine code instrumentation. The machine code can be binary code, thus implementing dynamic binary code instrumentation. This dynamic machine code instrumentation method eliminates the need to modify the scheduler script and reduces the intrusiveness of the performance monitoring program on the scheduler.

[0052] On the other hand, since the performance monitoring program obtains the performance data of the application executing the data processing task on the target parallel processing unit based on the identifier of the performance analysis tool interface, the performance data of the application on the parallel processing unit obtained by the performance analysis tool interface can reflect the performance data of the parallel processing unit at the granularity of the data processing task, and therefore, more fine-grained performance monitoring can be achieved.

[0053] In some embodiments, when writing a performance monitoring program, the user may specify the location where the performance monitoring program is inserted in the scheduler. The insertion location may be the entry and / or exit of the target function in the scheduler. For example, Figure 3As shown, the insertion location is the entry and exit of the optimizer.step() function. optimizer.step() is the function name of a function in the scheduler. Accordingly, the performance monitoring program may include: the insertion location of the performance monitoring program in the scheduler. Accordingly, the second machine code may be inserted at the target location corresponding to the first machine code at the insertion location during the execution of the first machine code. For example, in some embodiments, the insertion location is the entry and exit of the target function of the scheduler, and the second machine code may be inserted at the target location corresponding to the first machine code at the entry and exit of the target function during the execution of the first machine code. Specifically, the second machine code may be inserted at the memory location of the entry and exit of the target function.

[0054] Among them, the performance monitoring program specifies the insertion position, and can directly insert the second machine code at the target position corresponding to the first machine code at the insertion position. The performance monitoring program can be directly inserted into the code segment of the first machine code, which can realize dynamic machine code insertion while reducing the difficulty of insertion.

[0055] In some embodiments, as Figure 3 As shown, during the execution of the first machine code, a jump instruction pointing to the second machine code can be inserted at the target location through process tracing (Ptrace), thereby inserting the second machine code at the target location. Ptrace is a system call interface that allows one process (commonly called a tracer or debugger) to control the execution of another process (the traced process or target process) and read and modify its memory and register state. Ptrace allows for instrumentation during the execution of the first machine code without accessing the source code of the first machine code. Because Ptrace operates at the operating system level, it does not directly modify the content of the first machine code. This ensures that even if instrumentation fails or encounters a problem, the scheduler can be restored to its original state, minimizing the impact on the scheduler and further reducing its intrusiveness to the scheduler.

[0056] Specifically, in the Ptrace technology, the process of executing the performance monitoring method (i.e. Figure 3 The first process in the list) is the parent process, which executes the scheduler's process (i.e. Figure 3 The parent process is used to control the child process; the parent process can observe and control the execution of its child process and allows the parent process to inspect and replace the kernel image (including register values) of the child process.

[0057] Furthermore, the parent process can be configured to dynamically instrument the child process's machine code using Ptrace technology. Specifically, the parent process can determine the memory address space corresponding to the target instrumentation location from the child process's memory address space and insert a jump instruction pointing to the second machine code into the memory address space, thereby inserting the second machine code at the target instrumentation location.

[0058] In other embodiments, the second machine code can be inserted at the target location during the execution of the first machine code through hot patching technology. Hot patching technology can dynamically replace or add code by modifying the memory image of the running program, thereby achieving dynamic instrumentation.

[0059] Specifically, the target process running the first machine code can be suspended, and the memory protection attribute at the target location can be changed to a writable mode. Subsequently, the second machine code can be written into the memory corresponding to the target location to insert the second machine code at the target location. Alternatively, a jump instruction for the second machine code can be written into the memory corresponding to the target location to insert the second machine code at the target location. Subsequently, the memory protection attribute corresponding to the target location can be restored to its original attribute, and the target process can be resumed.

[0060] In some embodiments, the second machine code may be machine code in a dynamic link library. Accordingly, during the execution of the first machine code, a jump instruction pointing to the second machine code in the dynamic link library may be inserted at the target location, thereby inserting the second machine code at the target location. Because the second machine code is machine code in the dynamic link library, when the first machine code is subsequently executed to the target location, the second machine code in the dynamic link library can be loaded into the process address space via the jump instruction, eliminating the need to store the second machine code locally. This can save local storage space and reduce the storage overhead of the performance monitoring program.

[0061] After the second machine code is dynamically inserted into the first machine code, the second machine code can be used to obtain the performance data of the application executing the data processing task in the target parallel processing unit according to the identifier of the performance analysis tool interface (such as CUPTI), thereby realizing performance monitoring at the data processing task level and providing more fine-grained performance monitoring.

[0062] Specifically, during the running of the first machine code, after the second machine code is dynamically inserted, the first machine code can continue to run normally. Figure 3As shown, when the first machine code runs to the target location, it jumps to the second machine code through the aforementioned jump instruction; thereafter, the second machine code can be executed, and during the execution of the second machine code, the performance data of the application program that performs the data processing task in the target parallel processing unit is obtained according to the identifier of the performance analysis tool interface.

[0063] In some embodiments, the performance monitoring program is an eBPF program. When the first machine code runs to the target location, it can jump to the second machine code through the aforementioned jump instruction; and execute the second machine code in the user space through the uBPF technology. Furthermore, in the process of executing the second machine code in the user space, the performance data of the application program that performs the data processing task in the target parallel processing unit can be obtained according to the identifier of the performance analysis tool interface. In this way, using uBPF technology to enable the eBPF program to run in the user space can enable the eBPF program to monitor the scheduler in the user space, and can continue the advantage of the eBPF program that there is no need to modify the monitored scheduler, which can reduce the intrusiveness of the performance monitoring program to the scheduler. On the other hand, even if the performance monitoring program is abnormal, it will not affect the scheduler.

[0064] In an embodiment where the performance analysis tool interface is compiled as a library function, the identifier of the performance analysis tool interface is the function name of the library function corresponding to the performance analysis tool interface. During the execution of the second machine code, the library function corresponding to the performance analysis tool interface can be called according to the function name of the library function corresponding to the performance analysis tool interface; and the library function corresponding to the performance analysis tool interface can be used to obtain performance data of the application program in the target parallel processing unit. Because the performance analysis tool interface is compiled as a library function, it is easier for users to set related schedules or dependencies when writing performance monitoring programs, reducing the difficulty and cost of developing user-defined performance monitoring programs.

[0065] After the second machine code completes execution, it can return to the target location of the first machine code to continue executing the first machine code. It can also delete the jump instruction from the target location, thereby stopping the execution of the second machine code, that is, stopping the execution of the performance monitoring program. This enables the performance monitoring program to be started and stopped on demand, which can improve the flexibility of performance monitoring. Furthermore, since the performance monitoring program can be started and stopped on demand, it does not need to be running all the time, which can reduce the processor and memory resource overhead of running the performance monitoring program.

[0066] In the embodiments of this application, Figure 3 As shown in "Data Export", after obtaining the performance data of the application executing the data processing task in the target parallel processing unit, the performance data of the application in the target parallel processing unit can also be exported to the performance analysis process ( Figure 3(Not shown). Specifically, the performance data of the application in the target parallel processing unit can be exported to the performance analysis process through the inter-process communication (IPC) mechanism. The IPC mechanism is a set of protocols and mechanisms provided by the operating system that allow different processes to communicate and synchronize with each other. Transmitting performance data through the IPC mechanism allows the performance analysis process to obtain performance metrics without affecting the normal operation of the application.

[0067] The performance analysis process can perform performance analysis on the application based on the application's performance data in the target PPU. For example, the performance analysis process can obtain performance metrics to be monitored, such as GPU utilization, memory bandwidth, and / or instruction throughput, from the application's performance data in the target PPU. The performance analysis process can also calculate statistics for these performance metrics, such as the average, maximum, minimum, and / or standard deviation. Furthermore, the performance metric statistics can be used to detect anomalies in the application.

[0068] For example, the performance analysis process can determine the code blocks that consume the most resources or execute the slowest based on the performance data of the application in the target parallel processing unit, and output the code blocks that consume the most resources or execute the slowest so that the application developers can optimize these code blocks.

[0069] In some embodiments, performance data of the application in the target parallel processing unit may also be stored. For example, the performance data of the application in the target parallel processing unit may be persistently stored to facilitate backtracking or analysis of some anomalies based on the performance data of the application in the target parallel processing unit.

[0070] To facilitate understanding of the above-mentioned performance monitoring method, the following specific description of the above-mentioned performance monitoring method is given by taking the parallel processing unit as a GPU, the performance analysis tool interface as CUPTI, and the data processing task as an AI task as an example. In some embodiments, a first process and a second process are running in the CPU. The first process is used to perform GPU performance detection; the second process is used to execute the first machine code of the scheduler of the data processing task. The AI task includes an AI model training task and / or an AI model reasoning task. The application is used to perform the AI model training task and / or the AI model reasoning task.

[0071] The first process can initialize the injection logic, CUPTI, and uBPF runtime of the performance monitoring program. The specific process by which the first process can initialize the injection logic, CUPTI, and uBPF runtime of the performance monitoring program can be found in the relevant content of the aforementioned embodiment and will not be repeated here. In this embodiment, the performance monitoring program is an eBPF program, which includes a CUPTI identifier, such as the function name of the CUPTI library function, and also includes the location of the performance monitoring program's insertion in the scheduler.

[0072] The first process can respond to the performance monitoring trigger operation to obtain the second machine code of the performance monitoring program; and during the execution of the first machine code, insert the second machine code at the target location corresponding to the first machine code at the insertion location through process tracking. The second process can continue to execute the first machine code, and trigger a jump instruction when it reaches the target location, jumping to the second machine code of the performance monitoring program. Afterwards, the first process can use uBPF technology to run the second machine code in user space, and schedule CUPTI during the execution of the second machine code; and collect performance data of the AI task application on the target GPU through CUPTI.

[0073] It should be noted that the execution entity of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution entity of steps 101 and 102 can be device A; for another example, the execution entity of step 101 can be device A, and the execution entity of step 102 can be device B; and so on.

[0074] In addition, some of the processes described in the above embodiments and the accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The sequence numbers of the operations, such as 101, 102, etc., are merely used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel.

[0075] Accordingly, an embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the performance monitoring method provided in the aforementioned embodiment.

[0076] The embodiment of the present application also provides a computer program product, including a computer program, which, when executed by one or more processors, causes the one or more processors to execute the steps of the performance monitoring method provided in the aforementioned embodiment. In the embodiment of the present application, the specific implementation form of the computer program product is not limited. In some embodiments, the computer program product can be implemented as an application (Application, APP), a mini-program, a computer-side client, a program module, a plug-in, an installation package, a software development kit (Software Development Kit, SDK), an image file of a CD (such as an ISO file), a plug-in or software in the form of Software as a Service (SaaS), etc., but is not limited thereto.

[0077] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 4 As shown, the electronic device includes a memory 40a and a processor 40b. Processor 40b includes a CPU 40b1 and at least one parallel processing unit 40b2. Memory 40a is used to store computer programs. CPU 40b1 runs first machine code of a scheduler, which is used to assign data processing tasks to target parallel processing units in the at least one parallel processing unit.

[0078] CPU40b1 is coupled to the memory 40a, and is used to execute a computer program for: obtaining a second machine code of a performance monitoring program; the performance monitoring program includes: an identifier of a performance analysis tool interface provided by the parallel processing unit; the performance analysis tool interface is used to collect performance data of an application program that performs data processing tasks in a target parallel processing unit; and, during the execution of the first machine code, inserting the second machine code into the first machine code to use the second machine code to obtain the performance data of the application program in the target parallel processing unit according to the identifier of the performance analysis tool interface.

[0079] In some embodiments, the performance monitoring program includes: an insertion position of the performance monitoring program in the scheduler; when CPU40b1 inserts the second machine code into the first machine code, it is specifically used to: during the execution of the first machine code, insert the second machine code at the insertion position at the target position corresponding to the first machine code.

[0080] Specifically, when CPU40b1 inserts the second machine code at the target position corresponding to the first machine code at the insertion position, it is specifically used to: during the execution of the first machine code, insert a jump instruction pointing to the second machine code at the target position through process tracking to insert the second machine code at the target position.

[0081] Optionally, the second machine code is machine code in a dynamic link library. When inserting the second machine code at the target location corresponding to the first machine code at the insertion position, CPU 40b1 is specifically configured to: during the execution of the first machine code, insert a jump instruction pointing to the second machine code in the dynamic link library at the target location, thereby inserting the second machine code at the target location.

[0082] In some embodiments, CPU 40b1 is further configured to: utilize the second machine code to obtain performance data of the application program in the target parallel processing unit according to an identifier of the performance analysis tool interface.

[0083] In some embodiments, the performance monitoring program is an eBPF program; the second machine code is inserted into the target location of the first machine code. Accordingly, when CPU 40b1 utilizes the second machine code to obtain performance data of the application in the target parallel processing unit based on the identifier of the performance analysis tool interface, the CPU 40b1 is specifically configured to: jump to the second machine code when the first machine code reaches the target location; execute the second machine code in user space using uBPF technology; and obtain performance data of the application in the target parallel processing unit based on the identifier of the performance analysis tool interface during the execution of the second machine code.

[0084] Optionally, CPU 40b1 is further configured to: after the execution of the second machine code is completed, return to the target location to continue executing the first machine code; and delete the jump instruction from the target location to stop running the second machine code.

[0085] In some embodiments, the performance analysis tool interface is compiled as a library function; the identifier of the performance analysis tool interface is the function name of the library function corresponding to the performance analysis tool interface. When CPU 40b1 obtains performance data of the application in the target parallel processing unit based on the identifier of the performance analysis tool interface, the CPU 40b1 is specifically configured to: during the execution of the second machine code, call the library function corresponding to the performance analysis tool interface based on the function name of the library function corresponding to the performance analysis tool interface; and use the library function to obtain performance data of the application in the target parallel processing unit.

[0086] Optionally, CPU 40b1 is further configured to export the performance data of the application in the target parallel processing unit to the performance analysis process through an inter-process communication mechanism.

[0087] In the electronic device provided in this embodiment, the CPU runs the first machine code of a scheduler that schedules data processing tasks. During the execution of the first machine code of the scheduler, the machine code corresponding to the performance monitoring program is inserted into the first machine code, implementing dynamic machine code instrumentation. This dynamic machine code instrumentation method eliminates the need to modify the scheduler script and reduces the intrusiveness of the performance monitoring program on the scheduler.

[0088] On the other hand, since the performance monitoring program obtains the performance data of the application executing the data processing task on the target parallel processing unit based on the identifier of the performance analysis tool interface, the performance data of the application on the parallel processing unit obtained by the performance analysis tool interface can reflect the performance data of the parallel processing unit at the granularity of the data processing task, and therefore, more fine-grained performance monitoring can be achieved.

[0089] In some optional embodiments, such as Figure 4 As shown, the electronic device may further include optional components such as a communication component 40c, a power component 40d, a display component 40e and an audio component 40f. Figure 4 Some components are shown schematically, and this does not mean that the electronic device must include Figure 4 All components shown do not mean that the electronic device can only include Figure 4 Components shown.

[0090] in addition, Figure 4 The components within the dashed box are optional, not mandatory, and may vary depending on the form factor of the electronic device. The electronic device of this embodiment can be implemented as a terminal device such as a desktop computer, laptop computer, mobile phone, or IoT device; or as a server device such as a traditional server, cloud server, or server cluster.

[0091] In an embodiment of the present application, the memory is used to store computer programs and can be configured to store various other data to support operations on the device in which it is located. The processor can execute the computer program stored in the memory to implement the corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), electrical programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0092] In the embodiments of the present application, the processor may be any hardware processing device capable of executing the logic of the above-described method. Optionally, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), or a microcontroller unit (MCU); may also be a programmable device such as a field-programmable gate array (FPGA), a programmable array logic device (PAL), a general array logic device (GAL), or a complex programmable logic device (CPLD); or may be an advanced reduced instruction set compute (RISC) processor (Advanced RISC Machines, ARM) or a system on chip (SoC), etc., but is not limited thereto.

[0093] In an embodiment of the present application, the communication component is configured to facilitate wired or wireless communication between the device in which it is located and other devices. The device in which the communication component is located can access a wireless network based on a communication standard, such as Wireless Fidelity (WiFi), 2G or 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can also be implemented based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology or other technologies.

[0094] In embodiments of the present application, the display assembly may include a liquid crystal display (LCD) and a touch panel (TP). If the display assembly includes a touch panel, the display assembly may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensors may not only sense the boundaries of a touch or slide action, but also the duration and pressure associated with the touch or slide action.

[0095] In embodiments of the present application, a power supply assembly is configured to provide power to various components of the device in which it is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0096] In an embodiment of the present application, the audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as call mode, recording mode, and voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component further includes a speaker for outputting audio signals. For example, for a device with a language interaction function, voice interaction with the user may be achieved through the audio component.

[0097] It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to different types.

[0098] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0099] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0102] In a typical configuration, a computing device includes one or more processors (CPU, etc.), input / output interfaces, network interfaces, and memory.

[0103] Memory may include non-permanent storage in a computer-readable medium, in the form of random-access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0104] Computer storage media is readable storage media, also referred to as "readable media." Readable storage media, including permanent and non-permanent, removable and non-removable media, can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0105] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, commodity, or apparatus comprising the aforementioned elements.

[0106] The above contents are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A performance monitoring method, applicable to a central processing unit of an electronic device, characterized in that: The electronic device further comprises: at least one parallel processing unit; the central processing unit runs a first machine code of a scheduler, the scheduler being configured to assign a data processing task to a target parallel processing unit in the at least one parallel processing unit; The method comprises: Obtaining a second machine code of a performance monitoring program; the performance monitoring program includes: an identifier of a performance analysis tool interface provided by a parallel processing unit; the performance analysis tool interface is used to collect performance data of an application executing the data processing task in the target parallel processing unit; the performance monitoring program is an extended Berkeley Packet Filter (eBPF) program; During the execution of the first machine code, the second machine code is inserted into the target position of the first machine code. When the first machine code runs to the target location, jumping to the second machine code; The second machine code is executed in the user space through the eBPF technology in the user space, and during the execution of the second machine code, the performance data of the application in the target parallel processing unit is obtained according to the identifier of the performance analysis tool interface.

2. The method according to claim 1, characterized in that The performance monitoring program includes: an insertion position of the performance monitoring program in the scheduler; and inserting the second machine code into the first machine code during the execution of the first machine code includes: During the execution of the first machine code, the second machine code is inserted at the insertion position at a target position corresponding to the first machine code.

3. The method according to claim 2, characterized in that The method of inserting the second machine code at the target position corresponding to the first machine code during the execution of the first machine code includes: During the execution of the first machine code, a jump instruction pointing to the second machine code is inserted at the target location in a process tracing manner, so as to insert the second machine code at the target location.

4. The method according to claim 2, characterized in that The second machine code is a machine code in a dynamic link library; and during the execution of the first machine code, inserting the second machine code at the insertion position at a target position corresponding to the first machine code includes: During the execution of the first machine code, a jump instruction pointing to the second machine code in the dynamic link library is inserted at the target location, so as to insert the second machine code at the target location.

5. The method according to claim 3 or 4, characterized in that The method further comprises: After the execution of the second machine code is completed, returning to the target location to continue executing the first machine code; and The jump instruction is deleted from the target location to stop executing the second machine code.

6. The method according to any one of claims 1 to 4, characterized in that The performance analysis tool interface is compiled into a library function; the identifier of the performance analysis tool interface is the function name of the library function corresponding to the performance analysis tool interface; The step of obtaining performance data of the application program in the target parallel processing unit according to the identifier of the performance analysis tool interface during the execution of the second machine code includes: During execution of the second machine code, calling the library function according to the function name of the library function corresponding to the performance analysis tool interface; The library function is used to obtain performance data of the application program in the target parallel processing unit.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The performance data of the application program in the target parallel processing unit is exported to a performance analysis process through an inter-process communication mechanism.

8. The method according to any one of claims 1 to 4, characterized in that The data processing tasks include: artificial intelligence (AI) model training tasks and / or AI model reasoning tasks.

9. An electronic device, characterized in that: include: A memory and a processor; wherein the memory is used to store computer programs; the processor includes a central processing unit and at least one parallel processing unit; the central processing unit runs a first machine code of a scheduler, and the scheduler is used to allocate data processing tasks to a target parallel processing unit in the at least one parallel processing unit; The central processing unit is coupled to the memory and is configured to execute the computer program to perform the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to perform the steps of the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that The method comprises a computer program which, when executed by one or more processors, causes the one or more processors to perform the steps of the method according to any one of claims 1 to 8.

Images