Performance monitoring method and device, storage medium and program product

Abstract

The embodiment of the invention provides a performance monitoring method and device, a storage medium and a program product. In the embodiment of the invention, the CPU runs the first machine code of the scheduling program for scheduling the data processing task, and the machine code corresponding to the performance monitoring program is inserted into the first machine code in the running process of the first machine code of the scheduling program, so that dynamic machine code instrumentation is realized. The dynamic machine code instrumentation mode does not need to modify a script of a scheduling program, and the invasiveness of a performance monitoring program to the scheduling program can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of performance monitoring, and in particular to a performance monitoring method, device, storage medium and program product. Background Art

[0002] With the rapid development of artificial intelligence fields such as Large Language Models (LLMs), the parameter scale of artificial intelligence (AI) models is increasing, which poses a challenge to computing resources. Parallel processing units such as graphics processing units (GPUs) often have large parallel processing capabilities and are widely used to perform training or reasoning tasks of AI models.

[0003] However, parallel processing units such as GPUs have performance bottlenecks in computing power, which greatly limits the research and evolution of algorithm models. Therefore, there is an urgent need to improve the utilization rate of computing resources in the context of overall computing resource shortages. By deeply understanding the performance of each task executed by a parallel processing unit, computing resources such as GPUs can be more reasonably allocated to tasks to ensure that each parallel processing unit can be fully utilized. This requires detecting the performance data of each task in parallel processing units such as GPUs. However, existing performance detection methods are invasive to user code. Summary of the invention

[0004] Embodiments of the present application provide a performance monitoring method, device, storage medium, and program product to reduce the intrusiveness of performance detection on a scheduling program of a data processing task.

[0005] The embodiment of the present application provides a performance monitoring method, which is applicable to a central processing unit of an electronic device, wherein the electronic device further comprises: at least one parallel processing unit; the central processing unit runs a first machine code of a scheduler, wherein the scheduler is used to assign a data processing task to a target parallel processing unit in the at least one parallel processing unit; The method comprises: Acquire a second machine code of a performance monitoring program; the performance monitoring program includes: an identifier of a performance analysis tool interface provided by a parallel processing unit; the performance analysis tool interface is used to collect performance data of an application program executing the data processing task in the target parallel processing unit; During the execution of the first machine code, the second machine code is inserted into the first machine code, so as to obtain the performance data of the application in the target parallel processing unit by using the second machine code according to the identifier of the performance analysis tool interface.

[0006] The embodiment of the present application also provides an electronic device, comprising: a memory and a processor; wherein the memory is used to store a computer program; the processor comprises a central processing unit and at least one parallel processing unit; the central processing unit runs a first machine code of a scheduler, and the scheduler is used to allocate a data processing task to a target parallel processing unit in the at least one parallel processing unit; The central processing unit is coupled to the memory and is used to execute the computer program to perform the steps in the aforementioned performance monitoring method.

[0007] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the aforementioned performance monitoring method.

[0008] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by one or more processors, causes the one or more processors to execute the steps in the aforementioned performance monitoring method.

[0009] In an embodiment of the present application, the CPU runs a first machine code of a scheduler that schedules data processing tasks. During the operation of the first machine code of the scheduler, the machine code corresponding to the performance monitoring program is inserted into the first machine code, thereby implementing dynamic machine code plugging. This dynamic machine code plugging method does not require modifying the script of the scheduler, and can reduce the intrusiveness of the performance monitoring program to the scheduler. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of a performance monitoring method according to an embodiment of the present invention; Figure 2 A schematic diagram of the internal structure of an electronic device provided in an embodiment of the present application; Figure 3 A schematic diagram of the process interaction between processes of the performance monitoring method provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0012] It should be noted that, in the case where the embodiments of the present application involve user information, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0013] The following is an explanation of the terms or concepts involved in the embodiments of the present application.

[0014] Artificial Intelligence (AI): AI is a technology that aims to automate tasks and make decisions by simulating human intelligent behavior through computers.

[0015] Extended Berkeley Packet Filter (eBPF): eBPF is a Linux (an operating system) kernel technology that allows users to monitor and dynamically modify the behavior of the operating system kernel in real time through secure and flexible programs at runtime without modifying the kernel source code.

[0016] Python: An interpreted object-oriented programming language, it is a commonly used programming language in current AI scenarios. For example, PyTorch and Tensorflow both provide Python-based programming interfaces by default.

[0017] TensorFlow: An open source AI framework that provides a set of flexible and efficient tools and libraries for building, training, and deploying machine learning (ML) and AI models.

[0018] PyTorch: An open source AI framework that is widely used in ML and AI model development in academic research and industrial production for its advantages of dynamic graph computing, ease of use, and flexibility.

[0019] Graphics Processing Unit (GPU): A type of hardware specialized for parallel computing that accelerates graphics rendering and complex computing tasks and is widely used in areas such as gaming, AI, and scientific computing.

[0020] Compute Unified Device Architecture (CUDA): A parallel computing platform and programming model that allows developers to use GPUs to accelerate the execution of compute-intensive applications.

[0021] CUDA Profiling Tools Interface (CUPTI): A development tool for CUDA that collects and analyzes performance data of CUDA applications to help developers optimize and debug.

[0022] Dynamic Binary Instrumentation (DBI): A technology that modifies the binary code of a program in real time and inserts analysis code while the program is running. It is used for performance analysis, debugging, and security checking without modifying the source code.

[0023] Nsight: A GPU performance monitoring tool. Nsight is a performance analysis tool for CUDA. Nsight can provide device-level performance analysis indicators that can reveal in-depth details of GPU workloads. For example, Nsight can display GPU utilization, identify whether there are idle periods or overload conditions, analyze kernel scheduling and execution, including whether the grid dimension settings are reasonable and whether stream concurrency fully utilizes GPU resources, and calculate the GPU instruction execution rate to evaluate performance bottlenecks and potential optimization space.

[0024] However, Nsight needs to modify the way applications that perform data processing tasks (such as AI tasks) are started, which is somewhat invasive to the applications.

[0025] In some embodiments of the present application, the CPU runs a first machine code of a scheduler that schedules data processing tasks. During the running of the first machine code of the scheduler, the machine code corresponding to the performance monitoring program is inserted into the first machine code, thereby implementing dynamic machine code plugging. This dynamic machine code plugging method does not require modifying the script of the scheduler, and can reduce the intrusiveness of the performance monitoring program to the scheduler.

[0026] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0027] It should be noted that the same reference numerals represent the same object or the same step in the following drawings and embodiments, and therefore, once an object or a step is defined in one drawing or embodiment, there is no need to further discuss it in the subsequent drawings and embodiments.

[0028] Figure 1 The following is a flow chart of the performance monitoring method provided in the embodiment of the present application. Figure 1 As shown, the performance monitoring method includes: 101. Obtain a second machine code of a performance monitoring program; the performance monitoring program includes: an identifier of a performance analysis tool interface provided by a parallel processing unit; the performance analysis tool interface is used to collect performance data of a data processing task in a target parallel processing unit.

[0029] 102. During the execution of the first machine code, insert the second machine code into the first machine code, so as to obtain performance data of the application program executing the data processing task in the target parallel processing unit by using the second machine code according to the identifier of the performance analysis tool interface.

[0030] The performance monitoring method provided in the embodiment of the present application is mainly executed by a central processing unit (CPU) on the electronic device. Figure 2 As shown, the electronic device may include a CPU 10 and a parallel processing unit 20. The parallel processing unit 20 refers to a processor with parallel processing capabilities, which may be a GPU, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC) chip. There may be one or more parallel processing units 20. Multiple means more than two (including two).

[0031] The parallel processing unit 20 is often used to perform data processing tasks, such as AI model training tasks and / or AI model reasoning tasks, due to its powerful computing power. Among them, the AI ​​model refers to a mathematical model constructed by a machine learning method, which can analyze data patterns and realize intelligent functions such as prediction and decision-making. The AI ​​model training task refers to model training for the AI ​​model. The AI ​​model reasoning task refers to reasoning or prediction using the trained AI model. The driver and scheduler of the parallel processing unit 20 are run on the CPU 10. The driver of the parallel processing unit 20 is used to drive the parallel processing unit 20 to act. The scheduler is used to assign data processing tasks to the parallel processing unit 20. The machine code of the scheduler, such as binary code, is run on the CPU 10.

[0032] In this embodiment, if Figure 3 As shown, the insertion logic of the performance monitoring program can be initialized. For example, the Frida insertion logic can be initialized. Among them, Frida is a powerful dynamic code injection and instrumentation framework, which is widely used in security research, reverse engineering, software testing and other fields. The initialization process of the insertion logic of the performance monitoring program includes loading the library that the performance monitoring program depends on, selecting the target process to be inserted (in the embodiment of the present application, the target process is the process of executing the scheduler), writing the performance monitoring program, and compiling the performance monitoring program.

[0033] The parallel processing unit 20 is provided with a performance analysis tool interface, through which the performance data of the data processing task on the parallel processing unit can be collected. For example, the performance analysis tool interface provided by some GPUs is CUPTI. Through the performance analysis tool interface (such as CUPTI, etc.), the kernel execution time, memory bandwidth usage, and number of instructions of the application program that performs the data processing task in the parallel processing unit can be obtained. Through the performance analysis tool interface (such as CUPTI, etc.), various performance events and counter data can also be collected from the parallel processing unit, which helps to identify performance bottlenecks. Through the performance analysis tool interface (such as CUPTI, etc.), the application program that performs the data processing task can also be traced (Tracing), such as tracking the start time and completion time of a function in the application program.

[0034] In some embodiments of the present application, in order to obtain the performance data of the application executing the data processing task on the parallel processing unit, the identifier of the performance analysis tool interface (such as CUPTI) provided by the parallel processing unit can be set in the performance monitoring program. In this way, the performance monitoring program can call the performance analysis tool interface and obtain the performance data of the application executing the data processing task in the parallel processing unit through the performance analysis tool interface.

[0035] In some embodiments, the performance analysis tool interface provided by the parallel processing unit can be pre-compiled into a library function. Among them, the library function refers to a pre-written function that can be reused by multiple programs. In this way, when the user writes the performance monitoring program, the library function corresponding to the performance analysis tool interface can be directly called in the performance monitoring program. Specifically, the function name of the library function corresponding to the performance analysis tool interface can be added in the performance monitoring program. Specifically, a programming interface can be provided, and the user can customize the insertion position of the performance monitoring program in the scheduler of the data processing task according to the needs, and add the function name of the library function corresponding to the performance analysis tool interface in the performance monitoring program to implement a user-defined performance monitoring program. In this way, the performance monitoring program can directly call the library function corresponding to the performance analysis tool interface when it is running to obtain the application program that performs the data processing task and the performance data in the parallel processing unit, which can reduce the development cost of the performance monitoring program.

[0036] In order to be able to call the library functions corresponding to the performance analysis tool interface, such as Figure 3 As shown, the library function corresponding to the performance analysis tool interface can be initialized, such as initializing the CUPTI library. Specifically, the performance monitoring program can subscribe to the CUPTI event, register a subscriber identifier (Identify, ID) for the performance monitoring program, and define the performance monitoring program to handle notifications from the performance analysis tool interface (such as CUPTI, etc.), etc., to achieve the initialization of the performance analysis tool interface.

[0037] In some embodiments, the performance monitoring program may be an eBPF program. The eBPF program allows users to monitor and dynamically modify the behavior of the operating system kernel in real time through a safe and flexible program at runtime, without modifying the kernel source code, which can reduce the intrusion of the performance monitoring program into the scheduler. Since the eBPF program runs in kernel space, and the scheduler for data processing tasks runs in user space. In order to make the eBPF program insertable into the user space scheduler, the user-space extended Berkeley Packet Filter (User-space eBPF, uBFP) technology can be used to run the eBPF program in user space, so as to monitor and analyze the user space scheduler.

[0038] In order to use uBPF technology, such as Figure 3 As shown, the uBPF runtime can be initialized. Specifically, the dependent libraries of the eBPF program can be loaded, and a context object can be created for uBPF, which will manage the state of the entire runtime. Further, the compiled eBPF program can be read, and the built-in verifier can be used to check whether the eBPF program is safe and conforms to the expected behavior. If the eBPF program is safe and conforms to the expected behavior, the eBPF program is loaded into the uBPF virtual machine, thereby realizing the uBPF runtime initialization.

[0039] Using uBPF technology to run eBPF programs in user space allows eBPF programs to monitor the scheduler in user space and continue the advantage of eBPF programs not needing to modify the monitored scheduler, which can reduce the intrusiveness of performance monitoring programs on the scheduler. On the other hand, even if the performance monitoring program is abnormal, it will not affect the scheduler.

[0040] In the embodiment of the present application, in order to reduce the intrusiveness of the performance monitoring program to the scheduler, a dynamic binary plugging mechanism is introduced to realize the dynamic insertion of the performance monitoring program. The performance monitoring program is pre-compiled into machine code, that is, binary code. In the embodiment of the present application, for the convenience of description and distinction, the machine code of the scheduler run by the CPU is defined as the first machine code; and the machine code of the performance monitoring program is defined as the second machine code.

[0041] In this embodiment, the target parallel processing unit refers to a parallel processing unit that performs data processing tasks, and the parallel processing unit runs an application that performs data processing tasks, such as an AI model training program, an AI model reasoning program, or an application that performs other data processing tasks.

[0042] In this embodiment, in order to reduce the intrusiveness of the performance monitoring program to the scheduler of the data processing task, in step 101, the second machine code of the performance monitoring program can be obtained, and in step 102, during the execution of the first machine code of the scheduler, the second machine code is inserted into the first machine code, thereby realizing dynamic machine code plugging. The machine code can be a binary code, thus realizing dynamic binary code plugging. This dynamic machine code plugging method does not require modifying the script of the scheduler, and can reduce the intrusiveness of the performance monitoring program to the scheduler.

[0043] On the other hand, since the performance monitoring program obtains the performance data of the application executing the data processing task on the target parallel processing unit according to the identifier of the performance analysis tool interface, the performance data of the application on the parallel processing unit obtained by the performance analysis tool interface can reflect the performance data of the parallel processing unit at the granularity of the data processing task. Therefore, more fine-grained performance monitoring can be achieved.

[0044] In some embodiments, when writing a performance monitoring program, the user may specify the insertion location of the performance monitoring program in the scheduler. The insertion location may be the entry and / or exit of the target function in the scheduler. For example, Figure 3As shown, the insertion location is the entry and exit of the optimizer.step() function. optimizer.step() is the function name of a function in the scheduler. Accordingly, the performance monitoring program may include: the insertion location of the performance monitoring program in the scheduler. Accordingly, the second machine code may be inserted at the insertion location at the target location corresponding to the first machine code during the execution of the first machine code. For example, in some embodiments, the insertion location is the entry and exit of the target function of the scheduler, and the second machine code may be inserted at the target location corresponding to the first machine code at the entry and exit of the target function during the execution of the first machine code. Specifically, the second machine code may be inserted at the memory location of the entry and exit of the target function.

[0045] Among them, the performance monitoring program specifies the insertion position, and the second machine code can be directly inserted at the target position corresponding to the first machine code at the insertion position. The performance monitoring program can be directly inserted into the code segment of the first machine code, which can reduce the difficulty of insertion while realizing dynamic machine code insertion.

[0046] In some embodiments, Figure 3 As shown, during the running process of the first machine code, a jump instruction pointing to the second machine code can be inserted at the target location by process tracing (ProcessTrace, Ptrace), thereby inserting the second machine code at the target location. Ptrace is a system call interface that allows a process (usually called a tracer or debugger) to control the execution of another process (the traced process or the target process) and read and modify its memory and register state. Ptrace can be used to insert a stub while the first machine code is running without accessing the source code of the first machine code. Since the operation of Ptrace is performed at the operating system level, the content of the first machine code will not be directly modified. This ensures that even if the stub fails or a problem occurs, the scheduler can be restored to the original state, reducing the impact on the scheduler, and therefore, the invasiveness to the scheduler can be further reduced.

[0047] Specifically, in the Ptrace technology, the process of executing the performance monitoring method (i.e. Figure 3 The first process in the list) is the parent process, and the process that executes the scheduler (i.e. Figure 3 The parent process is used to control the child process; the parent process can observe and control the execution of its child process, and allows the parent process to check and replace the value of the child process's kernel image (including registers).

[0048] Furthermore, the parent process can be set to dynamically insert machine code into the child process through the Ptrace technology. Specifically, the parent process can determine the memory address space corresponding to the target insertion position from the memory address space of the child process, and insert a jump instruction pointing to the second machine code in the memory address space, thereby inserting the second machine code at the target insertion position.

[0049] In other embodiments, the second machine code may be inserted at the target location by using a hot patching technique during the execution of the first machine code, wherein the hot patching technique may dynamically replace or add code by modifying the memory image of the running program to implement dynamic insertion.

[0050] Specifically, the target process running the first machine code may be suspended, and the memory protection attribute at the target location may be changed to a writable mode. Afterwards, the second machine code may be written into the memory corresponding to the target location to insert the second machine code into the target location. Alternatively, a jump instruction of the second machine code may be written into the memory corresponding to the target location to insert the second machine code into the target location. Afterwards, the memory protection attribute corresponding to the target location may be restored to the original attribute, and the target process may be resumed.

[0051] In some embodiments, the second machine code may be a machine code in a dynamic link library. Accordingly, during the execution of the first machine code, a jump instruction pointing to the second machine code in the dynamic link library may be inserted at the target location, thereby inserting the second machine code at the target location. Since the second machine code is a machine code in the dynamic link library, when the first machine code is subsequently executed to the target location, the second machine code in the dynamic link library may be loaded into the process address space through the jump instruction, without the need to store the second machine code locally, thus saving local storage space and reducing the storage overhead of the performance monitoring program.

[0052] After the second machine code is dynamically inserted into the first machine code, the second machine code can be used to obtain the performance data of the application executing the data processing task in the target parallel processing unit according to the identifier of the performance analysis tool interface (such as CUPTI), thereby realizing performance monitoring at the data processing task level and providing more fine-grained performance monitoring.

[0053] Specifically, during the running of the first machine code, after the second machine code is dynamically inserted, the first machine code can continue to run normally. Figure 3As shown, when the first machine code runs to the target position, it jumps to the second machine code through the aforementioned jump instruction; thereafter, the second machine code can be executed, and during the execution of the second machine code, the performance data of the application program that executes the data processing task in the target parallel processing unit is obtained according to the identifier of the performance analysis tool interface.

[0054] In some embodiments, the performance monitoring program is an eBPF program. When the first machine code runs to the target location, the jump instruction can be used to jump to the second machine code; and the second machine code can be executed in the user space through the uBPF technology. Furthermore, in the process of executing the second machine code in the user space, the performance data of the application program that performs the data processing task in the target parallel processing unit can be obtained according to the identifier of the performance analysis tool interface. In this way, the use of uBPF technology allows the eBPF program to run in the user space, which enables the eBPF program to monitor the user space scheduler, and can continue the advantage of the eBPF program that there is no need to modify the monitored scheduler, which can reduce the intrusiveness of the performance monitoring program to the scheduler. On the other hand, even if the performance monitoring program is abnormal, it will not affect the scheduler.

[0055] For the embodiment in which the performance analysis tool interface is compiled as a library function, the identifier of the performance analysis tool interface is the function name of the library function corresponding to the performance analysis tool interface. Then, during the execution of the second machine code, the library function corresponding to the performance analysis tool interface can be called according to the function name of the library function corresponding to the performance analysis tool interface; and the performance data of the application in the target parallel processing unit can be obtained by using the library function corresponding to the performance analysis tool interface. Since the performance analysis tool interface is compiled as a library function, it is convenient for users to set related scheduling or dependencies when writing performance monitoring programs, reducing the difficulty and cost of developing user-defined performance monitoring programs.

[0056] After the second machine code is executed, it can return to the target location of the first machine code to continue executing the first machine code, and can also delete the jump instruction from the target location, thereby stopping the second machine code, that is, stopping the performance monitoring program, and realizing the on-demand startup and shutdown of the performance monitoring program, which can improve the flexibility of performance monitoring. On the other hand, the performance monitoring program is started and closed on demand, and there is no need to run the performance monitoring program all the time, which can reduce the overhead of processor resources and memory resources for running the performance monitoring program.

[0057] In the embodiments of the present application, Figure 3 As shown in "Data Export", after obtaining the performance data of the application executing the data processing task in the target parallel processing unit, the performance data of the application in the target parallel processing unit can also be exported to the performance analysis process ( Figure 3(not shown). Specifically, the performance data of the application in the target parallel processing unit can be exported to the performance analysis process through the inter-process communication (IPC) mechanism. The IPC mechanism is a set of protocols and mechanisms provided by the operating system that allow different processes to communicate and synchronize with each other. The performance data is transmitted through the IPC mechanism, so that the performance analysis process can obtain performance indicators without affecting the normal operation of the application.

[0058] The performance analysis process can perform performance analysis on the application according to the performance data of the application in the target parallel processing unit. For example, the performance analysis process can obtain the performance indicators to be monitored, such as GPU utilization, memory bandwidth and / or instruction throughput, from the performance data of the application in the target parallel processing unit; and can also calculate the statistics of these performance indicators, such as the average, maximum, minimum and / or standard deviation. Furthermore, anomaly detection can be performed on the application according to the statistics of the performance indicators.

[0059] For another example, the performance analysis process can determine the code blocks that consume the most resources or execute the slowest based on the performance data of the application in the target parallel processing unit, and output the code blocks that consume the most resources or execute the slowest so that the application developers can optimize these code blocks.

[0060] In some embodiments, the performance data of the application in the target parallel processing unit may also be stored, such as persistent storage of the performance data of the application in the target parallel processing unit, so as to facilitate backtracking or analysis of some exceptions based on the performance data of the application in the target parallel processing unit.

[0061] To facilitate understanding of the above performance monitoring method, the following takes the parallel processing unit as GPU, the performance analysis tool interface as CUPTI, and the data processing task as AI task as an example to specifically illustrate the above performance monitoring method. In some embodiments, a first process and a second process are running in the CPU. Among them, the first process is used to perform GPU performance detection; the second process is used to execute the first machine code of the scheduler of the data processing task. Among them, the AI ​​task includes an AI model training task and / or an AI model reasoning task. The application is used to perform an AI model training task and / or an AI model reasoning task.

[0062] The first process can initialize the injection logic, CUPTI and uBPF runtime of the performance monitoring program. The specific process of the first process initializing the injection logic, CUPTI and uBPF runtime of the performance monitoring program can refer to the relevant content of the aforementioned embodiment, which will not be repeated here. In this embodiment, the performance monitoring program is an eBPF program, which includes the CUPTI identifier, such as the function name of the CUPTI library function, and also includes the plug-in position of the performance monitoring program in the scheduler.

[0063] The first process can respond to the performance monitoring trigger operation to obtain the second machine code of the performance monitoring program; and during the execution of the first machine code, insert the second machine code at the target position corresponding to the first machine code at the insertion position through process tracking. The second process can continue to execute the first machine code, and trigger a jump instruction when it runs to the target position, jumping to the second machine code of the performance monitoring program. Afterwards, the first process can use uBPF technology to run the second machine code in user space, and schedule CUPTI during the execution of the second machine code; and collect performance data of the application of the AI ​​task on the target GPU through CUPTI.

[0064] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 and 102 can be device A; for another example, the execution subject of step 101 can be device A, and the execution subject of step 102 can be device B; and so on.

[0065] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel, and the serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.

[0066] Accordingly, an embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the performance monitoring method provided in the aforementioned embodiment.

[0067] The embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by one or more processors, the one or more processors are caused to execute the steps in the performance monitoring method provided in the aforementioned embodiment. In the embodiment of the present application, the specific implementation form of the computer program product is not limited. In some embodiments, the computer program product can be implemented as an application (Application, APP), a mini-program, a computer-side client, a program module, a plug-in, an installation package, a software development kit (Software Development Kit, SDK), an image file of a CD (such as an ISO file), a plug-in or software in the form of software as a service (Software as a Service, SaaS), etc., but is not limited to this.

[0068] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device includes: a memory 40a and a processor 40b. The processor 40b includes: a CPU 40b1 and at least one parallel processing unit 40b2. The memory 40a is used to store computer programs. The CPU 40b1 runs a first machine code of a scheduler, and the scheduler is used to assign data processing tasks to target parallel processing units in the at least one parallel processing unit.

[0069] CPU40b1 is coupled to the memory 40a, and is used to execute a computer program for: obtaining a second machine code of a performance monitoring program; the performance monitoring program includes: an identifier of a performance analysis tool interface provided by the parallel processing unit; the performance analysis tool interface is used to collect performance data of an application program that performs data processing tasks in a target parallel processing unit; and, during the execution of the first machine code, inserting the second machine code into the first machine code, so as to use the second machine code to obtain the performance data of the application program in the target parallel processing unit according to the identifier of the performance analysis tool interface.

[0070] In some embodiments, the performance monitoring program includes: an insertion position of the performance monitoring program in the scheduling program; when CPU40b1 inserts the second machine code into the first machine code, it is specifically used to: during the execution of the first machine code, insert the second machine code at the insertion position at the target position corresponding to the first machine code.

[0071] Specifically, when CPU40b1 inserts the second machine code at the target position corresponding to the first machine code at the insertion position, it is specifically used to: during the execution of the first machine code, insert a jump instruction pointing to the second machine code at the target position through process tracking to insert the second machine code at the target position.

[0072] Optionally, the second machine code is a machine code in a dynamic link library. When CPU40b1 inserts the second machine code at the target position corresponding to the first machine code at the insertion position, it is specifically used to: during the execution of the first machine code, at the target position, insert a jump instruction pointing to the second machine code in the dynamic link library to insert the second machine code at the target position.

[0073] In some embodiments, CPU 40b1 is further used to: use the second machine code to obtain performance data of the application in the target parallel processing unit according to the identifier of the performance analysis tool interface.

[0074] In some embodiments, the performance monitoring program is an eBPF program; the second machine code is inserted into the target position of the first machine code. Accordingly, when CPU40b1 uses the second machine code to obtain the performance data of the application in the target parallel processing unit according to the identifier of the performance analysis tool interface, it is specifically used to: when the first machine code runs to the target position, jump to the second machine code; execute the second machine code in the user space through the uBPF technology, and during the execution of the second machine code, obtain the performance data of the application in the target parallel processing unit according to the identifier of the performance analysis tool interface.

[0075] Optionally, CPU 40b1 is further configured to: after the execution of the second machine code is completed, return to the target location to continue executing the first machine code; and delete the jump instruction from the target location to stop running the second machine code.

[0076] In some embodiments, the performance analysis tool interface is compiled into a library function; the identifier of the performance analysis tool interface is the function name of the library function corresponding to the performance analysis tool interface. When CPU40b1 obtains the performance data of the application in the target parallel processing unit according to the identifier of the performance analysis tool interface, it is specifically used to: during the execution of the second machine code, according to the function name of the library function corresponding to the performance analysis tool interface, call the library function corresponding to the performance analysis tool interface; and use the library function to obtain the performance data of the application in the target parallel processing unit.

[0077] Optionally, CPU 40b1 is also used to export the performance data of the application in the target parallel processing unit to the performance analysis process through an inter-process communication mechanism.

[0078] In the electronic device provided in this embodiment, the CPU runs a first machine code of a scheduler for scheduling data processing tasks. During the execution of the first machine code of the scheduler, the machine code corresponding to the performance monitoring program is inserted into the first machine code, thereby implementing dynamic machine code plugging. This dynamic machine code plugging method does not require modifying the script of the scheduler, and can reduce the intrusiveness of the performance monitoring program to the scheduler.

[0079] On the other hand, since the performance monitoring program obtains the performance data of the application executing the data processing task on the target parallel processing unit according to the identifier of the performance analysis tool interface, the performance data of the application on the parallel processing unit obtained by the performance analysis tool interface can reflect the performance data of the parallel processing unit at the granularity of the data processing task. Therefore, more fine-grained performance monitoring can be achieved.

[0080] In some optional embodiments, such as Figure 4 As shown, the electronic device may also include optional components such as a communication component 40c, a power component 40d, a display component 40e and an audio component 40f. Figure 4 The components are shown schematically only, and do not mean that the electronic device must include Figure 4 The components shown do not necessarily mean that the electronic device can only include Figure 4 Components shown.

[0081] in addition, Figure 4 The components in the dashed box are optional components, not mandatory components, and may depend on the product form of the electronic device. The electronic device of this embodiment may be implemented as a terminal device such as a desktop computer, a laptop computer, a mobile phone, or an IoT device; or may be various server devices such as a traditional server, a cloud server, or a server cluster.

[0082] In an embodiment of the present application, the memory is used to store a computer program and can be configured to store various other data to support operations on the device where it is located. Among them, the processor can execute the computer program stored in the memory to implement the corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random-Access Memory, SRAM), electrically erasable programmable read only memory (Electrically Erasable Programmable Read Only Memory, EEPROM), erasable programmable read only memory (Electrical Programmable Read Only Memory, EPROM), programmable read only memory (Programmable Read Only Memory, PROM), read only memory (Read Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0083] In the embodiment of the present application, the processor may be any hardware processing device that can execute the logic of the above method. Optionally, the processor may be a central processing unit (CPU), a graphics processing unit (GPU) or a microcontroller unit (MCU); it may also be a field programmable gate array (FPGA), a programmable array logic device (PAL), a general array logic device (GAL), a complex programmable logic device (CPLD) or other programmable devices; or it may be an advanced reduced instruction set (RISC) processor (Advanced RISC Machines, ARM) or a system on chip (System on Chip, SoC), etc., but not limited thereto.

[0084] In an embodiment of the present application, the communication component is configured to facilitate wired or wireless communication between the device in which it is located and other devices. The device in which the communication component is located can access a wireless network based on a communication standard, such as Wireless Fidelity (WiFi), 2G or 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can also be implemented based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology or other technologies.

[0085] In an embodiment of the present application, the display component may include a liquid crystal display (LCD) and a touch panel (TP). If the display component includes a touch panel, the display component may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0086] In an embodiment of the present application, a power supply component is configured to provide power to various components of the device in which it is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0087] In an embodiment of the present application, the audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal. For example, for a device with a language interaction function, voice interaction with a user may be achieved through an audio component.

[0088] It should be noted that the descriptions such as “first” and “second” in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit “first” and “second” to different types.

[0089] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program codes.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0093] In a typical configuration, a computing device includes one or more processors (CPU, etc.), input / output interfaces, network interfaces, and memory.

[0094] Memory may include non-permanent storage in a computer-readable medium, in the form of random-access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0095] Computer storage media is readable storage media, also known as readable media. Readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0096] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the above elements.

[0097] The above contents are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A performance monitoring method, applicable to a central processing unit of an electronic device, characterized in that: The electronic device further comprises: at least one parallel processing unit; the central processor runs a first machine code of a scheduler, the scheduler being used to assign data processing tasks to a target parallel processing unit in the at least one parallel processing unit; The method comprises: Acquire a second machine code of a performance monitoring program; the performance monitoring program includes: an identifier of a performance analysis tool interface provided by a parallel processing unit; the performance analysis tool interface is used to collect performance data of an application program executing the data processing task in the target parallel processing unit; During the execution of the first machine code, the second machine code is inserted into the first machine code, so as to obtain the performance data of the application in the target parallel processing unit by using the second machine code according to the identifier of the performance analysis tool interface.

2. The method according to claim 1, characterized in that: The performance monitoring program includes: the insertion position of the performance monitoring program in the scheduler; and during the running of the first machine code, inserting the second machine code into the first machine code includes: During the execution of the first machine code, the second machine code is inserted at the insertion position at a target position corresponding to the first machine code.

3. The method according to claim 2, characterized in that The method of inserting the second machine code at the target position corresponding to the first machine code at the insertion position during the running of the first machine code includes: During the execution of the first machine code, a jump instruction pointing to the second machine code is inserted at the target location by process tracking, so as to insert the second machine code at the target location.

4. The method according to claim 2, characterized in that: The second machine code is a machine code in a dynamic link library; and during the execution of the first machine code, inserting the second machine code at the insertion position at a target position corresponding to the first machine code includes: During the execution of the first machine code, a jump instruction pointing to the second machine code in the dynamic link library is inserted at the target location, so as to insert the second machine code at the target location.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: The performance data of the application program in the target parallel processing unit is obtained by using the second machine code according to the identifier of the performance analysis tool interface.

6. The method according to claim 5, characterized in that The performance monitoring program is an extended Berkeley packet filter eBPF program; the second machine code is inserted into a target position of the first machine code; and the using the second machine code to obtain the performance data of the application in the target parallel processing unit according to the identifier of the performance analysis tool interface includes: When the first machine code runs to the target position, jump to the second machine code; The second machine code is executed in the user space through the eBPF technology in the user space, and during the execution of the second machine code, the performance data of the application in the target parallel processing unit is obtained according to the identifier of the performance analysis tool interface.

7. The method according to claim 6, characterized in that The method further comprises: After the execution of the second machine code is completed, returning to the target location to continue executing the first machine code; and, The jump instruction is deleted from the target location to stop the execution of the second machine code.

8. The method according to claim 6, characterized in that The performance analysis tool interface is compiled into a library function; the identifier of the performance analysis tool interface is the function name of the library function corresponding to the performance analysis tool interface; The step of obtaining the performance data of the application program in the target parallel processing unit according to the identifier of the performance analysis tool interface during the execution of the second machine code includes: During the execution of the second machine code, calling the library function according to the function name of the library function corresponding to the performance analysis tool interface; The library function is used to obtain performance data of the application program in the target parallel processing unit.

9. The method according to claim 5, characterized in that The method further comprises: The performance data of the application program in the target parallel processing unit is exported to the performance analysis process through an inter-process communication mechanism.

10. The method according to any one of claims 1 to 4, characterized in that: The data processing tasks include: artificial intelligence (AI) model training tasks and / or AI model reasoning tasks.

11. An electronic device, characterized in that: include: A memory and a processor; wherein the memory is used to store computer programs; the processor includes a central processing unit and at least one parallel processing unit; the central processing unit runs a first machine code of a scheduler, and the scheduler is used to assign data processing tasks to a target parallel processing unit in the at least one parallel processing unit; The central processing unit is coupled to the memory and is configured to execute the computer program to perform the steps in the method according to any one of claims 1 to 10.

12. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the method according to any one of claims 1 to 10.

13. A computer program product, characterized in that The method comprises a computer program, which, when executed by one or more processors, causes the one or more processors to execute the steps of the method according to any one of claims 1 to 10.

Images