Performance monitoring method and device, equipment, storage medium and program product

Abstract

The invention discloses a performance monitoring method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining a model script comprising a plurality of lines of codes; a script function corresponding to the model script is realized through at least one module function, and part of codes corresponding to the module function are used for calling operators to realize the module function; and in the process of executing each row of codes in the model script, monitoring the execution condition of each row of codes through a dynamic instrumentation technology to obtain performance data of each row of codes. Therefore, acquisition of the performance data of the code granularity is realized, performance analysis can be carried out on the code granularity, and the performance problem in the code can be identified and optimized more accurately.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a performance monitoring method, apparatus, device, storage medium, and program product. Background Technology

[0002] In the field of artificial intelligence, performance monitoring of various models typically focuses on two levels: the operator level and the module level. Operator-level performance monitoring mainly focuses on the performance data of individual operators, while module-level performance monitoring focuses on the performance data of individual modules. Current technologies can only perform performance monitoring at the operator and model levels, and cannot accurately locate local performance bottlenecks within complex models. Summary of the Invention

[0003] In view of the above, embodiments of this application provide at least one performance monitoring method, apparatus, device, storage medium, and program product.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] On one hand, embodiments of this application provide a performance monitoring method, the method comprising: obtaining a model script including multiple lines of code; the script function corresponding to the model script is implemented through at least one module function, and the part of the code corresponding to the module function is used to call an operator to implement the module function; during the execution of each line of code in the model script, the execution status of each line of code is monitored by dynamic instrumentation technology to obtain performance data of each line of code.

[0006] On the other hand, embodiments of this application provide a performance monitoring device, the device comprising: an acquisition module, configured to acquire a model script including multiple lines of code; the script function corresponding to the model script is implemented through at least one module function, and the partial code corresponding to the module function is used to call operators to implement the module function; and a monitoring module, configured to monitor the execution of each line of code in the model script by means of dynamic instrumentation technology, and obtain performance data of each line of code.

[0007] In another aspect, embodiments of this application provide a computer device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0008] In another aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0009] In another aspect, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement some or all of the steps in the above-described method.

[0010] In this embodiment, a model script comprising multiple lines of code is obtained. The script function corresponding to the model script is implemented through at least one module function, and the code corresponding to the module function is used to call operators to implement the module function. During the execution of each line of code in the model script, the execution status of each line of code is monitored to obtain performance data for each line of code. Thus, by monitoring the execution status of each line of code to obtain its performance data, compared to performance monitoring at the operator and module levels, performance data at the code granularity is obtained. This enables performance analysis at the code granularity, allowing for more accurate identification and optimization of performance issues in the code.

[0011] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0013] Figure 1 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 1 ;

[0014] Figure 2 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 2 ;

[0015] Figure 3 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 3 ;

[0016] Figure 4 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 4 ;

[0017] Figure 5 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 5 ;

[0018] Figure 6 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 6 ;

[0019] Figure 7 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 7 ;

[0020] Figure 8 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 8 ;

[0021] Figure 9 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 9 ;

[0022] Figure 10 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 10 ;

[0023] Figure 11 This is a schematic diagram of the composition structure of a performance monitoring device provided in an embodiment of this application;

[0024] Figure 12 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0028] In related technologies, performance monitoring of models mainly focuses on the operator and module levels, failing to provide detailed performance data for each line of code, such as execution time and memory usage changes. This limitation makes it difficult for developers to accurately pinpoint performance issues for each line of code.

[0029] To achieve performance monitoring at the code granularity level, this application provides a performance monitoring method that obtains a model script comprising multiple lines of code. The script function corresponding to the model script is implemented through at least one module function, and the code corresponding to the module function is used to call operators to implement the module function. During the execution of each line of code in the model script, the execution status of each line of code is monitored to obtain performance data for each line of code. In this way, by monitoring the execution status of each line of code to obtain its performance data, compared to performance monitoring at the operator and module levels, performance data at the code granularity level is obtained. This enables performance analysis at the code granularity level, allowing for more accurate identification and optimization of performance issues in the code.

[0030] This application provides a performance monitoring method, which can be executed by the processor of a computer device. The computer device refers to a device with data processing capabilities, such as a server, laptop, tablet, desktop computer, smart TV, set-top box, or mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device).

[0031] Figure 1 A schematic diagram of the implementation process of a performance monitoring method provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the method includes the following steps S101 to S102:

[0032] Step S101: Obtain a model script containing multiple lines of code; the script function corresponding to the model script is implemented through at least one module function, and the part of the code corresponding to the module function is used to call the operator to implement the module function.

[0033] The model script is the script file corresponding to the model, and it consists of multiple lines of code. Running the model script enables the functionality of the model. A model can be understood as a complex computer program that processes input data and ultimately produces output results. The model can be various machine learning models, deep learning models, or any other mathematical model. For example, the model is a deep learning model for image classification, the model script is the script corresponding to the deep learning model, and the script's function is image classification.

[0034] In this context, "module function" refers to the functionality implemented by a module within the model. Typically, a model can be subdivided into multiple modules, each corresponding to a portion of code in the model script and performing a specific function. For example, a convolutional neural network (CNN) model might include an input module, a convolutional module, a pooling module, and an output module. The convolutional module extracts features from the input data. The pooling module compresses these features.

[0035] In this context, operators are functions or classes used to perform various mathematical operations and calculations. As the basic building blocks of the model, operators enable data transfer, transformation, and computation. Different code segments require the invocation of different operators during execution; the code corresponding to a module's functionality implements that functionality by calling operators.

[0036] In some embodiments, performance monitoring of the model is required during model execution to obtain various performance data, analyze the data, and identify and optimize performance bottlenecks within the model. In related technologies, performance monitoring of the model typically considers both operator-level and module-level monitoring. Operator-level monitoring can obtain performance data for each operator, while module-level monitoring can obtain performance data for each module. These approaches have significant limitations, especially in code-level performance monitoring. Current methods lack performance monitoring of every line of code in the model script, failing to obtain code-level performance data. This makes it difficult for developers to accurately locate and optimize performance bottlenecks in the code, limiting their effectiveness in optimizing complex models.

[0037] In some embodiments, to obtain performance data at the code level, a model script comprising multiple lines of code is acquired for the model. During the execution of each line of code in the model script, the execution status of each line is monitored to obtain performance data for that line. Therefore, the first step is to acquire a model script comprising multiple lines of code.

[0038] Step S102: During the execution of each line of code in the model script, the execution of each line of code is monitored using dynamic instrumentation technology to obtain performance data for each line of code.

[0039] Performance data refers to data related to the various resources consumed by the code during execution. For example, performance data includes the code execution time and the amount of video memory used during code execution.

[0040] By monitoring the execution of each line of code in the model script, performance data for each line can be obtained. For example, the execution time of each line can be monitored during execution to obtain the runtime of each line.

[0041] In some embodiments, during the execution of each line of code in the model script, monitoring data before and after the execution of each line of code is obtained, and the performance data of each line of code is determined based on the difference between the monitoring data before and after the execution of each line of code.

[0042] In some embodiments, during the execution of each line of code in the model script, monitoring data of each line of code during execution is obtained, and the monitoring data is averaged or otherwise processed to obtain performance data of each line of code.

[0043] In some embodiments, during the execution of each line of code in the model script, performance monitoring code is non-intrusively inserted using dynamic instrumentation. This performance monitoring code monitors the execution of each line of code and obtains performance data for each line. Compared to manually inserting code, this approach avoids modifying a large amount of code, preventing the introduction of errors, and also avoids the difficulties in maintaining and updating the code due to frequent modifications.

[0044] For example, the model script is the script corresponding to the deep learning model, and the performance data is the running time of the code. During the execution of each line of code in the model script, the running time of each line of code is monitored to obtain the performance data of each line of code.

[0045] In this embodiment, a model script comprising multiple lines of code is obtained. The script function corresponding to the model script is implemented through at least one module function, and the code corresponding to the module function is used to call operators to implement the module function. During the execution of each line of code in the model script, the execution status of each line of code is monitored to obtain performance data for each line of code. Thus, by monitoring the execution status of each line of code to obtain its performance data, compared to performance monitoring at the operator and module levels, performance data at the code granularity is obtained. This enables performance analysis at the code granularity, allowing for more accurate identification and optimization of performance issues in the code.

[0046] Figure 2 This is a schematic diagram of the implementation process of a performance monitoring method provided in an embodiment of this application. Figure 2 This method can be executed by the processor of a computer device. Based on Figure 1 , Figure 1 S102 in the middle can be updated to S201 to S202, which will combine Figure 2 The steps shown are explained.

[0047] Step S201: During the execution of each line of code in the model script, obtain the monitoring data before and after the execution of each line of code.

[0048] The monitoring data refers to performance-related data monitored during code execution. For example, the monitored performance might be the code's runtime, and the monitoring data could be the system time at the start and end of code execution.

[0049] In some embodiments, during the execution of each line of code in the model script, each line of code is extracted from the model script sequentially using a code extraction tool; monitoring data before the execution of each line of code is obtained, and each line of code is executed; after the execution of each line of code is completed, monitoring data after the execution of each line of code is obtained.

[0050] In some embodiments, performance monitoring code is non-intrusively inserted during the execution of each line of code in the model script. This performance monitoring code includes a first part of monitoring code before code execution and a second part of monitoring code after code execution. The first part of the monitoring code obtains monitoring data before each line of code is executed, and the second part of the monitoring code obtains monitoring data after each line of code is executed.

[0051] Step S202: Based on the difference between the monitoring data before and after the execution of each line of code, determine the performance data of each line of code.

[0052] The monitoring data is updated in real time during code execution. The monitoring data changes before and after each line of code is executed. Based on the difference between the two monitoring data, the performance data of each line of code can be determined.

[0053] In some embodiments, the monitoring data value before code execution is less than the monitoring data value after code execution. The performance data is obtained by subtracting the monitoring data before execution from the monitoring data after execution.

[0054] In some embodiments, the monitoring data value before code execution is greater than the monitoring data value after code execution. The performance data is obtained by subtracting the monitoring data after execution from the monitoring data before execution.

[0055] For example, the performance data is the execution time of the code, and the monitoring data is the current system time. Before a line of code is executed, the system time is 10ms, and after the line of code is executed, the system time is 20ms. It can be determined that the execution time of the code is the difference of 10ms between the two system times.

[0056] In this embodiment, during the execution of each line of code in the model script, monitoring data before and after the execution of each line of code is acquired. Based on the difference between the monitoring data before and after the execution of each line of code, the performance data of each line of code is determined. In this way, by acquiring the monitoring data before and after code execution, the performance data of the code can be accurately obtained.

[0057] Figure 3 This is a schematic diagram of the implementation process of a performance monitoring method provided in an embodiment of this application. Figure 3 This method can be executed by the processor of a computer device. Based on Figure 2 , Figure 2 S201 in the middle can be updated to S301 to S303, which will combine Figure 3 The steps shown are explained.

[0058] Step S301: During the execution of each line of code in the model script, each line of code is extracted from the model script sequentially using a code extraction tool.

[0059] The code extraction tool is used to extract a single line of code. This tool can be software with code extraction capabilities, or it can be a simple function or module. The code extraction tool can access the model script and extract each line of code from it. For example, the code extraction tool can be a function `func.__func__.__code__`.

[0060] In order to obtain monitoring data for each line of code, it is first necessary to obtain the code that needs to be executed. After obtaining the code, the monitoring data before and after the code execution is obtained. This can achieve the matching between monitoring data and code.

[0061] In some embodiments, during the execution of each line of code in the model script, a code extraction tool sequentially extracts each line of code from the model script. Each extracted line of code is marked, and monitoring data before and after its execution is obtained. After obtaining the monitoring data for that line of code, the code extraction tool then extracts the next line of code.

[0062] Step S302: Obtain the monitoring data before the execution of each line of code, and execute each line of code.

[0063] The monitoring data before execution refers to the monitoring data corresponding to the code before execution. Therefore, it is necessary to obtain the monitoring data before code execution first, and then execute the corresponding code. In this way, by obtaining the monitoring data before execution in a preset order, the accuracy of the obtained monitoring data can be guaranteed.

[0064] In some embodiments, performance data includes execution time. The first timing data of the timer is obtained as monitoring data before code execution, and then the corresponding code is executed.

[0065] In some embodiments, performance data includes video memory usage data. The first video memory usage is obtained through the video memory monitoring interface provided in the model framework as monitoring data before execution, and then the corresponding code is executed.

[0066] Step S303: After each line of code is executed, obtain the monitoring data after the execution of each line of code.

[0067] The post-execution monitoring data refers to the monitoring data corresponding to the code execution. Therefore, the post-execution monitoring data can only be obtained after the code execution is complete. In this way, by executing the code first and then obtaining the post-execution monitoring data according to a preset order, the accuracy of the obtained monitoring data can be guaranteed.

[0068] In some embodiments, performance data includes execution time, where the code is executed first, and then second timing data from the timer is obtained as monitoring data after code execution.

[0069] In some embodiments, performance data includes video memory usage data. The code is executed first, and then the second video memory usage is obtained through the video memory monitoring interface provided in the model framework as the monitoring data after execution.

[0070] For example, the model script includes three thousand lines of code. The first line of code is extracted in chronological order using a code extraction tool. The time before the execution of the first line of code is obtained, the first line of code is executed, and the time after the execution of the first line of code is obtained after the execution of the first line of code is completed. This process is repeated for each line of code.

[0071] In this embodiment, during the execution of each line of code in the model script, a code extraction tool sequentially extracts each line of code from the model script; monitoring data before the execution of each line of code is obtained, and each line of code is executed; after the execution of each line of code is completed, monitoring data after the execution of each line of code is obtained. In this way, by obtaining the code and the monitoring data before and after code execution in a preset order, the monitoring data corresponding to each line of code can be accurately obtained.

[0072] Figure 4 This is a schematic diagram of the implementation process of a performance monitoring method provided in an embodiment of this application. Figure 4 This method can be executed by the processor of a computer device. Based on Figure 2 The performance data includes execution time. Figure 2 S201 in the middle can be updated to S401. Figure 2S202 in the middle can be updated to S402, which will combine Figure 4 The steps shown are explained.

[0073] Step S401: In response to the start of execution of each line of code, obtain the first timing data of the timer; in response to the completion of execution of each line of code, obtain the second timing data of the timer.

[0074] Execution time refers to the time required for the code to run.

[0075] The timer is a built-in software function that can perform timing.

[0076] The first timing data is the moment when the code starts executing, and the second timing data is the moment when the code finishes executing.

[0077] In some embodiments, the execution time of the code is determined by using a timer. When a line of code begins to execute, the value of the timer is obtained as the first timing data, and when the code finishes executing, the value of the timer is obtained as the second timing data.

[0078] In some embodiments, the timer restarts for each line of code. When a line of code begins execution, the timer value is 0, which is used as the first timing data. When the code finishes execution, the timer value is used as the second timing data. When the next line of code begins execution, the timer value resets to 0 and starts timing again.

[0079] In some embodiments, the timer does not reset for each line of code. The timer starts counting before the model script executes and continues counting continuously during script execution. As each line of code is executed, the timer value increases. The timer value is retrieved as the first timing data at the start of code execution and as the second timing data at the end of code execution.

[0080] Step S402: Determine the difference between the second timing data and the first timing data as the execution time of each line of code.

[0081] During code execution, a timer keeps ticking. The first ticking data is the moment when the code starts executing, and the second ticking data is the moment when the code finishes executing. Therefore, the difference between the second ticking data and the first ticking data is the execution time of the code.

[0082] Specifically, for each line of code, the first timing data and the second timing data corresponding to the code are obtained, the difference is calculated, and the execution time of each line of code is obtained.

[0083] For example, for a line of code, the timer starts executing at 10ms and finishes executing at 20ms, so the execution time of the code is a difference of 10ms.

[0084] In this embodiment, in response to the start of execution of each line of code, first timing data of the timer is obtained; in response to the completion of execution of each line of code, second timing data of the timer is obtained; the difference between the second timing data and the first timing data is determined as the execution time of each line of code. In this way, the execution time of each line of code can be accurately obtained.

[0085] Figure 5 This is a schematic diagram of the implementation process of a performance monitoring method provided in an embodiment of this application. Figure 5 This method can be executed by the processor of a computer device. Based on Figure 2 The performance data includes video memory usage data; Figure 2 S201 can be updated to S501, and S202 can be updated to S502, which will be combined Figure 5 The steps shown are explained.

[0086] Step S501: In response to the start of execution of each line of code, obtain the first video memory usage through the video memory monitoring interface provided in the model framework; in response to the completion of execution of each line of code, obtain the second video memory usage through the video memory monitoring interface.

[0087] The video memory usage data refers to the amount of video memory required for successful code execution. During code execution, a certain amount of video memory is needed to store data that is about to be processed or has already been processed.

[0088] Among them, the model framework is a software tool that allows you to easily build the required model.

[0089] The video memory monitoring interface is an application programming interface (API) that allows access to the video memory used in the system.

[0090] The first video memory usage is the current video memory usage in the system when a line of code begins execution; the second video memory usage is the current video memory usage in the system when a line of code completes execution.

[0091] In some embodiments, the model framework provides a video memory monitoring interface, which allows access to the current video memory usage in the system. During code execution, video memory is used, increasing the video memory usage. At the start of code execution, the current video memory usage is obtained through the video memory monitoring interface as the first video memory usage. Upon completion of code execution, the video memory usage increases, and at this point, the current video memory usage is obtained through the video memory monitoring interface as the second video memory usage.

[0092] Step S502: Determine the difference between the second video memory usage and the first video memory usage as the video memory usage data for each line of code.

[0093] During code execution, the system's video memory usage continuously increases. The first video memory usage is the system's video memory usage at the start of code execution, and the second video memory usage is the system's video memory usage at the end of code execution. Therefore, the difference between the second and first video memory usages is the video memory usage data required for code execution.

[0094] Specifically, for each line of code, the first and second video memory usages corresponding to the code are obtained, the difference is calculated, and the video memory usage data for each line of code is obtained.

[0095] For example, for a line of code, the first video memory usage when the code starts executing is 0.2G, and the second video memory usage when the code finishes executing is 0.21G. Therefore, the video memory usage data of the code is a difference of 0.01G.

[0096] In this embodiment, in response to the start of each line of code execution, a first video memory usage is obtained through the video memory monitoring interface provided in the model framework; in response to the completion of each line of code execution, a second video memory usage is obtained through the video memory monitoring interface; the difference between the second video memory usage and the first video memory usage is determined as the video memory usage data for each line of code. In this way, the video memory usage data for each line of code can be accurately obtained.

[0097] Figure 6 This is a schematic diagram of the implementation process of a performance monitoring method provided in an embodiment of this application. Figure 6 This method can be executed by the processor of a computer device. Based on Figure 1 , Figure 1 S102 in the middle can be updated to S601 to S602, which will combine Figure 6 The steps shown are explained.

[0098] Step S601: During the execution of each line of code in the model script, performance monitoring code is inserted non-intrusively.

[0099] The performance monitoring code is a piece of code corresponding to the performance monitoring function, used to obtain performance data for each line of code.

[0100] In some embodiments, dynamic instrumentation technology can non-intrusively insert performance monitoring code during model script runtime. This code monitors the execution of each line of code, obtaining performance data for that line. Dynamic instrumentation refers to modifying the target program at runtime, embedding the inserted code into the program's execution flow. Non-intrusive means that inserting performance monitoring code does not affect the normal operation of the original model script. The performance monitoring code is not displayed in the model script but runs in the background.

[0101] In some embodiments, performance monitoring code is non-intrusively inserted during the execution of each line of code in the model script. The performance monitoring code monitors the execution of each line of code and obtains the performance data of each line of code.

[0102] Step S602: Monitor the execution of each line of code using the performance monitoring code to obtain performance data for each line of code.

[0103] In some embodiments, during the execution of each line of code, when it is necessary to obtain monitoring data of the code, performance monitoring code is executed to obtain monitoring data of the code before and after execution, thereby obtaining performance data of each line of code.

[0104] In some embodiments, the performance data of each line of code is directly obtained by executing performance monitoring code during the execution of each line of code.

[0105] In some embodiments, the performance monitoring code includes tracing code, and the performance data includes code information. The step of monitoring the execution of each line of code through the performance monitoring code to obtain the performance data of each line of code includes: during the execution of each line of code in the model script, obtaining the code information of the currently executing line of code by executing the tracing code.

[0106] For example, `sys.settrace` is a function provided by the built-in Python module `sys` that allows you to set a trace function to monitor program execution. Once `sys.settrace` is set, the Python interpreter calls this trace function whenever a specific event occurs (such as the execution of a line of code). The trace function can then analyze these events and record the execution actions of each line of code.

[0107] In some embodiments, the tracing code includes frame parameters and event parameters, and the code information includes a code object, a line number, and an event type. Obtaining the code information of the currently executing line of code by executing the tracing code includes: obtaining the code object and line number of the currently executing line of code by executing the frame parameters; and obtaining the event type of the currently executing line of code by executing the event parameters.

[0108] For example, the tracing function includes a frame parameter and an event parameter. The frame parameter provides information about the current code frame. The code object can be obtained through frame.f_code, and the current line number can be obtained through frame.f_lineno. The event parameter indicates the type of the current event. For example, 'line' indicates the execution of a line of code. We mainly focus on the 'line' event because it needs to record the execution action of each line of code.

[0109] In some embodiments, the performance monitoring code includes time monitoring code, and the performance data includes runtime. Based on the time monitoring code, a high-precision timer is used to time the execution phase of the target code to obtain the runtime of the target code. The target code is the currently executing code. Starting the time monitoring code invokes the high-precision timer to perform timing, and the runtime of the target code is determined by the count value of the high-precision timer.

[0110] In some embodiments, the time monitoring code includes an acquisition code and a time reset code. The step of obtaining the running time of the target code by timing the execution phase of the target code using a high-precision timer based on the time monitoring code includes: resetting the high-precision timer using the time reset code in response to the start of execution of the target code; and acquiring the timing data of the high-precision timer as the running time of the target code using the time acquisition code in response to the completion of execution of the target code.

[0111] The code retrieves timing data from a high-precision timer, while the time reset code resets the high-precision timer. When the target code begins execution, the time reset code resets the high-precision timer to 0. It then counts during the target code's execution. After the target code completes execution, the retrieval code retrieves the high-precision timer's timing data, which is then used as the target code's runtime.

[0112] In some embodiments, the performance monitoring code includes video memory monitoring code, and the performance data includes video memory usage. Based on the video memory monitoring code, the video memory usage of the target code during the execution phase is obtained through the video memory monitoring interface. Specifically, starting the video memory monitoring code will call the video memory monitoring interface to obtain the current video memory usage of the system.

[0113] In some embodiments, the video memory monitoring code includes a first video memory monitoring code and a second video memory monitoring code. The step of obtaining the video memory usage of the target code during the execution phase through a video memory monitoring interface based on the video memory monitoring code includes: obtaining a first video memory usage through the first video memory monitoring code in response to the start of execution of the target code; obtaining a second video memory usage through the second video memory monitoring code in response to the completion of execution of the target code; and using the difference between the second video memory usage and the first video memory usage as the video memory usage of the target code during the execution phase.

[0114] Specifically, when the target code starts executing, the first video memory monitoring code calls the video memory monitoring interface to obtain the current video memory usage of the system as the first video memory usage, and the second video memory monitoring code calls the video memory monitoring interface to obtain the current video memory usage of the system as the second video memory usage.

[0115] In some embodiments, obtaining the current video memory usage of a device through a video memory monitoring interface based on video memory monitoring code includes: calling the video memory monitoring interface through the video memory monitoring code; and, when the video memory monitoring interface is called, communicating with the graphics processor through the driver to obtain the current video memory usage of the device.

[0116] For example, the `memory_allocated()` function calculates the change in video memory usage before and after executing each line of code. This function calls the CUDA (Compute Unified Device Architecture) memory management API `cudaMemGetInfo` to query the current memory usage of the device. When `cudaMemGetInfo` is called, the CUDA driver communicates directly with the graphics processing unit (GPU) to query video memory usage. The driver sends a memory query command, and the GPU returns the current memory status data. Internally, the GPU has a memory management unit responsible for managing the allocation and use of video memory.

[0117] For example, in the process of executing each line of code in the model script, performance monitoring code for code execution time is inserted non-intrusively. Through the performance monitoring code, the execution of each line of code is monitored, and the execution time of each line of code is obtained.

[0118] In this embodiment, performance monitoring code is non-intrusively inserted during the execution of each line of code in the model script. This code monitors the execution of each line and obtains its performance data. Thus, accurate performance data for each line of code can be obtained without affecting the normal operation of the model script.

[0119] In some embodiments, the performance data includes resource usage data for at least one dimension; the method further includes: obtaining a preset resource usage threshold for each of the dimensions; and generating an alert message corresponding to the target dimension when the resource usage data for the target dimension exceeds the target dimension's resource usage threshold.

[0120] At least one dimension is required for performance monitoring, and the resource usage data refers to the data corresponding to the resources required for code execution. For example, in the time dimension, the resource usage data is the code's execution time; in the memory dimension, the resource usage data is the amount of memory required for code execution.

[0121] The resource usage threshold is a preset threshold set by the system. If the resource usage data exceeds the threshold, the resources consumed by the code execution are too large and exceed the normal level; if the resource usage data does not exceed the threshold, the resources consumed by the code execution are within a reasonable range.

[0122] The warning message is a notification indicating that resource usage data has exceeded the resource usage threshold. For example, the warning message might be a notification displayed on the user interface indicating that the execution time of the third line of code is too long.

[0123] In some embodiments, the system presets a resource usage threshold for each dimension of the performance data. During the execution of the model script, the system monitors the resource usage data for each dimension of each line of code, compares the obtained resource usage data with the corresponding resource usage threshold, and generates a warning message for the target dimension if the resource usage data of the target dimension exceeds the target dimension's resource usage threshold.

[0124] For example, performance data includes resource usage data in two dimensions: the execution time of each line of code in the time dimension, and the GPU memory usage of each line of code in the GPU memory dimension. During the execution of each line of code in the model script, the execution time and GPU memory usage of each line of code are monitored. If the execution time of the code exceeds a certain threshold, a warning message corresponding to the execution time of the target code is generated; if the GPU memory usage of the code exceeds a certain threshold, a warning message corresponding to the GPU memory usage of the target code is generated.

[0125] In this embodiment, a preset resource usage threshold for each dimension is obtained. If the resource usage data for a target dimension exceeds the target dimension's resource usage threshold, a warning message corresponding to that target dimension is generated. In this way, by comparing the resource usage data with the resource usage threshold, the resource usage of the code can be accurately assessed, and the warning message can promptly inform the user of performance issues in the code.

[0126] Figure 7 This is a schematic diagram of the implementation process of a performance monitoring method provided in an embodiment of this application. Figure 7 This method can be executed by the processor of a computer device. Based on Figure 1 The method further includes step S701, combining Figure 7 The steps shown are explained.

[0127] Step S701: Perform data analysis on the performance data of each line of code to obtain the performance analysis results of the model script; the performance analysis results are used to locate the performance problems of the model script at the level of each line of code.

[0128] The performance data for each line of code is analyzed to obtain the performance analysis results of the model script. These results reveal performance issues within the model script. Performance issues refer to problems arising from the failure of code performance data to meet requirements. For example, the performance analysis results show that the execution time of the third line of code is too long and the memory usage of the fifth line of code is too high.

[0129] In some embodiments, performance data analysis is performed on the performance data of each line of code to obtain the performance analysis results for each line of code. The performance analysis results indicate whether the resource usage data of each line of code is normal in at least one dimension. The performance analysis results of all lines of code are then summarized to obtain the performance analysis results of the model script.

[0130] The performance analysis results for each line of code show the performance issues corresponding to that line, while the performance analysis results for the model script show the performance issues present in all lines of code. Therefore, the performance issues of the model script can be located at the level of each line of code.

[0131] In some embodiments, the performance data includes resource usage data in at least one dimension. Based on the performance analysis strategy corresponding to the dimension, the resource usage data of each line of code in the dimension is analyzed to obtain the performance analysis results of the model script in the dimension.

[0132] For example, the performance data includes the runtime of the code in the time dimension. Data analysis of the runtime of 100 lines of code in the model script shows that the runtime of lines 20 and 35 changes drastically and exceeds the normal level, while the runtime of other lines of code is normal. The performance analysis results of the model script are then obtained by summarizing the data.

[0133] In this embodiment, performance data for each line of code is analyzed to obtain the performance analysis results of the model script. This allows for precise location and identification of performance bottlenecks within the model script.

[0134] Figure 8 This is a schematic diagram of the implementation process of a performance monitoring method provided in an embodiment of this application. Figure 8 This method can be executed by the processor of a computer device. Based on Figure 7 The performance data includes resource usage data in at least one dimension; Figure 7 The S701 in the middle can be updated to S801, which will combine Figure 8 The steps shown are explained.

[0135] Step S801: Based on the performance analysis strategy corresponding to the dimension, analyze the resource usage data of each line of code under the dimension to obtain the performance analysis results of the model script under the dimension.

[0136] Among them, performance analysis strategy refers to the method of analyzing resource usage data, and different dimensions correspond to different performance analysis strategies.

[0137] The performance analysis results include at least one of the following: the location of the code line with the performance problem, an alarm message carrying the location of the code line with the performance problem, optimization suggestions for the code line with the performance problem, and a trend chart of resource usage data during the execution of the model script.

[0138] Here, the line position refers to the location of a line of code in the model script, which can be the line number of the code or other arbitrary identifier.

[0139] The alarm information is related to code with performance issues. The alarm information carries the location of the line of code with performance issues, and may also carry the resource usage data and resource usage threshold corresponding to the code with performance issues.

[0140] Among them, optimization suggestions refer to how to optimize lines of code that have performance issues.

[0141] The trend chart shows the trend of resource usage data for each line of code. Each dimension corresponds to one trend chart. The trend chart can be any chart representing the trend, such as a line chart or bar chart. For example, for a line chart representing the execution time dimension, the horizontal axis can be the line number of the code, and the vertical axis can be the execution time corresponding to each line of code.

[0142] In some embodiments, after obtaining resource usage data for at least one dimension, the resource usage data of each line of code under the dimension can be analyzed based on the performance analysis strategy corresponding to the dimension to obtain the performance analysis results of the model script under the dimension.

[0143] In some embodiments, the performance analysis results include the location of the code line with the performance problem, and a preset resource usage threshold for the obtained dimension; based on the resource usage threshold and the resource usage data of each line of code in the dimension, the location of the code line with the performance problem is determined.

[0144] In some embodiments, the performance analysis results include optimization suggestions for lines of code with performance issues. Based on the location of the problematic lines of code, the code segment corresponding to the location of the problematic lines of code is obtained from multiple lines of code. The code segment is then input into a pre-trained large language model to generate optimization suggestions for the problematic lines of code.

[0145] In some embodiments, different dimensions correspond to different performance analysis strategies. At least one dimension includes a time dimension and a memory dimension. The resource usage threshold corresponding to the time dimension is the execution time threshold, and the resource usage threshold corresponding to the memory dimension is the memory usage threshold. The large language model corresponding to the time dimension is a model trained based on multiple lines of code and the execution time of each line of code, and the large language model corresponding to the memory dimension is a model trained based on multiple lines of code and the memory usage of each line of code.

[0146] In some embodiments, the performance analysis strategies for different dimensions can be the same. For both the time and memory dimensions, performance data can be statistically analyzed using line charts.

[0147] For example, performance data includes memory usage at the video memory level. Based on a preset memory usage threshold at the video memory level, the memory usage of each line of code is analyzed, and the lines of code whose memory usage exceeds the threshold are identified as line numbers 3, 53, and 90. Alerts are issued for these code lines with performance issues, and the alert information includes line numbers 3, 53, and 90. The code segments corresponding to line numbers 3, 53, and 90 are analyzed based on the Big Oracle model corresponding to the video memory level, and optimization suggestions are provided for these lines. A line chart is then used to statistically analyze the memory usage of each line of code, resulting in a trend chart of memory usage changes.

[0148] In this embodiment, based on a performance analysis strategy corresponding to a dimension, the resource usage data of each line of code under that dimension is analyzed to obtain the performance analysis results of the model script under that dimension. The performance analysis results include at least one of the following: the location of the code line with performance problems, an alarm message carrying the location of the code line with performance problems, optimization suggestions for the code line with performance problems, and a trend graph of resource usage data changes during the execution of the model script. In this way, by using a performance analysis strategy to analyze the resource usage data of each line of code under that dimension, diverse performance analysis results are obtained, effectively identifying and optimizing performance problems in the code.

[0149] Figure 9 This is a schematic diagram of the implementation process of a performance monitoring method provided in an embodiment of this application. Figure 9 This method can be executed by the processor of a computer device. Based on Figure 8 The performance analysis results include the location of the lines of code where performance issues exist; Figure 8 S801 in the middle can be updated to S901 to S902, which will combine Figure 9 The steps shown are explained.

[0150] Step S901: Obtain the preset resource usage threshold under the dimension.

[0151] Specifically, performance issues are determined by comparing the resource usage data and resource usage thresholds for each line of code within a given dimension. Therefore, the first step is to obtain the preset resource usage thresholds for that dimension. These preset thresholds can be set according to actual needs, and they are typically different or the same across different dimensions. For example, the resource usage threshold corresponding to code execution time could be 20ms.

[0152] Step S902: Based on the resource usage threshold and the resource usage data of each line of code in the dimension, determine the location of the line of code with performance problems.

[0153] Specifically, after obtaining the resource usage threshold, the resource usage data of each line of code is compared with the resource usage threshold. If the resource usage data of the code exceeds the resource usage threshold, it is determined that the code has a performance problem, and the location of the code line with the performance problem is recorded.

[0154] In some embodiments, the resource usage threshold is an execution time threshold. The execution time of each line of code is compared with the execution time threshold to identify the code whose execution time exceeds the execution time threshold. The location of that line of code is then identified as the location of the code with a performance problem.

[0155] In some embodiments, the resource usage threshold is a video memory usage threshold. The video memory usage of each line of code is compared with the video memory usage threshold to determine the code whose video memory usage exceeds the video memory usage threshold. The location of this code line is then identified as the location of the code line with a performance problem.

[0156] In this embodiment, a preset resource usage threshold is obtained for each dimension. Based on the resource usage threshold and the resource usage data of each line of code within that dimension, the location of the code line with the performance problem is determined. Thus, by comparing the resource usage data with the resource usage threshold, the accurate location of the code line with the performance problem can be obtained.

[0157] Figure 10 This is a schematic diagram of the implementation process of a performance monitoring method provided in an embodiment of this application. Figure 10 This method can be executed by the processor of a computer device. Based on Figure 8 The performance analysis results include optimization suggestions for lines of code with performance issues; Figure 8 S801 in the middle can be updated to S1001 to S1002, which will combine Figure 10 The steps shown are explained.

[0158] Step S1001: Based on the location of the code line with the performance problem, obtain the code segment corresponding to the code line location from the multiple lines of code.

[0159] A code segment consists of at least one line of code, which is a combination of at least one interrelated line of code. A code segment can implement a logic or function.

[0160] In some embodiments, after identifying the line of code with performance issues, the performance problem is usually not caused by this single line of code, but by the code before and after it. Therefore, when analyzing the performance problem, it is necessary to obtain the code segment corresponding to the line of code, analyze the code segment, and obtain optimization suggestions for the line of code with performance issues.

[0161] In some embodiments, the code segment corresponding to the line position of a code segment can be obtained from multiple lines of code, and the user can directly specify which lines of code constitute the code segment based on experience.

[0162] In some embodiments, a code segment recognition model is pre-trained, and the location of the code line with performance issues is input into the model to obtain the code segment.

[0163] Step S1002: Input the code segment into a pre-trained large language model to generate optimization suggestions for lines of code with performance issues.

[0164] The large language model is a model trained on a large-scale dataset that can understand and generate natural language text. In this embodiment, the large language model can analyze the input code segment and generate optimization suggestions for lines of code with performance issues.

[0165] In some embodiments, a large language model is trained based on data related to code execution time to obtain a trained large language model that can optimize code execution time. Code segments with high execution times are input into the large language model to generate optimization suggestions. These suggestions can be ways to adjust code segments to reduce execution time.

[0166] In some embodiments, a large language model is trained based on data related to code memory usage, resulting in a trained large language model capable of optimizing code memory usage. Code segments with high memory usage are input into the large language model, generating optimization suggestions. These suggestions can be ways to adjust the code segments to reduce memory usage.

[0167] In this embodiment, based on the location of the code line with the performance problem, the code segment corresponding to that location is obtained from multiple lines of code. This code segment is then input into a pre-trained large language model to generate optimization suggestions for the code line with the performance problem. In this way, the trained large language model can generate optimization suggestions for specific code lines, helping developers quickly understand the root cause of the problem and providing concrete solutions, thus accelerating the optimization process.

[0168] The following describes the application of the performance monitoring method provided in the embodiments of this application in a real-world scenario.

[0169] In the field of deep learning, performance analysis typically focuses on two levels: the operator level and the module level. Operator-level analysis primarily focuses on the execution time and memory usage of each operator (corresponding to the memory usage data in the above examples), while module-level analysis focuses on the execution time and memory usage of each module in the model. These methods are mainly implemented through various performance analysis tools, such as TensorBoard and PyTorch's Profiler.

[0170] For operator-level analysis, the performance analysis tool monitors the performance of each operator during the training process, records the execution time and memory usage, and provides graphical analysis results.

[0171] At the module level, the execution time and memory usage of each module are recorded by analyzing the API calls provided by the framework. This data can also be viewed through a graphical interface, helping developers understand the model's performance bottlenecks.

[0172] These technical solutions provide a foundation for deep learning performance analysis, but they also have significant limitations, especially in code-level performance analysis. Current methods lack the ability to analyze the execution time and memory changes for each line of code, which limits their effectiveness in optimizing complex models.

[0173] A module consists of multiple lines of code, and each line of code may contain multiple operators. The granularity of an operator is the smallest, the granularity of code is larger than the granularity of an operator, and the granularity of a module is larger than the granularity of an operator.

[0174] In some embodiments, the loss calculation part of the currently popular large model training scripts can be considered a module function, and each calculation (or each line of function) calls many operators. This method focuses on the memory changes before and after the execution of a line. For example, in the two lines of functions, shift_logits = logits[…,:-1,:].contiguous() and shift_labels = labels[…,1:].contiguous(), each line copies logits and labels once. If it is a module-level analysis tool, it can only capture the memory changes before and after the execution of the entire module, and cannot obtain the memory changes of the two lines shown; and the memory changes at the operator level are even more difficult to monitor, because the execution of a single operator is correct, but due to unreasonable code writing, multiple copies are called. However, code-level analysis tools can clearly find that an extra copy is performed here, which greatly affects performance.

[0175] The existing technology has the following shortcomings:

[0176] (1) Lack of code-level performance analysis: Current performance analysis tools mainly focus on the operator and module levels, and cannot provide detailed performance data for each line of code, such as execution time and memory usage. This limitation makes it difficult for developers to accurately locate and optimize performance bottlenecks in the code.

[0177] (2) Incomplete Analysis Coverage: Module-level performance analysis tools often fail to cover all code, especially dynamically generated or conditionally executed code segments, leading to incomplete analysis results. For example, the code required for a complete deep learning model includes code within and outside the module, requiring performance data for all code within the entire deep learning model process. While performance analysis tools can monitor the performance data for code within a module, they cannot obtain the corresponding performance data for code outside the module. Operator-level performance analysis tools, after obtaining the performance data for each operator, often fail to effectively identify the operator's position within the entire model. For example, a model may contain multiple networks, each containing multiple operators. Operators in different networks can be the same, and the same operator can be executed in multiple networks.

[0178] (3) Manual adjustments and in-depth understanding of the code are required: In order to use existing module-level performance analysis tools, developers need to manually adjust the code, such as inserting specific monitoring code. This not only increases the burden on developers, but also requires them to have a deep understanding of the underlying implementation.

[0179] To address the aforementioned shortcomings of existing technologies, this application proposes a tool for deep learning performance analysis at the code granularity, which can solve the following technical problems:

[0180] (1) Provide detailed performance data at the code granularity: The embodiments of this application can analyze the execution time and memory usage of each line of code, providing developers with more detailed performance data, thereby helping them to more accurately identify and optimize performance bottlenecks.

[0181] (2) Comprehensive code execution coverage: Through automated code-level performance monitoring, this application embodiment does not require additional manual code adjustments and can automatically capture performance data of all execution paths, including dynamically generated code segments and conditionally executed code segments.

[0182] (3) Simplified performance analysis process: The implementation of this application embodiment does not require users to have in-depth knowledge of the underlying code. Users can start performance analysis with simple configuration, which greatly reduces the threshold for performance optimization.

[0183] This application provides a deep learning performance analysis tool at the code level, belonging to the field of artificial intelligence, and specifically relates to a performance analysis tool for deep learning model training at the code level. In graphics processing unit (GPU) applications, i.e., when training large models, it is used to collect and analyze the running speed and memory usage of deep learning at the Python code level.

[0184] This application aims to provide a deep learning performance analysis tool that can perform performance analysis at the code granularity level, solving the problem that existing performance analysis tools cannot meticulously track the performance of every line of code. This will help developers accurately locate performance bottlenecks and optimize the execution efficiency of deep learning models.

[0185] This application provides a comprehensive performance analysis tool specifically designed for deep learning training applications on GPUs, enabling detailed monitoring and analysis of execution time and GPU memory usage at the code granularity. The following is a detailed technical implementation plan, including the system composition, steps, and functions.

[0186] In some embodiments, the system includes a performance data collection module, a data analysis and processing module, and a result display and user interaction module.

[0187] The performance data collection module comprises a code injection submodule and a data collection submodule. The code injection submodule injects monitoring code (corresponding to the performance monitoring code in the above embodiment) non-intrusively at runtime using dynamic instrumentation technology. The data collection submodule stores performance data using an improved C++ data structure (e.g., unordered_map). The code injection submodule interacts directly with the Python interpreter, while the data collection submodule connects to memory and GPU memory status monitoring tools (e.g., pycuda) to achieve real-time data capture. The function of the performance data collection module is to monitor the execution of Python code in real time, collecting the execution time and GPU memory usage of each line of code.

[0188] The data analysis and processing module includes a performance analysis submodule and an optimization suggestion generation submodule. The performance analysis submodule processes and analyzes time and memory data. The optimization suggestion generation submodule automatically generates improvement suggestions based on the analysis results. After processing the data, the performance analysis submodule passes the results to the optimization suggestion generation submodule, which then provides feedback to the user. The function of the data analysis and processing module is to analyze the collected performance data and identify bottlenecks in the code.

[0189] The results display and user interaction module includes a graphical display submodule and a report generation submodule. The graphical display submodule generates a graphical interface to display data. The report generation submodule is used to compile detailed performance reports. Both the graphical display and report generation submodules receive data processed by the data analysis and processing module, ensuring that information is accurately conveyed to the end user. The function of the results display and user interaction module is to present the analysis results to the user in graphical and report formats, including performance heatmaps, memory usage graphs, etc.

[0190] In some embodiments, the implementation and application of this technical solution are described in detail to ensure that those skilled in the art can easily deploy the system. A large model training scenario under the colossal AI framework is used as an example. The implementation steps include system initialization, performance data collection, data analysis and optimization suggestion generation, result display, and interaction.

[0191] System initialization includes setting parameters and environment, and preparing data structures. In the parameter and environment setting step, the necessary monitoring and data processing software (performance analysis tools) is installed in the user's development environment, and relevant parameters of the software are configured. These parameters include the sensitivity of performance monitoring and the time interval for video memory monitoring (corresponding to the resource usage threshold in the above embodiment). The sensitivity of performance monitoring can be used to issue alarms when performance data changes significantly. The time interval for video memory monitoring is used to issue alarms when the execution time of a single line of code is too long. For example, alarms are issued when a significant change in video memory usage is detected, and when the code execution time is long. In the data structure preparation step, the data structures required for software operation are initialized. These data structures include an efficient C++ unordered_map structure for storing performance and video memory data. The unordered_map structure is an unordered collection of key-value pairs.

[0192] The performance data collection process includes code injection and data capture. In the code injection step, performance monitoring code is inserted into the user's Python code using dynamic code injection technology. This code tracks execution time and memory usage changes without affecting the original code logic. For example, the performance monitoring code includes time monitoring code and memory monitoring code. The time monitoring code obtains the execution time before and after the code execution, while the memory monitoring code obtains the memory usage before and after the code execution. For example, the performance monitoring code is not displayed in the Python code but runs in the background. Before executing a line of Python code, the current time and memory usage are recorded; after executing the line of Python code, the current time and memory usage are recorded again. The difference between the recorded times and memory usage is used to obtain the execution time of the line of code, and the difference between the recorded memory usage is used to obtain the memory usage of the line of code. For each line of the user's Python code, the performance monitoring code records performance data before and after the code execution.

[0193] In some embodiments, code execution tracing is one of the core functionalities, recording and analyzing performance data by capturing every detail of code execution. This work primarily relies on Python's `sys.settrace` function and tracing function (corresponding to the tracing code in the above embodiments). The following is a detailed explanation of this process: `sys.settrace` is a function provided by the Python built-in module `sys`, which allows setting a tracing function to monitor program execution. Once `sys.settrace` is set, the Python interpreter calls this tracing function whenever the program executes a specific event (e.g., line of code execution). The tracing function can record the execution action of each line of code by analyzing these events. When the program executes the modified function, this tracing function is called frequently, performing the following steps: 1. Obtain current frame information: The `frame` parameter (corresponding to the frame parameter in the above embodiments) provides information about the current code frame. The code object can be obtained through `frame.f_code`, and the current line number can be obtained through `frame.f_lineno`. 2. Capture event type: The event parameter (corresponding to the event parameter in the above example) indicates the type of the current event. For example, 'line' indicates the execution of a line of code. We mainly focus on the 'line' event because it needs to record the execution action of each line of code.

[0194] In the data capture step, during code execution, the execution time and video memory usage of each line of code are monitored, and the data is written to a pre-defined data structure in real time. This includes two main operations: acquiring the Python code and acquiring the changes in the GPU device state before and after running that line of Python code. This tool focuses primarily on time and video memory usage. For example, for a specific line of Python code, the tool first acquires the time and video memory usage before the line of code is executed, then acquires the line of code, executes the line of code, and finally acquires the time and video memory usage after the line of code is executed.

[0195] Specifically, when acquiring Python code, it is done using `func.__func__.__code__` (corresponding to the code extraction tool in the above embodiment), avoiding processing duplicate code objects. When acquiring time and GPU memory, a high-precision timer is used to acquire the time, and the GPU memory, including allocated and currently occupied GPU memory, is acquired through the GPU memory monitoring interface provided in the deep learning framework. For example, the high-precision timer starts counting before running a line of Python code and stops counting when the code execution ends; the GPU memory usage is checked through the GPU memory monitoring interface before and after code execution.

[0196] In some embodiments, before and after acquiring the current frame, the memory usage change before and after executing each line of code is calculated by encapsulating the `memory_allocated()` and `memory_reserved()` functions (corresponding to the memory monitoring code in the above embodiments). `torch.cuda.memory_allocated()` and `torch.cuda.memory_reserved()` are two important functions used to query the actual memory occupied by the current model and the total memory reserved by the model, respectively. The function implementation is as follows: The function calls the CUDA memory management API `cudaMemGetInfo` to query the memory usage of the current device. `cudaMemGetInfo` is the CUDA memory management API, which returns the available memory (`free_mem`) and total memory (`total_mem`) on the current device. By calculating `total_mem - free_mem`, the amount of memory currently allocated can be obtained. When `cudaMemGetInfo` is called, the CUDA driver communicates directly with the GPU to query the memory usage. The driver maintains a memory table that records the memory allocation status of each process. The CUDA driver interacts directly with the GPU hardware via the GPU's communication bus (such as the high-speed serial bus standard (Peripheral Component Interconnect Express, PCIe)). The driver sends a memory query command, and the GPU returns the current memory status data. Internally, the GPU has a Memory Management Unit (MMU) responsible for managing memory allocation and usage. The MMU maintains a memory allocation table and status, including which memory blocks have been allocated and which are free. When the driver queries the memory status, the MMU returns the current memory usage, including allocated and free memory blocks. The overall flow is as follows: Python layer: torch.cuda.memory_allocated → PyTorch C++ backend: calls the CUDA API → CUDA API: cudaMemGetInfo → CUDA driver: communicates with the GPU → GPU MMU: returns memory usage information.

[0197] The data analysis and optimization suggestion generation includes data processing and suggestion generation. In the data processing step, after data collection, a dedicated analysis module processes the data (execution time and GPU memory usage for each line of code) to identify performance bottlenecks and GPU memory usage peaks (corresponding to the performance analysis results in the above embodiments). For example, in currently common large model building scripts, there is significant GPU memory waste in the processing of logits because logits are copied three times; however, logits are typically huge tensors (usually reaching several gigabytes or even tens of gigabytes). Manual analysis of this part requires a certain level of skill from the programmer. When using this tool, it collects this data on drastic GPU memory changes and prepares it for subsequent report generation.

[0198] In the suggestion generation step, performance optimization suggestions are automatically generated based on the analysis results. These suggestions help developers understand the causes of performance bottlenecks and provide specific solutions. This tool integrates a large language model. Specifically, it passes the code segments with drastic changes in memory usage (corresponding to the code segments in the above examples) to a pre-trained code-domain large language model, generating several code optimization suggestions to provide programmers with solutions or ideas for optimization. Code segments with drastic changes in runtime are also passed to the pre-trained code-domain large language model to generate code optimization suggestions. For code segments with drastic changes in runtime, the pre-trained code-domain large language model generates several code optimization suggestions. For example, if the second line of code has a longer runtime, the related code (the first to third lines of code) is passed to the large language model to obtain the results.

[0199] In some embodiments, passing the code segment that monitors abnormal changes in video memory to the large language model will provide reasonable suggestions. The code segment showing abnormal changes in video memory is shown below:

[0200] if labels is not None:

[0201] shift_logits=logits[…,:-1,:].contiguous()

[0202] shift_labels=labels[…,1:].contiguous()

[0203] logits = None

[0204] The large language model will warn that there are two lines of code in the provided code that involve calls to 'contiguous'. These calls may result in additional GPU memory usage because 'contiguous' returns a tensor with a contiguous memory layout, which may lead to data reallocation and thus increase GPU memory usage.

[0205] Optimization suggestions:

[0206] To optimize video memory usage, consider the following methods:

[0207] 1. Check continuity

[0208] First, check if 'logits' and 'labels' are already contiguous. This can be checked using the 'is_contiguous()' method, for example, print(logits.is_contiguous()) and print(labels.is_contiguous()). If they are already contiguous, there is no need to call 'contiguous()'.

[0209] 2. Try to avoid slicing.

[0210] Slicing operations typically return non-contiguous tensors. If slicing results always require 'contiguous()', consider optimizing memory layout during data generation or preprocessing to avoid frequent runtime operations. For example, during data preprocessing, ensure that data is stored in the required order and memory layout.

[0211] The results display and interaction include graphical interface presentation and report generation. In the graphical interface presentation step, performance analysis results are displayed through a user-friendly graphical interface, including heatmaps, execution time graphs, and memory usage graphs (corresponding to the trend graphs in the above embodiments), helping users intuitively understand performance issues. For example, a line graph represents the execution time and memory usage of each line of code, with the horizontal axis representing the line number and the vertical axis representing execution time or memory usage. Data with drastic performance changes is highlighted in the execution time graph and memory usage graph as heatmaps. In the report generation step, a detailed performance analysis report is generated, highlighting the areas with significant memory usage changes. This avoids requiring users to perform line-by-line memory usage analysis at the code level, allowing for further analysis and recording, reducing analysis difficulty and saving considerable time. The performance analysis report includes detailed performance analysis results and generated recommendations.

[0212] This application's embodiments, through clear steps and detailed operating instructions, ensure the efficient execution of the technical solution, enabling the system to run stably in various development environments and providing real-time, precise performance optimization support for deep learning developers. The above technical solution addresses the main shortcomings of existing technologies, significantly improving the efficiency of deep learning model development and optimization.

[0213] The embodiments of this application bring several beneficial technical effects to the field of deep learning performance analysis, significantly improving the efficiency and accuracy of code optimization. Specifically, these effects include the following:

[0214] (1) Improved accuracy of performance optimization: By performing performance analysis at the code level, the embodiments of this application can provide performance data beyond the traditional operator or module level. This enables developers to accurately identify the specific code location where performance bottlenecks exist, thereby enabling targeted optimization.

[0215] (2) Dynamic monitoring of video memory usage: In addition to time performance analysis, the embodiments of this application can also monitor and analyze video memory usage, which is especially important for applications with limited video memory. By monitoring video memory usage in real time, developers can optimize memory management, reduce memory overflow or waste, and improve the overall stability and efficiency of the application.

[0216] (3) Automatic generation of optimization suggestions: Based on the performance analysis results, this system can automatically generate optimization suggestions, which not only helps developers quickly understand the root cause of the problem, but also provides specific solutions and speeds up the optimization process.

[0217] (4) Automated and user-friendly analysis tools: This application provides an easy-to-use interface, and the automated data collection and analysis process greatly reduces the workload of developers. Through graphical data display, developers can intuitively understand performance issues without having to delve into complex performance data.

[0218] (5) Adaptability and scalability: The technical solution of this application is designed with different development environments and application requirements in mind, and has good adaptability and scalability. It can be easily deployed and function effectively in both small-scale projects and large-scale industrial applications.

[0219] Through these technical effects, the embodiments of this application not only improve the accuracy and efficiency of performance analysis, but also provide strong support for the development and optimization of deep learning models, which helps to promote the development and application of related technologies.

[0220] In some embodiments, performance monitoring is achieved through manual code injection. Code is manually added to critical code segments to monitor and record video memory usage. This method typically involves inserting API calls before and after code execution to capture snapshots of video memory usage. While this approach directly controls the location and method of monitoring and can pinpoint specific code segments with great precision, it is cumbersome, requires extensive code modifications, and is prone to introducing errors. Furthermore, frequent modifications can lead to code that is difficult to maintain and update.

[0221] In some embodiments, performance analysis is achieved through code review by professional programmers. Relying on programmers with deep knowledge to review the code and use their experience to determine which parts might have performance bottlenecks or improper memory usage, this approach can leverage expert experience and intuition to potentially uncover complex problems that automated tools struggle to identify. However, it is highly dependent on personal experience, inefficient, and difficult to scale to large-scale codebases. Furthermore, the need for experts makes it costly and difficult to guarantee continuous code quality control.

[0222] While these alternatives may be effective in certain situations, they generally cannot provide the level of automation, accuracy, and ease of use achievable with the embodiments of this application. Manually adding monitoring code is feasible, but complex and error-prone, while relying on code reviews by professional programmers is costly and difficult to systematize. In contrast, the embodiments of this application provide a systematic solution that can automatically and accurately track performance and memory usage during code execution, significantly improving the efficiency and operability of performance optimization.

[0223] The core of this application's embodiments lies in providing a tool capable of analyzing the performance and memory usage of deep learning models at the code granularity level. By dynamically injecting code to monitor Python code execution in real time, it collects data on the execution time and memory usage of each line of code, achieving a performance and memory monitoring method accurate to the line of code—a departure from traditional analysis methods limited to the operator or module level. Through automatic code injection, real-time data capture, and the use of efficient data structures to store and process performance data, an automated data collection and processing system is achieved. This system can be non-intrusively integrated into existing development environments, automatically performing performance data collection and preliminary analysis. By generating recommendation algorithms and logic, including how to identify problems and propose effective solutions based on execution time and memory usage data, the system can automatically generate specific performance optimization suggestions based on the collected data, helping developers quickly understand and resolve performance bottlenecks. The user interface design and interaction methods for displaying results, including data visualization methods and techniques, provide an intuitive and easy-to-understand interface, presenting performance analysis results in graphical and report formats. The software architecture supports multiple programming environments and hardware configurations, as well as the scalability of subsequent functions, considering different development environments and application requirements, exhibiting good adaptability and scalability.

[0224] In this embodiment, dynamic code injection technology enables real-time monitoring of the execution time and GPU memory changes for each line of code. Existing performance analysis tools often focus on the operator or module level, while this embodiment provides line-level performance data. This allows developers to pinpoint the specific location of performance bottlenecks, rather than simply knowing the overall performance of a module or operator. An extended performance data collection module captures and analyzes GPU memory-related data, providing detailed reports on GPU memory usage. In addition to traditional execution time analysis, this embodiment integrates GPU memory usage monitoring, which is particularly important for resource-constrained deep learning applications. Developers can optimize lines of code that use inappropriate GPU memory, effectively avoiding memory overflow or unnecessary resource waste. By analyzing the collected detailed performance data, the system automatically identifies problems and proposes targeted optimization strategies. In existing technologies, performance optimization suggestions typically require developers to manually formulate them based on analysis results; this embodiment automatically generates specific optimization suggestions, reducing developer workload and accelerating the optimization process. This embodiment represents a significant advancement in improving the accuracy, convenience, and practicality of performance analysis, providing a more comprehensive and efficient solution compared to existing technologies.

[0225] Based on the foregoing embodiments, this application provides a performance monitoring device, which includes various units and modules included in each unit. It can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0226] Figure 11 This is a schematic diagram of the composition structure of a performance monitoring device provided in an embodiment of this application, as shown below. Figure 11 As shown, the performance monitoring device 1100 includes: an acquisition module 1110 and a monitoring module 1120, wherein:

[0227] The acquisition module 1110 is used to acquire a model script that includes multiple lines of code; the script function corresponding to the model script is implemented through at least one module function, and the part of the code corresponding to the module function is used to call operators to implement the module function;

[0228] The monitoring module 1120 is used to monitor the execution of each line of code in the model script and obtain the performance data of each line of code.

[0229] In some embodiments, the monitoring module 1120 is further configured to acquire monitoring data before execution and monitoring data after execution of each line of code in the model script during the execution of each line of code; and determine the performance data of each line of code based on the difference between the monitoring data before execution and the monitoring data after execution.

[0230] In some embodiments, the monitoring module 1120 is further configured to extract each line of code from the model script sequentially using a code extraction tool during the execution of each line of code in the model script; obtain monitoring data before the execution of each line of code; execute each line of code; and obtain monitoring data after the execution of each line of code after the execution of each line of code is completed.

[0231] In some embodiments, the performance data includes execution time; the monitoring module 1120 is further configured to, in response to the start of execution of each line of code, acquire first timing data of the timer; in response to the completion of execution of each line of code, acquire second timing data of the timer; and determine the difference between the second timing data and the first timing data as the execution time of each line of code.

[0232] In some embodiments, the performance data includes video memory usage data; the monitoring module 1120 is further configured to, in response to the start of execution of each line of code, obtain a first video memory usage through the video memory monitoring interface provided in the model framework; in response to the completion of execution of each line of code, obtain a second video memory usage through the video memory monitoring interface; and determine the difference between the second video memory usage and the first video memory usage as the video memory usage data of each line of code.

[0233] In some embodiments, the monitoring module 1120 is further configured to non-intrusively insert performance monitoring code during the execution of each line of code in the model script; and to monitor the execution of each line of code through the performance monitoring code using dynamic instrumentation technology to obtain performance data for each line of code.

[0234] In some embodiments, the performance data includes resource usage data for at least one dimension; the monitoring module 1120 is further configured to obtain a preset resource usage threshold for each of the dimensions; and generate an alert message corresponding to the target dimension if the resource usage data for the target dimension exceeds the target dimension's resource usage threshold.

[0235] In some embodiments, the monitoring module 1120 is further configured to perform data analysis on the performance data of each line of code to obtain the performance analysis results of the model script; the performance analysis results are used to locate the performance problems of the model script at the level of each line of code.

[0236] In some embodiments, the performance data includes resource usage data of at least one dimension; the monitoring module 1120 is further configured to analyze the resource usage data of each line of code under the dimension based on the performance analysis strategy corresponding to the dimension, and obtain the performance analysis result of the model script under the dimension; wherein, the performance analysis result includes at least one of the following: the location of the code line with performance problems, an alarm message carrying the location of the code line with performance problems, optimization suggestions for the code line with performance problems, and a trend graph of resource usage data changes during the execution of the model script.

[0237] In some embodiments, the performance analysis results include the location of the lines of code with performance problems; the monitoring module 1120 is further configured to obtain a preset resource usage threshold under the dimension; and based on the resource usage threshold and the resource usage data of each line of code under the dimension, determine the location of the lines of code with performance problems.

[0238] In some embodiments, the performance analysis results include optimization suggestions for lines of code with performance issues. The monitoring module 1120 is further configured to obtain the code segment corresponding to the code line position in the multiple lines of code based on the position of the code line with performance issues; input the code segment into a pre-trained large language model to generate optimization suggestions for the code lines with performance issues.

[0239] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this application can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0240] It should be noted that, in the embodiments of this application, if the above-described performance monitoring method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0241] This application provides a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0242] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0243] This application provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0244] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0245] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0246] Figure 12 This application provides a hardware entity diagram of a computer device as an embodiment of the present application, such as... Figure 12 As shown, the hardware entity of the computer device 1200 includes a processor 1201 and a memory 1202, wherein the memory 1202 stores a computer program that can run on the processor 1201, and the processor 1201 executes the program to implement the steps in the method of any of the above embodiments.

[0247] The memory 1202 stores computer programs that can run on the processor. The memory 1202 is configured to store instructions and applications that can be executed by the processor 1201. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 1201 and various modules in the computer device 1200. It can be implemented by flash memory or random access memory (RAM).

[0248] The processor 1201 implements the steps of any of the above-mentioned performance monitoring methods when executing a program. The processor 1201 typically controls the overall operation of the computer device 1200.

[0249] This application provides a computer storage medium that stores one or more programs, which can be executed by one or more processors to implement the steps of the performance monitoring method as described in any of the above embodiments.

[0250] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0251] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0252] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0253] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A performance monitoring method, characterized in that, The method includes: Obtain a model script containing multiple lines of code; the script function corresponding to the model script is implemented through at least one module function, and the code corresponding to the module function is used to call operators to implement the module function; During the execution of each line of code in the model script, the execution of each line of code is monitored using dynamic instrumentation technology to obtain performance data for each line of code.

2. The method according to claim 1, characterized in that, During the execution of each line of code in the model script, dynamic instrumentation technology is used to monitor the execution of each line of code, obtaining performance data for each line of code, including: During the execution of each line of code in the model script, monitoring data before and after the execution of each line of code is obtained. Based on the difference between the monitoring data before and after the execution of each line of code, the performance data of each line of code is determined.

3. The method according to claim 2, characterized in that, During the execution of each line of code in the model script, monitoring data before and after the execution of each line of code is obtained, including: During the execution of each line of code in the model script, each line of code is extracted from the model script sequentially using a code extraction tool. Obtain monitoring data before the execution of each line of code, and then execute each line of code; After each line of code is executed, obtain the monitoring data after the execution of each line of code.

4. The method according to claim 2, characterized in that, The performance data includes execution time; wherein... During the execution of each line of code in the model script, the process of acquiring monitoring data before and after the execution of each line of code includes: acquiring first timing data of the timer in response to the start of execution of each line of code; and acquiring second timing data of the timer in response to the completion of execution of each line of code. The step of determining the performance data of each line of code based on the difference between the monitoring data before and after the execution of each line of code includes: determining the difference between the second timing data and the first timing data as the execution time of each line of code.

5. The method according to claim 2, characterized in that, The performance data includes video memory usage data; wherein... During the execution of each line of code in the model script, the monitoring data before and after the execution of each line of code is obtained, including: in response to the start of execution of each line of code, obtaining the first video memory usage through the video memory monitoring interface provided in the model framework; in response to the completion of execution of each line of code, obtaining the second video memory usage through the video memory monitoring interface. The step of determining the performance data of each line of code based on the difference between the performance parameters before and after execution includes: determining the difference between the second video memory usage and the first video memory usage as the video memory usage data of each line of code.

6. The method according to any one of claims 1 to 5, characterized in that, During the execution of each line of code in the model script, dynamic instrumentation technology is used to monitor the execution of each line of code, obtaining performance data for each line of code, including: Performance monitoring code is inserted non-intrusively during the execution of each line of code in the model script; The performance monitoring code monitors the execution of each line of code to obtain performance data for each line.

7. The method according to any one of claims 1 to 5, characterized in that, The performance data includes resource usage data in at least one dimension; the method further includes: Obtain the preset resource usage threshold for each of the aforementioned dimensions; If the resource usage data for the target dimension exceeds the resource usage threshold for the target dimension, a warning message corresponding to the target dimension will be generated.

8. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The performance data of each line of code is analyzed to obtain the performance analysis results of the model script; the performance analysis results are used to locate the performance problems of the model script at the level of each line of code.

9. The method according to claim 8, characterized in that, The performance data includes resource usage data in at least one dimension; the data analysis of the performance data of each line of code to obtain the performance analysis results of the model script includes: Based on the performance analysis strategy corresponding to the dimension, the resource usage data of each line of code under the dimension is analyzed to obtain the performance analysis results of the model script under the dimension; The performance analysis results include at least one of the following: the location of the code line with the performance problem, an alarm message carrying the location of the code line with the performance problem, optimization suggestions for the code line with the performance problem, and a trend chart of resource usage data during the execution of the model script.

10. The method according to claim 9, characterized in that, The performance analysis results include the locations of code lines with performance issues. Based on the performance analysis strategy corresponding to the dimension, the resource usage data of each line of code under that dimension is analyzed to obtain the performance analysis results of the model script under that dimension, including: Obtain the preset resource usage threshold under the aforementioned dimension; Based on the resource usage threshold and the resource usage data of each line of code in the dimension, the location of the code line with the performance problem is determined.

11. The method according to claim 9, characterized in that, The performance analysis results include optimization suggestions for lines of code with performance issues. Based on the performance analysis strategy corresponding to the dimension, the resource usage data of each line of code under that dimension is analyzed to obtain the performance analysis results of the model script under that dimension, including: Based on the location of the code line with the performance problem, obtain the code segment corresponding to the code line location from the multiple lines of code; The code segment is input into a pre-trained large language model to generate optimization suggestions for lines of code with performance issues.

12. A performance monitoring device, characterized in that, The device includes: An acquisition module is used to acquire a model script that includes multiple lines of code; the script function corresponding to the model script is implemented through at least one module function, and the code corresponding to the module function is used to call operators to implement the module function; The monitoring module is used to monitor the execution of each line of code in the model script through dynamic instrumentation technology, and obtain the performance data of each line of code.

13. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 11.

Images