Acquisition method of kernel calling information of operator, terminal equipment and storage medium

Abstract

The embodiment of the invention provides a method for obtaining kernel calling information of an operator, terminal equipment and a storage medium. The method comprises the steps of obtaining multiple pieces of target format data of multiple training event records for a neural network model; based on each piece of target format data, obtaining an operator tree which corresponds to each training event and is used for representing an operator calling relationship; wherein the operator tree comprises a calling relation between operators and a calling relation between the operators and a kernel; and obtaining kernel calling information of each operator in each training event according to the calling relationship of the operator trees corresponding to each training event. The method is used for achieving the effect of reducing the labor resource cost.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a method for acquiring kernel call information of an operator, a terminal device, and a storage medium. Background Art

[0002] With the development of artificial intelligence technology, the application scenarios of neural network models are becoming more and more extensive, and the demand for neural network models is also increasing. The performance of neural network models needs to be analyzed during training.

[0003] Currently, the performance of neural network models can be analyzed through PyTorch Profiler, and the PyTorch Profiler performance data can be parsed, processed, and visualized through the PyTorch Profiler TensorBoard Plugin (referred to as "plugin" for short).

[0004] However, during the implementation process, the applicant discovered that traditional technology can only obtain kernel call information at the model level during the neural network training process, that is, the kernel call information after the entire neural network model training process is aggregated, and it is impossible to directly obtain fine-grained operator-level kernel call information. In the performance comparison analysis of multiple neural network training processes, there are at least problems of inefficiency and high labor costs. Summary of the invention

[0005] The embodiments of the present application provide a method for obtaining kernel call information of an operator, a terminal device, and a storage medium, so as to achieve the effect of reducing the cost of manual resources.

[0006] In a first aspect, an embodiment of the present application provides a method for obtaining kernel call information of an operator, the method comprising:

[0007] Acquire multiple target format data of multiple training event records for a neural network model;

[0008] Based on each target format data, an operator tree corresponding to each training event is obtained for characterizing the operator call relationship; wherein the operator tree includes the call relationship between operators and between operators and kernels;

[0009] According to the calling relationship of the operator trees corresponding to each training event, the kernel calling information of each operator in each training event is obtained.

[0010] In a possible implementation, the method further includes:

[0011] In response to a call information display request triggered by the user, the kernel call information of each operator in each training event is compared and displayed in the operation visual page.

[0012] In a possible implementation, the kernel call information includes the total number of kernel calls of the operator, the kernel name information called by the operator, and the number of calls of the operator calling the kernel; according to the call relationship of the operator tree corresponding to each training event, the kernel call information of each operator in each training event is obtained, including:

[0013] According to the calling relationship of the operator tree corresponding to each training event, the kernel name information of each kernel called by each operator in each training event, the calling number of each kernel called by each operator and the total calling number of the kernel of each operator are obtained;

[0014] The operation visual page compares and displays the kernel call information of each operator in each training event, including:

[0015] In the operation visual page, the total number of kernel calls of each operator in each training event is compared and displayed, and the kernel information display control of each operator calling the kernel in multiple training events is displayed;

[0016] In response to a triggering operation on at least one kernel information display control, kernel name information of each kernel called by an operator corresponding to the kernel information display control and the number of calls of each kernel are displayed.

[0017] In a possible implementation, in response to a call information display request triggered by a user, the kernel call information of each operator in each training event is displayed comparatively in the operation visual page, including:

[0018] In response to the call information display request, obtain kernel name information corresponding to each kernel called by each operator in each training event and the number of times each operator calls each kernel, which are transmitted in a preset transmission form;

[0019] Convert the kernel name information and call count corresponding to each kernel called by each operator in each training event in a preset transmission format to obtain kernel call information for display;

[0020] The kernel call information of each operator in each training event is displayed in the operation visual page.

[0021] In a possible implementation, obtaining kernel name information corresponding to each kernel called by each operator in each training event and the number of times each operator calls each kernel, which are transmitted in a preset transmission form, includes:

[0022] According to the kernel name information of each kernel called by each operator in each training event and the number of calls of each kernel, a kernel call dictionary for each operator is obtained;

[0023] The kernel call dictionary is converted into data to obtain kernel name information of each kernel called by each operator in a preset transmission form and the number of calls of each kernel.

[0024] In a possible implementation, in response to a call information display request, obtaining kernel name information corresponding to each kernel called by each operator in each training event and the number of times each operator calls each kernel, which are transmitted in a preset transmission form, includes:

[0025] In response to the call information display request, the kernel name information corresponding to each kernel called by each operator in each training event and the call times of each kernel transmitted in a preset transmission form are transmitted to the front-end module of the display device.

[0026] In one possible implementation, the preset transmission format is a string format, the kernel name information of each kernel called by each operator is transmitted in the form of a first string list, and the number of calls of each kernel called by each operator is transmitted in the form of a second string list corresponding to the first string list.

[0027] In a possible implementation, the method further includes:

[0028] In response to a call information display request triggered by a user, the kernel call information of each operator in each training event is displayed in the operation visual page; the kernel call information includes the total number of kernel calls by the operator;

[0029] In the operation visual page, the difference in the total number of kernel calls of each operator between multiple training events is displayed; the difference in the total number of kernel calls is obtained based on the total number of kernel calls in multiple training events.

[0030] In a second aspect, an embodiment of the present application provides a terminal device, including: a memory, a processor;

[0031] Memory stores computer-executable instructions;

[0032] The processor executes the computer-executable instructions stored in the memory, so that the processor executes various possible implementations of the method as described above.

[0033] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement various possible implementation methods of the method as described above.

[0034] Compared with the traditional technology, the method for obtaining the kernel call information of the operator, the terminal device and the storage medium provided in the embodiments of the present application can help users directly obtain the kernel call information of each operator calling the kernel by constructing an operator tree while aggregating the kernel call information of each operator calling the kernel, so as to realize the comparative analysis of the neural network model performance in multiple neural network training processes (for example, the back-end migration process or the deep learning framework version upgrade process), thereby reducing the human resource cost of the neural network model performance analysis and improving the efficiency of the neural network model performance analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] Figure 1 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 1 ;

[0037] Figure 2 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 2 ;

[0038] Figure 3 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 3 ;

[0039] Figure 4 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 4 ;

[0040] Figure 5 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 5 ;

[0041] Figure 6 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 6 ;

[0042] Figure 7 A schematic diagram of the structure of a device for acquiring kernel call information of an operator provided in this application;

[0043] Figure 8 A schematic diagram of the structure of the terminal device provided for this application.

[0044] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] With the development of artificial intelligence technology, the application scenarios of neural network models are becoming more and more extensive, and the demand for neural network models is also increasing. The performance of neural network models needs to be analyzed during training. For example, the performance of the neural network training process can be analyzed through the PyTorch Profiler application. The PyTorch ProfilerTensorBoard Plugin is a TensorBoard plug-in officially open sourced by PyTorch for visualizing PyTorch Profiler performance data. It takes the json format performance data saved by the PyTorch Profiler as input, parses and processes the data through the back-end module, and interacts with the front-end module. Finally, it is visualized on the front-end of the web page as the output result, which is convenient for developers to perform performance analysis during neural network training.

[0047] Currently, the front end of the plug-in's web panel can display the performance diagnosis perspective of a single neural network training process, and also provides a comparative display function for two neural network training processes. Among them, by viewing the "Operator View" perspective of a single neural network training process, you can obtain specific statistical information for each CPU operator, including the total number of times the CPU operator is called, the running time on the Host side, the running time on the Device side, etc.; by viewing the "OperatorView" perspective of two neural network training processes, you can obtain the difference statistical information of each CPU operator being called during the two runs, including a comparison of the total number of times the operator is called and a comparison of the running time when the operator is called, etc. That is, in the related technology, you can view the relevant information of calling the CPU operator (the relevant information of the CPU operator being called).

[0048] However, the applicant found in the implementation process that the above function lacks the automatic acquisition and display of the information of the operator calling the kernel during multiple neural network training processes, resulting in the problem of consuming a lot of human resources in the performance comparison analysis of the neural network, relying on manual viewing of the Chrome trace timing diagram of the two neural network training processes to visually compare and summarize the process of the operator calling the kernel. Especially when the neural network is large in scale and calls many operators, the number of operator calls to the kernel is even greater, which is easy to cause high human resource costs, inefficient work, and easy to make mistakes.

[0049] The method for obtaining the kernel call information of the operator provided in the present application, by constructing an operator tree and aggregating the kernel call information of each operator calling the kernel, can help users directly obtain the kernel call information of each operator calling the kernel, so as to realize the performance comparison and analysis of the neural network model in multiple neural network training processes, thereby helping to solve the technical problem of high labor cost when performing performance comparison and analysis in multiple neural network training processes.

[0050] Furthermore, the method for obtaining the kernel call information of the operator provided in the present application can also compare and display the kernel call information of each operator in each training event in the operation visual page when the user initiates a call information display request, thereby realizing a visualization of the performance comparison of multiple neural network trainings.

[0051] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0052] The method for obtaining the kernel call information of the operator provided in this embodiment is illustrated by applying it to a terminal. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Figure 1 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 1 ,like Figure 1 As shown, the method includes S101 to S103, wherein:

[0053] S101, obtaining multiple target format data of multiple training event records for a neural network model.

[0054] The neural network model may be any neural network that needs to be trained. Multiple training events refer to multiple trainings of the neural network model. The target format data may be data obtained by recording various events in the neural network training process, and the target format data may be a performance data file in json format, for example, performance data generated by PyTorchProfiler.

[0055] Exemplarily, the terminal can record the performance data of the neural network model training during the neural network training process to obtain the target format data, and the back-end processing of the terminal can read the various target format data saved by the terminal. It can be understood that in the multiple training processes of the neural network model, multiple target format data corresponding to the multiple training processes can be obtained. For example, the terminal can turn on the PyTorch Profiler function to record various events in the neural network model training process and obtain a file in json format.

[0056] Optionally, multiple training events refer to multiple neural network model trainings, which can be multiple trainings of the same neural network model of different software versions on the same device, or multiple trainings of the same neural network model before and after migration on different devices. For example, if a performance comparison is made on the training of the neural network model before and after the migration of the back-end device, the training is performed on back-end device 1 and back-end device 2 respectively, and then json file 1 corresponding to back-end device 1 and json file 2 corresponding to back-end device 2 can be obtained. For another example, if a performance comparison is made on the training of the neural network model before and after a version update or upgrade, different versions of the neural network model can be trained multiple times on the same device to obtain json file A corresponding to version A of the neural network model and json file B corresponding to version B of the neural network model.

[0057] S102, based on each target format data, obtain an operator tree corresponding to each training event for characterizing the operator calling relationship; wherein the operator tree includes the calling relationship between operators and between operators and kernels.

[0058] The operator call relationship may include the call relationship between operators (for example, the relationship between a parent operator calling a child operator) and the call relationship between an operator calling a kernel. The operator tree refers to a data structure containing the operator call relationship.

[0059] Exemplarily, the terminal can read various target format data, and the back-end processing of the terminal can parse and process various target format data to construct operator trees corresponding to each training event. The operator tree can include call information between operators and information about operator calling kernels, so as to aggregate the call information of the operator calling the kernel.

[0060] In practical applications, based on the TensorBoard Plugin plug-in tool of PyTorch Profile, the implementation method of the present application can be used to modify or expand its implementation, so that it can execute the method or steps in the embodiment provided by the present application, so as to obtain a visualization plug-in for executing the method or steps provided by the present application. The json format file (performance data file) obtained by multiple training can be read through the visualization plug-in, each json format file is parsed and processed, and various events in the json format file are parsed to build the corresponding operator node, in which the operator name, start time, id number, etc. can be saved. The visualization plug-in can be based on the operator node, and the operator tree can be constructed to obtain the call relationship between the operator and the Kernel (kernel). In this way, it is conducive to comparing and displaying the call information of the kernel in the front-end module of the terminal, avoiding the loss of human resources caused by the call relationship in the process of manually viewing multiple neural network model training by developers.

[0061] S103, obtaining kernel calling information of each operator in each training event according to the calling relationship of the operator trees corresponding to each training event.

[0062] The kernel call information refers to information about an operator calling a kernel. For example, the kernel call information may include kernel name information of the operator calling the kernel, the number of times the kernel is called, the total number of times the kernel is called, etc.

[0063] For example, the terminal can obtain all operators and kernels by traversing the operator tree, and aggregate statistics on the operators and kernels to obtain statistical information of operator calls to the kernel. In this way, kernel call information corresponding to multiple trainings can be obtained.

[0064] In this embodiment, through the multiple target format data recorded corresponding to the multiple training events, it is possible to construct the operator trees corresponding to the multiple training events, and aggregate the kernel call information of each operator calling the kernel while constructing the operator tree, so as to facilitate the user to directly obtain the kernel call information of each operator calling the kernel, so as to realize the comparative analysis of the performance of the neural network model in multiple neural network training processes (for example, the backend migration process or the deep learning framework version upgrade process) directly based on the aggregated kernel call information, thereby reducing the human resource cost of the performance analysis of multiple neural network trainings and improving the efficiency of the performance analysis of the neural network model. At the same time, it is conducive to the subsequent comparative display of the kernel call information of each operator in each training event in the operation visual page, so as to make the kernel call information intuitively visualized, and the user can directly obtain the kernel call information of multiple training events for the neural network model from the operation visual page.

[0065] In an exemplary embodiment, Figure 2 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 2 ,like Figure 2 As shown, based on S101 to S103, the method further includes S104, wherein:

[0066] S104, in response to a call information display request triggered by the user, the kernel call information of each operator in each training event is compared and displayed in the operation visual page.

[0067] The call information display request refers to a function for displaying kernel call information of multiple neural network model trainings. The operation visual page can be a page modified or expanded based on the Operator View perspective, and the Operator View perspective refers to the perspective of the corresponding function of the PyTorch Profile. The operation visual page can be a page on the web page or a page on the application side.

[0068] For example, a user can trigger a request to display the call information of the kernel in the terminal, and the terminal can respond to the request by comparing and displaying the kernel call information of multiple trainings of the neural network in the operation visual page. In this way, the developer can reduce the comparison and viewing operation of the kernel call information of the operator in multiple trainings of the neural network model, thereby reducing the cost of manual resources.

[0069] For example, if the visualization plug-in installed on the terminal responds to the call information display request sent by the user's click, the front-end module of the visualization plug-in can obtain the kernel call information required for display from the back-end module, and visualize the kernel call information on the operation visual page of the web page.

[0070] In practical applications, by comparing the kernel call information of each operator in each training event, it is possible to assist in determining the performance differences of the neural network model in each training event. In general, multiple calls to the kernel by the same operator often result in performance degradation. Through the method provided in the embodiment of the present application, the user can intuitively understand the number of times the kernel is called, and can intuitively compare the number of kernel calls for multiple neural network trainings, which can visualize the performance of multiple trainings and reduce the manual resource cost of performance analysis of multiple training neural network models.

[0071] In some of the embodiments, the above steps S101 to S103 can be performed in real time during multiple training of neural network models, and the kernel call information of the operator calling the Kernel is statistically processed in the backend, and when the user triggers a call information display request, the processed kernel call information is directly displayed.

[0072] In other embodiments, in the above step S104, after the user triggers the call information display request, the above steps S101 to S103 are performed, that is, after responding to the call information display request triggered by the user, the steps S101 to S103 are performed. Therefore, the execution of the above steps S101 to S104 is not necessarily strictly limited in order, and these steps can also be performed in other orders.

[0073] In this embodiment, when the user initiates a request to display the call information, the kernel call information of each operator in each training event can be compared and displayed in the operation visual page. The user can directly obtain the kernel call information of multiple training events of the neural network model from the operation visual page, and realize the visualization of the performance analysis comparison of multiple neural network trainings, thereby further reducing the human resource cost of the performance analysis comparison of multiple neural network trainings.

[0074] In an exemplary embodiment, Figure 3 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 3 ,like Figure 3 As shown, the kernel call information includes the total number of kernel calls of the operator, the kernel name information called by the operator, and the number of calls of the operator calling the kernel; in this embodiment, Figure 1 and Figure 2 Based on the embodiment, a method for obtaining kernel call information of an operator is described in detail.

[0075] In step S103, according to the calling relationship of the operator trees corresponding to the training events, the kernel calling information of each operator in each training event is obtained, which may specifically include:

[0076] S301, according to the calling relationship of the operator tree corresponding to each training event, obtain the kernel name information of each kernel called by each operator in each training event, the calling number of each kernel called by each operator and the total calling number of the kernel of each operator.

[0077] The kernel name information refers to the name of the kernel called in the operator calling kernel, which can be used as the kernel identifier. The kernel call count refers to the number of times the kernel is called by an operator in a single training event.

[0078] Exemplarily, for each neural network model training, the terminal visualization plug-in can obtain the kernel name information of each kernel called by the operator, the number of times the kernel is called, and the total number of kernel calls of all kernels called by each operator based on the kernel information called by the operator in the operator tree corresponding to the training. In this way, the kernel name information of the operator calling the kernel, the number of times the kernel is called, and the total number of kernel calls can be further displayed on the front end, so that the user can intuitively understand the number of times the kernel is called by a certain operator, and can intuitively compare the number of times each operator calls the kernel in multiple neural network trainings, so as to realize the performance visualization display of multiple neural network trainings, thereby reducing the manual resource cost of performance analysis of multiple training neural network models.

[0079] Optionally, for each neural network model training, the backend module of the terminal visualization plug-in can aggregate the operator call information of the operator tree corresponding to this training, obtain all kernels called by each operator and its sub-operators, aggregate and count the specific call information of all kernels, and obtain the aggregated kernel information, which can include the total number of calls of each operator to the kernel, the call name of each kernel, and the corresponding number of calls of each kernel. In this way, it can be helpful to realize the visualization display of the number of calls of each operator to the kernel in each neural network training.

[0080] In step S104, the kernel call information of each training event is compared and displayed in the operation visual page, which may specifically include:

[0081] S302, in the operation visual page, compare and display the total kernel call times of each operator in each training event, and display the kernel information display control of each operator calling the kernel in multiple training events.

[0082] The total number of kernel calls may be the total number of times a certain operator calls all kernels.

[0083] Optionally, in the terminal's operation visual page, the kernel call information of each training event can be displayed separately, for example, it can be displayed in multiple columns. If there are two training events, the kernel call information of the two training events can be displayed in two columns respectively.

[0084] Exemplarily, in the terminal's operation visual page, the total kernel call times of multiple training events can be displayed simultaneously, and the kernel information display controls corresponding to each training event can be displayed. The user can choose to click the control to further display the detailed kernel information of the called kernel.

[0085] S303: In response to a triggering operation on at least one kernel information display control, kernel name information of each kernel called by an operator corresponding to the kernel information display control and the number of times each kernel is called are displayed.

[0086] Optionally, when a user triggers a click operation on a kernel information display control, the terminal can respond to the trigger operation and display the kernel name information of each kernel called by the operator corresponding to the kernel information display control and the number of calls of each kernel in the terminal's operation visual page. For example, the kernel name information and the corresponding number of calls of each operator call can be displayed in the form of a visual table, and the user can easily and conveniently view the kernel call difference statistics of each operator during multiple neural network training processes, which improves the analysis efficiency of specific performance issues, thereby helping to reduce the cost of human analysis.

[0087] In this embodiment, by displaying the total number of kernel calls for each training event in the operation visual page, the user can more intuitively understand the number of times each operator calls the kernel in each neural network training, and a kernel information display control is provided to provide the user with the option of viewing the kernel name information of each kernel called and the number of calls of each kernel, thereby realizing a multi-select visual display of kernel information, which is conducive to reducing the human resource cost of multiple neural network model training performance analysis.

[0088] In an exemplary embodiment, Figure 4 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 4 ,like Figure 4 As shown, in this embodiment Figure 1 and Figure 2 Based on the above, step S104 is described in detail. In step S104, in response to the call information display request triggered by the user, the kernel call information of each operator in each training event is compared and displayed in the operation visual page, including:

[0089] S401, in response to a call information display request, obtaining kernel name information corresponding to each kernel called by each operator in each training event and the number of times each operator calls each kernel, which are transmitted in a preset transmission form.

[0090] S402, converting the kernel name information and the calling times corresponding to each kernel called by each operator in each training event in a preset transmission format to obtain kernel calling information for display.

[0091] S403, displaying kernel call information of each operator in each training event for display in the operation visual page.

[0092] The preset transmission form refers to transmission in a preset specific format. Format conversion refers to converting the kernel call information in the preset transmission form into kernel call information that can be used for front-end display.

[0093] Exemplarily, the terminal can respond to a call information display request, and through the terminal's visualization plug-in, can transmit the kernel name information of each operator calling each kernel and the number of calls of each kernel by each operator obtained by parsing the operator tree by the back-end module to the front-end module of the visualization plug-in in a preset transmission form. In this way, the front-end module of the visualization plug-in can obtain the kernel name information and the number of calls corresponding to each kernel called by the operator in each training event transmitted in the preset transmission form, and can convert the kernel name information and the number of calls in the preset transmission form to obtain the kernel call information that can be used for display in the operation visual page. After the conversion is completed, the kernel call information of each operator calling the kernel in each training event for display can be displayed in the terminal's operation visual page.

[0094] In addition, the total number of kernel calls can be transmitted in the form of an integer, so there is no need to convert the total number of kernel calls, and the total number of kernel calls can be displayed in the operation visual page in the form of an integer. It can be understood that in step S403, the kernel call information used for display includes the kernel name information and call count after format conversion, and the total number of kernel calls transmitted in the form of an integer, that is, the kernel name information, call count and total number of kernel calls of each operator in each training event can be displayed in the operation visual page.

[0095] In this embodiment, by transmitting the kernel name information and the number of calls corresponding to each kernel called by each operator in each training event in a preset transmission form, it is possible to visualize the kernel name information corresponding to each operator calling each kernel and the number of calls each operator calling each kernel, thereby reducing the human resource cost of comparative analysis of multiple neural network model training performances.

[0096] Optionally, in step S401, obtaining kernel name information corresponding to each kernel called by each operator in each training event and the number of times each operator calls each kernel, which are transmitted in a preset transmission form, includes:

[0097] According to the kernel name information of each kernel called by each operator in each training event and the number of calls of each kernel, a kernel call dictionary for each operator is obtained;

[0098] The kernel call dictionary is converted into data to obtain kernel name information of each kernel called by each operator in a preset transmission form and the number of calls of each kernel.

[0099] The kernel call dictionary refers to kernel call data stored or recorded in a dictionary structure.

[0100] Exemplarily, for each training event, the backend module of the visualization plug-in of the terminal can construct a dictionary structure containing the kernel call information of the operator based on the kernel name information of each kernel called by each operator in the training event and the call count of each kernel, and obtain the kernel call dictionary of each operator. The backend module of the visualization plug-in can convert the kernel name information and the call count of each operator calling the kernel in the kernel call dictionary of each operator into the kernel name information and the call count in a preset transmission form, thereby facilitating data transmission from the backend to the frontend.

[0101] In practical applications, using the constructed dictionary structure data to convert the transmission format can ensure the accuracy of the above data transmission and avoid transmission errors from the back end to the front end, which is conducive to improving the accuracy of multiple neural network model training performance analysis.

[0102] Optionally, in step S401, in response to the call information display request, obtaining kernel name information corresponding to each kernel called by each operator in each training event and the number of times each operator calls each kernel, which are transmitted in a preset transmission form, includes:

[0103] In response to the call information display request, the kernel name information corresponding to each kernel called by each operator in each training event and the call times of each kernel transmitted in a preset transmission form are transmitted to the front-end module of the display device.

[0104] Exemplarily, the terminal can respond to the call information display request, and can transmit the kernel name information corresponding to each kernel called in each training event and the number of calls of each kernel in a preset transmission form to the front-end module of the display device by calling the kernel information visualization plug-in, which is conducive to the front-end module of the display device to display the kernel call information. Through the kernel information visualization plug-in, the kernel call information can be transmitted simply and conveniently, and the efficiency of visual display of kernel call information is improved. Among them, the kernel information visualization plug-in can be a visualization plug-in modified or expanded based on the TensorBoard Plugin plug-in tool of PyTorch Profiler, and the kernel information visualization plug-in can be used to implement the implementation method provided in this application.

[0105] In actual applications, the preset transmission format is a string format, the kernel name information of each kernel called by each operator is transmitted in the form of a first string list, and the number of calls of each kernel called by each operator is transmitted in the form of a second string list corresponding to the first string list.

[0106] Exemplarily, the kernel call information can be converted into a string and transmitted in the form of a string. The kernel name information of each kernel can be converted into a first string list, the number of calls of each kernel can be converted into a second string list, and the first string list and the second string list can be transmitted from the back end of the device to the front end of the device respectively. In this way, it is conducive to ensuring the accuracy of the transmission of the kernel call information.

[0107] In an exemplary embodiment, Figure 5 Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 5 ,like Figure 5 As shown, in this embodiment Figure 1 and Figure 2 Based on the embodiment, a method for obtaining kernel call information of an operator is described in detail; the method also includes:

[0108] S501, in response to a call information display request triggered by a user, display the kernel call information of each operator in each training event in the operation visual page; the kernel call information includes the total kernel call times of the operator calling the kernel.

[0109] S502, in the operation visual page, display the difference in the total number of kernel calls of each operator between multiple training events; the difference in the total number of kernel calls is obtained based on the total number of kernel calls in multiple training events.

[0110] The number difference may be the difference in the total number of kernel calls by the operator between two previous and subsequent training events.

[0111] Exemplarily, in response to a call information display request triggered by a user, the terminal can display the total number of kernel calls of each operator in each training event in the operation visual page, which is helpful for users to compare the total number of kernel calls of multiple neural network model trainings. The terminal can obtain the difference in the total number of kernel calls of each operator calling the kernel between multiple training events based on the total number of kernel calls of each operator in multiple training events, and the terminal can also display the difference in the number of kernel calls of each operator calling the kernel between multiple training events in the operation visual page. For example, if there are two neural network model trainings, the difference in the total number of kernel calls of each operator in the two trainings can be displayed in the operation visual page.

[0112] In this embodiment, by displaying the kernel call information of each operator in each training event in the operation visual page, the total kernel call times of each operator calling the kernel in each training can be more intuitively understood, and in the operation visual page, the difference in the kernel call times of each operator between multiple training events is displayed, which can more intuitively compare the total kernel call times of each operator calling the kernel in multiple trainings, thereby further reducing the human resource cost of multiple neural network model training performance analysis.

[0113] In an exemplary embodiment, in step S102, based on each target format data, an operator tree corresponding to each training event for representing an operator call relationship is obtained, including:

[0114] An operator call event contained in the target format data is obtained, and a plurality of operator nodes are obtained according to the operator call event; wherein the operator node is used to characterize basic attributes of the operator call event.

[0115] Determine the calling relationship between each operator node and the calling information of each operator node.

[0116] An operator tree is constructed based on the calling relationship between operator nodes and the calling information of each operator node.

[0117] Exemplarily, the visualization plug-in of the terminal can read the target format data (such as json file), and the target format data mainly records various events, including operator events, runtime events, kernel events, module events, etc. The visualization plug-in can parse various events in the json file and construct corresponding nodes, including operator nodes, runtime nodes, kernel nodes, module nodes, etc. The nodes store the basic attributes of the corresponding events, such as the operator name, start time, id number, etc. in the operator node. The visualization plug-in can build an operator tree based on all operator nodes, that is, the nodes of the operator tree are all from the operator nodes. During the construction of the operator tree, the operator call relationship is recorded by assigning the child members in the operator node, and the operator calling the runtime function is recorded by assigning the runtime members in the operator node. The kernel member in the runtime node records the underlying kernel node called, so that the calling relationship between the operator and the kernel can be fully recorded.

[0118] In this embodiment, an operator tree is constructed through the calling relationship between each operator node and the calling information of each operator node, and the kernel calling information is aggregated and counted, so that the kernel calling information of each operator can be directly obtained, and it helps to ensure the correctness of the kernel calling information of each operator.

[0119] In an exemplary embodiment, Figure 6Schematic diagram of the process of obtaining the kernel call information of the operator provided in this application Figure 6 ,like Figure 6 As shown, taking two neural network model trainings as an example, the method includes:

[0120] S601, read the json file. In practical applications, during the training of the neural network model, it is necessary to enable the PyTorchProfiler function to record various events of the training process and export them to a json file. The terminal can start the visualization plug-in and read the above-mentioned saved two training json files. Among them, the visualization plug-in can be obtained according to the above-mentioned embodiment.

[0121] S602, parse the original event and build the corresponding node. In actual application, various events in the json file are parsed to build corresponding data nodes. The data nodes may include operator nodes, runtime nodes, kernel nodes, module nodes, etc. The nodes store the basic attributes of the corresponding events, such as the operator name, start time, id number, etc. in the operator node.

[0122] S603, construct an operator tree and record the operator call relationship. In actual applications, an operator tree is constructed based on the operator nodes. The nodes of the operator tree all come from the constructed operator nodes. During the operator tree construction process, the operator call relationship is recorded by assigning the child members in the operator node, and the operator calls the runtime function by assigning the runtime members in the operator node. The Kernel member in the runtime node records the underlying Kernel node called, so that the call relationship between the operator and the Kernel can be fully recorded.

[0123] S604, traverse the operator tree, aggregate operator and kernel statistical information. In actual applications, by traversing the operator tree, all operators and kernels can be obtained, and then further aggregation operations can be performed to obtain aggregated statistical information of operators and kernels. For example, when aggregating operator information, the operator name can be used as the index key to aggregate the number of calls, HOST-side execution time, and DEVICE-side execution time to obtain statistical information of the corresponding attributes. In the process of traversing the operator tree, when aggregating operator information, all kernels called by the operator and its sub-operators are obtained at the same time, and the specific call information of all kernels is aggregated and counted. The aggregated information includes the total number of kernel calls, the name of each kernel call, and the corresponding number of calls.

[0124] S605, respond to the request of the front-end module and transmit the parsed information. In actual application, by viewing the operation visual page (e.g., "Operator View" perspective) of the two neural network model training processes, the difference in the total number of kernel calls of each operator in the two running processes is displayed in the operation visual page, and the specific kernel name and corresponding number of calls called by the operator in the two running processes are displayed in a visual table.

[0125] In this embodiment, the above-mentioned processing method and visual display of operator call Kernel information are helpful for users to view the Kernel call difference statistical information of each operator in two neural network training processes, thereby improving the user's analysis efficiency of specific performance problems, thereby reducing the user's human resource cost for neural network performance analysis.

[0126] Figure 7 A schematic diagram of the structure of the device for obtaining the kernel call information of the operator provided in this application, such as Figure 7 As shown, the device 70 for acquiring the kernel call information of the operator provided in this embodiment includes:

[0127] The target data acquisition module 701 is used to acquire multiple target format data of multiple training event records for the neural network model.

[0128] The operator tree acquisition module 702 is used to obtain the operator tree corresponding to each training event and used to characterize the operator call relationship based on each target format data; wherein the operator tree includes the call relationship between operators and between operators and kernels.

[0129] The kernel call acquisition module 703 is used to obtain the kernel call information of each operator in each training event according to the call relationship of the operator tree corresponding to each training event.

[0130] In one possible implementation, Figure 7 As shown, the kernel call information display device also includes an information display module 704, and the information display module 740 is used to respond to a call information display request triggered by a user, and compare and display the kernel call information of each operator in each training event in the operation visual page.

[0131] In a possible implementation, the kernel call information includes the total kernel call times of the operator, the kernel name information of the operator call, and the call times of the operator call kernel. The kernel call acquisition module 703 is used to obtain the kernel name information of each kernel called by each operator in each training event, the call times of each operator calling each kernel, and the total kernel call times of each operator according to the call relationship of the operator tree corresponding to each training event. The information display module 704 is used to compare and display the total kernel call times of each operator in each training event in the operation visual page, and display the kernel information display control of each operator calling the kernel in multiple training events; in response to the triggering operation of at least one kernel information display control, display the kernel name information of each kernel called by the operator corresponding to the kernel information display control and the call times of each kernel.

[0132] In one possible implementation, the information display module 704 is used to respond to a call information display request, obtain kernel name information corresponding to each kernel called by each operator in each training event and the number of calls of each kernel by each operator, which are transmitted in a preset transmission form; convert the kernel name information and the number of calls corresponding to each kernel called by each operator in each training event in a preset transmission form to obtain kernel call information for display; and display the kernel call information of each operator in each training event for display in an operation visual page.

[0133] In one possible implementation, the information display module 704 is used to obtain a kernel call dictionary for each operator based on the kernel name information of each kernel called by each operator in each training event and the number of calls of each kernel; the kernel call dictionary is converted into data to obtain the kernel name information of each kernel called by each operator in a preset transmission form and the number of calls of each kernel.

[0134] In one possible implementation, the information display module 704 is used to respond to a call information display request and transmit the kernel name information corresponding to each kernel called by each operator in each training event and the number of calls of each kernel in a preset transmission form to the front-end module of the display device.

[0135] In one possible implementation, the preset transmission format is a string format, the kernel name information of each kernel called by each operator is transmitted in the form of a first string list, and the number of calls of each kernel called by each operator is transmitted in the form of a second string list corresponding to the first string list.

[0136] In one possible implementation, the information display module 704 is also used to respond to a call information display request triggered by a user, and display the kernel call information of each operator in each training event in the operation visual page; the kernel call information includes the total kernel call times of the operator calling the kernel; in the operation visual page, the difference in the total kernel call times of each operator between multiple training events is displayed; the difference in the total kernel call times is obtained based on the total kernel call times in multiple training events.

[0137] The kernel call information display device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0138] Figure 8 This is a schematic diagram of the structure of the terminal device provided in this application. Figure 8 As shown, the terminal device 80 provided in this embodiment includes: at least one processor 801 and a memory 802. Optionally, the device 80 also includes a communication component 803. The processor 801, the memory 802 and the communication component 803 are connected via a bus 804.

[0139] In a specific implementation process, at least one processor 801 executes the computer-executable instructions stored in the memory 802, so that at least one processor 801 executes the above method.

[0140] The specific implementation process of the processor 801 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0141] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0142] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0143] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0144] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0145] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0146] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0147] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0148] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0149] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0150] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0151] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0152] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0153] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for obtaining kernel call information of an operator, characterized in that: The method comprises: Acquire multiple target format data of multiple training event records for a neural network model; Based on each of the target format data, an operator tree corresponding to each of the training events is obtained for characterizing the operator call relationship; wherein the operator tree includes the call relationship between operators and between operators and kernels; According to the calling relationship of the operator trees corresponding to the training events, the kernel calling information of each operator in the training events is obtained.

2. The method according to claim 1, characterized in that The method further comprises: In response to a call information display request triggered by a user, the kernel call information of each operator in each of the training events is comparatively displayed in the operation visual page.

3. The method according to claim 2, characterized in that The kernel call information includes the total number of kernel calls of the operator, the kernel name information called by the operator and the number of kernel calls called by the operator; The obtaining, according to the calling relationship of the operator trees corresponding to the training events, the kernel calling information of each operator in each training event includes: According to the calling relationship of the operator trees corresponding to the training events, kernel name information of each kernel called by each operator in the training events, the number of times each operator calls each kernel, and the total number of kernel calls of each operator are obtained; The comparative display of the kernel call information of each operator in each training event in the operation visual page includes: In the operation visual page, the total number of kernel calls of each operator in each of the training events is compared and displayed, and the kernel information display control of each operator calling the kernel in multiple training events is displayed; In response to a triggering operation on at least one of the kernel information display controls, kernel name information of each kernel called by the operator corresponding to the kernel information display control and the number of times each kernel is called are displayed.

4. The method according to claim 2, characterized in that: In response to the call information display request triggered by the user, the kernel call information of each operator in each training event is displayed comparatively in the operation visual page, including: In response to the call information display request, obtaining kernel name information corresponding to each kernel called by each operator in each training event and the number of times each operator calls each kernel, which are transmitted in a preset transmission form; Convert the kernel name information and the calling times corresponding to each kernel called by each operator in each training event in the preset transmission form to obtain kernel calling information for display; The kernel call information of each operator in each of the training events is displayed in the operation visual page.

5. The method according to claim 4, characterized in that The obtaining of kernel name information corresponding to each kernel called by each operator in each training event and the number of times each operator calls each kernel, which are transmitted in a preset transmission form, includes: Obtaining a kernel call dictionary for each operator according to kernel name information of each kernel called by each operator in each training event and the number of calls of each kernel; The kernel call dictionary is subjected to data conversion to obtain kernel name information of each kernel called by each operator in a preset transmission form and the number of calls of each kernel.

6. The method according to claim 5, characterized in that The step of obtaining, in response to the call information display request, kernel name information corresponding to each kernel called by each operator in each training event and the number of times each operator calls each kernel, transmitted in a preset transmission form, includes: In response to the call information display request, the kernel name information corresponding to each kernel called by each operator in each training event and the call count of each kernel transmitted in the preset transmission form are transmitted to the front-end module of the display device.

7. The method according to claim 6, characterized in that The preset transmission form is a string form, the kernel name information of each kernel called by each operator is transmitted in the form of a first string list, and the number of calls of each kernel called by each operator is transmitted in the form of a second string list corresponding to the first string list.

8. The method according to claim 2, characterized in that: The method further comprises: In response to a call information display request triggered by a user, kernel call information of each operator in each training event is displayed in the operation visual page; the kernel call information includes the total number of kernel calls by the operator calling the kernel; In the operation visual page, the difference in the total number of kernel calls of each operator between multiple training events is displayed; the difference in the total number of kernel calls is obtained based on the total number of kernel calls in multiple training events.

9. A terminal device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

Images