Method, device, computer equipment and storage medium for obtaining operator hardware time

Abstract

Embodiments of the present invention relate to a method, apparatus, computer equipment, and storage medium for obtaining operator hardware time, and relate to the field of deep learning technology. The method includes sending target data to a machine learning processor, which is configured with a pre-designed neural network model; calling multiple operators in the neural network model to process the target data; calling a target management plug-in to record the operation start information and operation end information of each operator; and determining the operator hardware time of each operator based on the operation start information and operation end information of each operator. In the scheme of the present invention, the purpose of supporting different types of heterogeneous devices to obtain operator hardware time is achieved through the target management plug-in. In addition, the modification to the Pytorch framework is reduced, the degree of coupling with the Pytorch framework is reduced, and on the basis of increasing support for the machine learning processor, the use of the original CPU / CUDA in Pytorch is not affected.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method, apparatus, computer equipment, and storage medium for obtaining operator hardware time. Background Art

[0002] Pytorch is a mainstream deep learning framework that is widely used to build neural network models. It can use heterogeneous devices for computing acceleration and is widely used in applications such as computer vision and natural language processing.

[0003] When using heterogeneous devices for computational acceleration, understanding the hardware execution status of operators on these devices is crucial. By analyzing the hardware execution of operators, users can quickly build the model architecture of AI (Artificial Intelligence) neural networks, obtain information such as the operator call sequence and call stack, and use this information to identify performance bottlenecks and optimize the neural network.

[0004] Currently, the profiler tool provided by Pytorch for deep learning neural networks only supports a limited number of heterogeneous devices and does not support other types of heterogeneous devices besides GPUs (Graphics Processing Units). Summary of the Invention

[0005] In order to at least partially solve the technical problems mentioned in the background technology, the solution disclosed in the present invention provides a method, apparatus, computer device and storage medium for obtaining operator hardware time, which can support different types of heterogeneous devices to obtain operator hardware time.

[0006] In a first aspect, the present invention discloses a method for obtaining operator hardware time, the method comprising: sending target data to a machine learning processor, on which a pre-designed neural network model is configured; in the process of calling multiple operators in the neural network model to process the target data, calling a target management plug-in to record the operation start information and operation end information of each operator, wherein the target management plug-in is deployed in a CPU; calling the target management plug-in to determine the operator hardware time of each operator based on the operation start information and operation end information of each operator.

[0007] In one embodiment, calling a target management plug-in to record the operation start information and operation end information of each operator includes: for each operator, when the operator starts running, generating the operator's operation start mark and obtaining the operator's original name, process number and device card number; when the operator ends running, generating the operator's operation end mark; for each operator, calling the target management plug-in to obtain the operation start information according to the operator's operation start mark, the operator's original name, process number and device card number; calling the target management plug-in to obtain the operation end information according to the operator's operation end mark, the operator's original name, process number and device card number.

[0008] In one embodiment, the calling target management plug-in determines the operator hardware time of each operator based on the operation start information and the operation end information of each operator, including: calling the target management plug-in to determine the calling information of each operator based on the operation start information and the operation end information of each operator; calculating the operator hardware time of the operator based on the calling information, wherein the calling information includes the operation start information and the operation end information with matching process number and device card number.

[0009] In one embodiment, the method further includes: saving the operator hardware information, the operation start information and the operation end information of each operator in a target data structure preset by the target management plug-in, wherein the operator hardware information includes the hardware address, the work queue and the operator calculation scale information.

[0010] In one embodiment, the method further includes: obtaining the operator name of each operator; for each operator, calling the target management plug-in to form a key-value pair corresponding to the operator according to the operator name of the operator and the data object corresponding to the operator in the target data structure, wherein the data object is a storage unit in the target data structure for storing the operator hardware information, the operator's operation start information and the operation end information; calling the target management plug-in to save the key-value pair corresponding to the operator in the database container preset by the target management plug-in.

[0011] In one embodiment, obtaining the operator name of each operator includes: for each operator, constructing the operator name of the operator according to the process ID, device card number and original name of the operator.

[0012] In one embodiment, the method further includes: for each operator, calling the target management plug-in to transmit the operator hardware information, the operation start information and the operation end information of each operator to the CPU, so that the CPU writes the operator hardware information, the operation start information and the operation end information of each operator into the event data structure of the CPU.

[0013] In one embodiment, the method further includes: for each operator, calling the target management plug-in to transmit the operator name of the operator to the CPU, so that the CPU updates the name of the operator in the event data structure.

[0014] In a second aspect, the present invention discloses a device for obtaining operator hardware time, which includes: a sending module for sending target data to a machine learning processor, which is configured with a pre-designed neural network model; an information recording module for calling a target management plug-in to record the operation start information and operation end information of each operator in the process of calling multiple operators in the neural network model to process the target data, wherein the target management plug-in is deployed in the CPU; a hardware time determination module for calling the target management plug-in to determine the operator hardware time of each operator according to the operation start information and operation end information of each operator.

[0015] In a third aspect, the present invention discloses a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for obtaining operator hardware time described in the first aspect are implemented.

[0016] In a fourth aspect, the present invention discloses a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for obtaining operator hardware time described in the first aspect.

[0017] The method, apparatus, computer device, and storage medium for obtaining operator hardware time provided by an embodiment of the present invention can support different types of heterogeneous devices in obtaining operator hardware time. The method includes: sending target data to a machine learning processor, which is configured with a pre-designed neural network model; in the process of calling multiple operators in the neural network model to process the target data, calling a target management plug-in to record the operation start information of each operator; calling the target management plug-in to record the operation end information of each operator; and determining the operator hardware time of each operator based on the operation start information and operation end information of each operator. In the scheme of the present invention, by calling the target management plug-in to record the operation start information and operation end information of each operator, and by calling the target management plug-in to determine the hardware time of each operator based on the operation start information and operation end information, the functional module for obtaining the hardware time of each operator is integrated into the target management plug-in, which reduces the modification to the Pytorch framework and the degree of coupling with the Pytorch framework. On the basis of increasing support for machine learning processors, it will not affect the use of the original CPU / CUDA in Pytorch. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0019] Figure 1 is a structural diagram illustrating a combined processing device according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram showing the structure of a heterogeneous device according to an embodiment of the present invention;

[0021] Figure 3 is a schematic flow chart illustrating a method for obtaining operator hardware time according to an embodiment of the present invention;

[0022] Figure 4 is a schematic diagram illustrating the logical relationship between a CPU, a machine learning processor, and a target management plug-in according to an embodiment of the present invention;

[0023] Figure 5 is a schematic diagram illustrating a method for calling a target management plug-in to record the operation start information of each operator according to an embodiment of the present invention;

[0024] Figure 6 is a schematic diagram illustrating a method for calling a target management plug-in to record the operation end information of each operator according to an embodiment of the present invention;

[0025] Figure 7 is a schematic diagram illustrating a method for determining operator hardware time according to an embodiment of the present invention;

[0026] Figure 8 is a flowchart illustrating another method for obtaining operator hardware time according to an embodiment of the present invention;

[0027] Figure 9 is a schematic diagram illustrating a method for centrally managing operator information according to an embodiment of the present invention;

[0028] Figure 10 is a schematic diagram illustrating an operator information storage structure according to an embodiment of the present invention;

[0029] Figure 11 is a structural block diagram illustrating an apparatus for obtaining operator hardware time according to an embodiment of the present invention;

[0030] Figure 12 The figure is a schematic diagram showing the structure of a board card according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0032] It should be understood that the terms "first," "second," and "third," etc. in the claims, description, and drawings of the present invention are used to distinguish different objects, rather than to describe a specific order. The terms "comprise" and "comprising" as used in the description and claims of the present invention indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0033] It should also be understood that the terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the present invention. As used in the specification and claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should further be understood that the term "and / or" as used in the specification and claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0034] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0035] In practical applications, determining operator hardware time is a prerequisite for analyzing operator hardware execution. Therefore, efficiently calculating operator hardware time across heterogeneous devices plays a crucial role in practical scientific research and engineering.

[0036] In the existing technology, in heterogeneous devices composed of CPU and GPU, Pytorch's built-in performance analysis tool profiler for deep learning neural networks can directly obtain the operator hardware time of each operator in the neural network model while the GPU is running the neural network model based on the GPU's corresponding CUDA (English: Compute Unified Device Architecture, Chinese: Compute Unified Device Architecture) code.

[0037] However, because GPUs are not specifically designed for neural networks, their computational efficiency is affected when using GPUs for training neural network models. To further improve the computational efficiency of neural network model training, existing technologies have proposed machine learning processors specifically for neural network learning, which achieve better computational acceleration using machine learning processors. However, because Pytorch's profiler tool currently only supports heterogeneous devices consisting of CPUs and GPUs, it is not possible to use the profiler tool to obtain the hardware time of operators of neural network models running on heterogeneous devices consisting of CPUs and other processors (such as machine learning processors).

[0038] Based on the above problems, the embodiments disclosed in the present invention provide a method, apparatus, computer equipment and storage medium for obtaining operator hardware time, which can support different types of heterogeneous devices to obtain operator hardware time. The method includes: sending target data to a machine learning processor, which is configured with a pre-designed neural network model; calling multiple operators in the neural network model to process the target data; calling a target management plug-in to record the start information of each operator's operation; calling a target management plug-in to record the end information of each operator's operation; and determining the operator hardware time of each operator based on the start information and end information of each operator. In the solution of the present invention, the hardware time of each operator is determined based on the start information and end information of the operation by calling the target management plug-in, and the functional module for obtaining the hardware time of each operator is integrated into the target management plug-in, which reduces the modification of the Pytorch framework and the degree of coupling with the Pytorch framework. On the basis of increasing support for machine learning processors, it will not affect the use of the original CPU / CUDA in Pytorch.

[0039] Furthermore, by calling the target management plug-in, the purpose of supporting different types of heterogeneous devices to obtain operator hardware time is achieved.

[0040] The following describes the technical solutions involved in the embodiments of the present invention in conjunction with the implementation environment of the embodiments of the present invention.

[0041] Figure 1 FIG. 1 is a structural diagram showing a combined processing device 100 according to an embodiment of the present invention. Figure 1 As shown in FIG, the combined processing device 100 includes a computing processing device 101, an interface device 102, other processing devices 103 and a storage device 104. According to different application scenarios, the computing processing device includes one or more computing devices 105, which can be configured to perform the processing described in conjunction with the attached Figures 3 to 9 The described operation.

[0042] In various embodiments, the computing and processing device 101 disclosed herein is configured to execute user-specified operations. In exemplary applications, the computing and processing device 101 is implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, one or more computing devices 105 included in the computing and processing device can be implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core. When multiple computing devices 105 are implemented as artificial intelligence processor cores or partial hardware structures of artificial intelligence processor cores, the computing and processing device disclosed herein can be considered to have a single-core structure or a homogeneous multi-core structure.

[0043] In an exemplary operation, the computing and processing device 101 disclosed in the present invention can interact with other processing devices 103 through the interface device 102 to jointly complete the operation specified by the user. Depending on the implementation, the other processing devices 103 disclosed in the present invention may include one or more types of processors in general and / or special processors such as a central processing unit (CPU), a graphics processing unit (GPU), an artificial intelligence processor, etc. When the other processing devices 103 include multiple types of processors, the processing device 100 constitutes a heterogeneous device, such as a heterogeneous device composed of a CPU and a GPU, or a heterogeneous device composed of a CPU and an artificial intelligence processor, or a heterogeneous device composed of a GPU and an artificial intelligence processor, etc. These processors may include, but are not limited to, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing and processing device disclosed in the present invention, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, the two can be regarded as forming a heterogeneous multi-core structure.

[0044] In one or more embodiments, the other processing device may serve as an interface between the computing device disclosed herein (which may be embodied as an artificial intelligence computing device, such as a neural network computing device) and external data and control, performing basic control including but not limited to data transfer and starting and / or stopping the computing device. In other embodiments, the other processing device may also collaborate with the computing device to jointly complete computing tasks.

[0045] like Figure 2 The figure shows a group of heterogeneous devices, with one CPU connected to multiple machine learning processors, a neural network model deployed on the machine learning processors, and a target management plug-in deployed on the CPU.

[0046] In one embodiment, Figure 3 As shown, a method for obtaining the operator hardware time is provided. This method is applied to Figure 1 In the computer processing device in, the method comprises the following steps:

[0047] Step 301: Send target data to a machine learning processor.

[0048] In an embodiment of the present invention, the target data is training data or test data, and the target data includes one or more of image data, speech data, and natural language data.

[0049] The CPU receives the target data from the external input and sends the target data to the machine learning processor, wherein the machine learning processor is configured with a pre-designed neural network model. The pre-designed neural network model is used to process the target data. Optionally, the machine learning processor in the embodiment of the present invention is Figure 1 Any processor in the processing device 103, this disclosure does not impose any limitation on this.

[0050] Optionally, after the neural network model is designed on the CPU side, the neural network model is copied from the CPU to the machine learning processor based on instructions, and then the neural network model is deployed on the machine learning processor.

[0051] Step 302: In the process of calling multiple operators in the neural network model to process the target data, the target management plug-in is called to record the operation start information and operation end information of each operator.

[0052] The run start information is recorded when each operator starts executing, and the run end information is recorded when each operator ends executing. The target management plug-in is deployed in the CPU to manage the machine learning processor.

[0053] The following describes the process of invoking multiple operators in a neural network model to process target data. In this embodiment of the present invention, a neural network model is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks and performs distributed parallel information processing. The neural network model includes multiple operators. An operator is a mapping O:X→X from one function space to another. Operators include expression methods and corresponding formulas.

[0054] The CPU calls each operator in sequence according to the pre-set calling order between the operators, with the target data as the input of the first-level operator and the output of the previous-level operator as the input of the next-level operator.

[0055] Optionally, the CPU is used to obtain a computational graph of the neural network model, and call each operator in sequence based on a pre-set calling order between operators in the computational graph.

[0056] The following describes the process of calling the target management plug-in to record the operation start information and operation end information of each operator: First, the logical relationship between the CPU, machine learning processor and target management plug-in in the embodiment of the present invention is described, such as Figure 4 As shown, the CPU is configured with a target management plug-in configuration and the Pytorch framework's built-in performance analysis tool for deep learning neural networks, the profiler (hereinafter referred to as the profiler tool). The machine learning processor is physically connected to the CPU, and the target management plug-in in the CPU is signal-connected to the machine learning processor. The target management plug-in can obtain the run start and run end information of each operator during the machine learning processor training process, and pass this run start and run end information to the profiler in the CPU. The profiler tool then stores the run start and run end information of each operator in the CPU's event data structure.

[0057] Operator states include the start registration state and the end registration state. The CPU can call an operator at any given moment. The operator state changes when the operator call starts and ends. The CPU can call the target management plug-in to record changes in the operator state. For example, when the operator call starts, the operator enters the operator execution queue, and the operator state is recorded as the start registration state. When the operator call ends, the operator exits the operator execution queue, and the operator state is recorded as the end registration state.

[0058] Optionally, in an embodiment of the present invention, in a multi-core CPU, the CPU may call multiple operators at each moment. In this case, the CPU may call a target management plug-in to record changes in the operator state of each operator.

[0059] In an embodiment of the present invention, when the operator state is the start registration state, the CPU can call the target management plug-in to record the operator's operation start information. When the operator state is the end registration state, the CPU can call the target management plug-in to record the operator's operation end information.

[0060] Optionally, in actual applications, when the convolutional layer, fully connected layer, and pooling layer in the neural network model extract features from the target data, they will perform multiplication, addition, nonlinear calculations, comparisons, screening, and other mathematical operations on the target data, and these mathematical operations need to be performed multiple times, so it is possible that an operator is called multiple times. In this case, each time the operator is called, the CPU can call the target management plug-in to record the start information of the operator's operation. Correspondingly, each time the call ends, the CPU can call the target management plug-in to record the end information of the operator's operation.

[0061] Step 303: Call the target management plug-in to determine the operator hardware time of each operator according to the operation start information and operation end information of each operator.

[0062] The operator hardware time of each operator refers to the runtime consumed when each operator runs on the machine learning processor.

[0063] In this embodiment of the present invention, the run start information is recorded when an operator begins execution and can be used to indicate the start time of the operator call. The run end information is recorded when the operator ends execution and can be used to indicate the end time of the operator call. Therefore, the operator hardware time can be determined based on the operator's run start information and run end information.

[0064] Optionally, the CPU can calculate the operator hardware time of each operator by calling the calculation hardware time interface in the target management plug-in. The calculation hardware time interface is implemented in C++.

[0065] In an embodiment of the present invention, for each operator, if an operator is called multiple times, multiple operation start information and multiple operation end information of the operator can be recorded, and the CPU can determine the operator hardware time of the operator in each call process by calling the target management plug-in.

[0066] In one optional implementation, after obtaining the operator hardware time for each operator, the CPU can also determine the number of operator calls, average runtime, and total runtime based on the operator hardware time. Obtaining the number of operator calls, average runtime, and total runtime provides a more accurate and detailed understanding of the actual execution status of the operator, facilitating more targeted operator optimization.

[0067] In an embodiment of the present invention, the target management plug-in is called to record the operation start information and the operation end information of each operator, and the hardware time of each operator is determined according to the operation start information and the operation end information by calling the target management plug-in. The functional module for obtaining the hardware time of each operator is integrated into the target management plug-in, which reduces the modification of the Pytorch framework and the degree of coupling with the Pytorch framework. On the basis of increasing support for machine learning processors, it will not affect the use of the original CPU / CUDA in Pytorch.

[0068] Furthermore, the target management plug-in can support different types of heterogeneous devices to obtain operator hardware time, so that the profiler tool carried by Pytorch is no longer limited to heterogeneous devices composed of the original CPU and GPU, expanding the application scope of the Pytorch framework.

[0069] In one embodiment of the present invention, Figure 5 As shown, Figure 5 A schematic diagram shows the technical process of calling the target management plug-in to record the operation start information of each operator. The technical process includes:

[0070] Step 501: For each operator, when the operator starts running, the target management plug-in is called to generate the running start mark of the operator, and the original operator name, process number and device card number of the operator are obtained.

[0071] In the embodiment of the present invention, when the operator starts to be called, the operator enters the operator execution queue. At this time, the CPU calls the target management plug-in to generate a running start mark and obtains the original operator name, process number and device card number of the operator.

[0072] The original operator name is stored in the operator record structure of the neural network model. The CPU can call the target management plug-in to obtain the original operator name from the operator record structure. A process is the basic unit of time resources allocated to an operator when it is called. The CPU can call the target management plug-in to obtain the process ID of the operator. The device card number is the device number of the machine learning processor running the operator.

[0073] Step 502: For each operator, the target management plug-in is called to obtain the operation start information according to the operation start mark of the operator, the original operator name, the process number and the device card number of the operator.

[0074] The operator's running start information can be used to determine the operator's starting running time. Optionally, the operator's running start information can be expressed as: <operator original name, process number, device card number, running start mark>.

[0075] Optionally, in order to simplify the expression of the operator's running start information, the operator's running start information can also be expressed as: <operator original name, running start tag>, where the operator original name can be linked to the process number and the device card number.

[0076] In practical applications, considering that a large number of identical operators are called in deep learning neural networks, there are many cases of having the same name. In order to effectively distinguish the records of different operators, in the embodiment of the present invention, for each operator, the CPU calls the target management plug-in to generate a new operator name new_name according to the original operator name, process number and device card number, and then generates the operator's running start information according to the new operator name new_name and the running start mark. For example, if the running start mark is push, then the operator's running start information can be expressed as<new_name,push> .

[0077] Optionally, in an embodiment of the present application, the execution start mark may also be expressed as a starting mark or other mark that can be used to indicate the starting state of the operator execution.

[0078] In this embodiment of the present invention, the target management plug-in is called to record the operator's run start mark and obtain the operator's original name, process ID, and device card number. The operator's run start information is then obtained based on the run start mark, original name, process ID, and device card number. This facilitates subsequent calculation of hardware time based on the run start information, improving hardware time accuracy.

[0079] In another embodiment of the present invention, Figure 6 As shown, Figure 6 A schematic diagram shows the technical process of calling the target management plug-in to record the operation end information of each operator. The technical process includes:

[0080] Step 601: For each operator, when the operator finishes running, the target management plug-in is called to generate the operator's running end mark, and the operator's original name, process number and device card number are obtained.

[0081] In the embodiment of the present invention, when the call to the operator is finished, the operator exits from the operator execution queue. At this time, the CPU can call the target management plug-in to generate an operation end mark of the operator.

[0082] At the same time, the target management plug-in can also be called to obtain the original operator name, process number, and device card number of the operator when the operator starts running.

[0083] Step 602: For each operator, the target management plug-in is called to obtain the operation end information according to the operation end mark of the operator, the original operator name, the process number and the device card number of the operator.

[0084] The operator's running end information can be used to determine the operator's running end time point.

[0085] Optionally, the operator's run end information can be expressed as: <operator original name, process number, device card number, run end marker>.

[0086] Optionally, in order to simplify the expression of the operator's run end information, the operator's run end information can also be expressed as: <operator original name, run end tag>, where the operator original name can be linked to the process number and the device card number.

[0087] In practical applications, considering that a large number of identical operators will be called in deep learning neural networks, there are many cases of having the same name. In order to effectively distinguish the records of different operators, in the embodiment of the present invention, for each operator, the CPU can call the target management plug-in to construct the operator name new_name of the operator according to the process number, device card number and original name of the operator, and then generate the operation end information of the operator according to the new operator name new_name and the operation end mark. For example, if the operation end mark is pop, the operation end information of the operator can be expressed as<new_name,pop> .

[0088] Optionally, in an embodiment of the present application, the execution end mark can also be expressed as End tag or other marks that can be used to indicate the execution end status of the operator.

[0089] In this embodiment of the present invention, the target management plug-in is called to record the operator's run-end marker and obtain the operator's original name, process ID, and device card number. The operator's run-end information is then obtained based on the run-end marker, operator name, process ID, and device card number. This facilitates subsequent calculation of hardware time based on the run-end information, improving hardware time accuracy.

[0090] In another embodiment of the present invention, Figure 7 As shown, Figure 7 A schematic diagram illustrating a technical process for determining operator hardware time in an embodiment of the present invention is shown. The technical process includes:

[0091] Step 701: The target management plug-in is called to determine the calling information of each operator according to the running start information and running end information of each operator.

[0092] The call information includes the operation start information and the operation end information in which both the process number and the device card number match.

[0093] In the embodiment of the present invention, for each run start information and each run end information, the calling target management plug-in determines the run start information and the run end information including the same process ID and device card number as one call information.

[0094] Optionally, an operator can have multiple call information. In this embodiment of the present invention, if there are multiple run start and end information for the same process ID and device card number, that is, an operator appears multiple times for the same process and card number. In this case, the CPU can call the target management plug-in to obtain the call order of each operator from the operator record structure, and then determine the call information corresponding to each operator call based on the operator call order. An operator can obtain multiple call information, and the number of call information indicates the number of times the operator has been called.

[0095] Step 702: Calculate the operator hardware time of the operator according to the call information.

[0096] The call information includes the operation start information and the operation end information in which both the process number and the device card number match.

[0097] In an embodiment of the present invention, for each call information of each operator, the time difference between the run start marker in the run start information and the run end marker in the run end information in the call information can be calculated, and the time difference is the operator hardware time of the operator.

[0098] In an embodiment of the present invention, the CPU records the running start mark when starting to call the operator and records the running end mark when ending the calling of the operator by calling the target management plug-in, and then determines the relative running time of the operator based on the running start mark and the running end mark, and then determines the operator hardware time of the operator based on this, thereby improving the accuracy of the operator hardware time.

[0099] The following examples and Figure 8 The method for obtaining operator hardware time provided by an embodiment of the present invention is described. Figure 8 A flowchart showing how to use the profiler tool to obtain the hardware time of operators running on a machine learning processor.

[0100] Step 801, enter the profiler.

[0101] In an embodiment of the present invention, when the CPU calls multiple operators in the neural network model to process target data, the context management tool can be used in the Pytorch front end to declare entry into the profiler tool.

[0102] Step 802: Record operator information.

[0103] The operator information includes the operator hardware information, operation start information, and operation end information of each operator. The operator hardware information may include hardware address, work queue, computing scale, and other information.

[0104] In an embodiment of the present invention, the CPU obtains operator information of an operator based on a profiler tool. The process of obtaining operator information of an operator based on the profiler tool includes the following:

[0105] The CPU calls the target management plug-in to obtain the operator information of the operators running on the machine learning processor and writes this operator information into the Pytorch profiler.

[0106] PyTorch includes a native profiler tool, but it only supports GPUs. To enable the profiler to support other machine learning processors besides GPUs, the present invention reconstructs a target management plug-in. The latter reuses the basic functionality of the former and adds support for machine learning processors.

[0107] By building and calling the target management plug-in, only a minimal amount of necessary code is added to Pytorch to achieve the purpose of obtaining operator information of operators of neural network models running on machine learning processors. This reduces the modification of the Pytorch framework and does not affect the use of native CPU / CUDA in Pytorch while increasing support for machine learning processors.

[0108] Step 803: Determine whether the recording of operator information is complete. If not, execute step 802. If complete, perform centralized management of the operator information.

[0109] The following describes the process of the CPU calling the target management plug-in to centrally manage operator information:

[0110] In the process of processing target data using multiple operators in the neural network model, operator hardware information, operation start information and operation end information of each operator are saved in the target data structure preset by the target management plug-in.

[0111] The operator hardware information includes the hardware address, work queue, and operator computing scale information.

[0112] Optionally, operator hardware information may include, for example, the hardware memory size requested during operator operation, the operator's peak memory usage, etc. Operator computation scale information refers to the actual computation scale of the operator running on the machine learning processor, and work queue information refers to the work queue in which the operator resides. This information can effectively restore the actual operation status of the operator.

[0113] In this embodiment of the present invention, after invoking the context management tool to enter the profiler tool, the CPU can invoke multiple operators in the neural network model to process the target data. During the processing, the CPU can invoke the target management plug-in to record the start and end information of each operator's operation. Simultaneously, the CPU can also invoke the target management plug-in to obtain the operator hardware information of each operator and write the operator hardware information, operation start information, and operation end information of each operator into the data object of the target data structure.

[0114] The target data structure is a pre-configured data structure in the target management plug-in. The data objects in the target data structure are the smallest storage units for recording hardware time. Optionally, the target data structure can be, for example, a linked list data structure, a tree data structure, or a graph data structure. The data objects in the target data structure store operator hardware information, run start information, and run end information for each operator.

[0115] Optional, such as Figure 9 As shown, in the embodiment of the present application, the process of storing the operator hardware information, the operation start information and the operation end information of each operator in the target data structure preset by the target management plug-in includes:

[0116] Step 901: Obtain the operator name of each operator.

[0117] In an optional implementation, the process of the CPU obtaining the operator name of the operator may be: the CPU may call the target management plug-in to obtain the original name of the operator from the operator record structure, and determine the original name of the operator as the operator name of the operator.

[0118] Considering that a large number of identical operators are called in deep learning neural networks, there are many cases of having the same name. In order to effectively distinguish the records of different operators, an embodiment of the present invention proposes another method for obtaining the operator name of an operator.

[0119] The CPU may call the target management plug-in to generate a new operator name new_name according to the original operator name, the process number, and the device card number, and determine the new operator name as the operator name of the operator.

[0120] Optionally, the process of generating a new operator name can also be as follows Figure 10 As shown, Figure 10In the operator, the CPU can obtain the operator's process number, device card number, original name of the operator, and the push start mark or pop end mark of the operation. Among them, push / pop is used to mark the operator status of the operator. The CPU can generate a new operator name based on the operator's process number, device card number, original name of the operator, and push / pop. The new operator name can be expressed as<new_name,push / pop> The new operator name is written into the data object of the target data structure preset by the target management plug-in.

[0121] Step 902: For each operator, call the target management plug-in to compose a key-value pair corresponding to the operator according to the operator name and the data object of the operator in the target data structure.

[0122] The data object is a storage unit in the target data structure for storing operator hardware information, operator operation start information, and operator operation end information.

[0123] For ease of description, data is used below to represent the data object of the operator in the target data structure.

[0124] For each operator, the CPU can obtain the operator name of the operator and the data object data of the operator in the target data structure. Optionally, the operator name can be the original operator name or a new operator name. Based on this, the CPU can call the target management plug-in to form a key-value pair corresponding to the operator with the operator name and the data object data of the operator in the target data structure. The key-value pair can be expressed as <operator name, data object in the target data structure>. Figure 10 The key-value pair <new operator name, data object in target data structure> consisting of the new operator name and the data object in the target data structure is exemplarily shown in FIG.

[0125] Step 903: Call the target management plug-in to save the key-value pair corresponding to the operator in a database container preset by the target management plug-in.

[0126] In the embodiment of the present invention, the CPU may call the target management plug-in to copy the key-value pair to a database container preset by the target management plug-in for storage.

[0127] Optionally, the database container can be, for example, a docker container, a std::multimap container, etc., wherein std::multimap is a standard library container that can store multiple key-value pairs with the same name. It should be noted that the present disclosure does not limit the type of database container. Figure 10As shown, the CPU can copy the key-value pair <new operator name, data object in the target data structure> to the database container, which is represented as database container <new operator name, data object in the target data structure>.

[0128] Furthermore, in an optional implementation, for each operator, the target management plug-in is called to transmit the operator hardware information, the operation start information, and the operation end information of each operator to the CPU, so that the CPU writes the operator hardware information, the operation start information, and the operation end information of each operator into the CPU's event data structure. The CPU's event data structure is an event data structure, which is a data structure predefined by the Pytorch framework configured in the CPU for encapsulating operator information.

[0129] In another optional implementation, in order to keep the operator name corresponding to the operator information of each operator stored in the Event data structure consistent with the operator name corresponding to the operator information of each operator stored in the target data structure of the target management plug-in, in an embodiment of the present invention, for each operator, after constructing the operator name of the operator according to the process number, device card number and original name of the operator, the target management plug-in can be called to transmit the operator name of the operator to the CPU so that the CPU can update the name of the operator in the event data structure, that is, replace the original name of the operator included in the data object of the operator in the event data structure with the new operator name, thereby achieving the purpose of updating the information in the Event data structure in real time in a global sense.

[0130] Step 804: Calculate operator hardware time.

[0131] After the neural network finishes running and the operator information of all operators running on the machine learning processor is recorded, the CPU can calculate the operator hardware time of each operator by calling the calculation hardware time interface in the target management plug-in, and return the operator hardware time to the Pytorch front-end for post-processing.

[0132] Among them, in the process of calculating the operator hardware time, it is first necessary to find the corresponding key-value pair in the database container according to the new operator name, and then obtain the operator's running start information and running end information based on the operator information corresponding to the data in the key-value pair. Finally, call the interface for calculating the hardware time to calculate the operator hardware time of the operator and return it to the Pytorch front-end.

[0133] Step 805: manually release memory.

[0134] During the calculation of the operator hardware time for each operator, in order to reduce the number of key-value pairs in the container and speed up container search, the memory of the machine learning processor can be manually released after the operator hardware time for each operator is calculated to avoid memory leaks.

[0135] It should be noted that, in the embodiment of the present invention, only the memory occupied by the operator information stored in the data object of the target data structure preset by the target management plug-in is released, and the operator information stored in the data object of the event data structure is not deleted.

[0136] Step 806, determine whether the calculation is completed. If it is completed, perform data post-processing. If it is not completed, return to execute the content of step 804 and step 805.

[0137] In the embodiment of the present invention, the user can perform in-depth analysis and processing on the operator information of each operator based on the record, wherein the CPU can also call the target management plug-in to query the operator hardware information of one or more operators.

[0138] It should be understood that although Figure 3-10 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 3-10 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0139] In one embodiment, Figure 11 As shown, a device 1100 for obtaining operator hardware time is provided, including: a sending module 1101, an information recording module 1102 and a hardware time determination module 1103, wherein:

[0140] A sending module 1101 is used to send target data to a machine learning processor configured with a pre-designed neural network model;

[0141] An information recording module 1102 is used to call a target management plug-in to record the start and end information of each operator during the process of calling multiple operators in the neural network model to process target data, wherein the target management plug-in is deployed in the CPU;

[0142] The hardware time determination module 1103 is used to call the target management plug-in to determine the operator hardware time of each operator according to the operation start information and operation end information of each operator.

[0143] In one embodiment, the information recording module 1102 is specifically used to: for each operator, when the operator starts running, generate the operator's running start mark and obtain the operator's original name, process number and device card number; when the operator ends running, generate the operator's running end mark; for each operator, call the target management plug-in to obtain the running start information according to the operator's running start mark, the operator's original name, process number and device card number; call the target management plug-in to obtain the running end information according to the operator's running end mark, the operator's original name, process number and device card number.

[0144] In one embodiment, the information recording module 1102 is specifically used to: call the target management plug-in to determine the call information of each operator based on the operation start information and operation end information of each operator; calculate the operator hardware time of the operator based on the call information, wherein the call information includes the operation start information and operation end information that match the process number and the device card number.

[0145] In one embodiment, the hardware time determination module 1103 is specifically used to save the operator hardware information, operation start information and operation end information of each operator in the target data structure preset by the target management plug-in, wherein the operator hardware information includes hardware address, work queue and operator calculation scale information.

[0146] In one embodiment, the hardware time determination module 1103 is specifically used to: obtain the operator name of each operator; for each operator, call the target management plug-in to form a key-value pair corresponding to the operator according to the operator name of the operator and the data object corresponding to the operator in the target data structure, wherein the data object is a storage unit in the target data structure for storing the operator hardware information, the operator's operation start information and the operation end information; call the target management plug-in to save the key-value pair corresponding to the operator in the database container preset by the target management plug-in.

[0147] In one embodiment, the hardware time determination module 1103 is specifically configured to: for each operator, construct an operator name according to the process ID, device card number and original name of the operator.

[0148] In one embodiment, the hardware time determination module 1103 is specifically used to: for each operator, call the target management plug-in to transmit the operator hardware information, operation start information and operation end information of each operator to the CPU, so that the CPU writes the operator hardware information, operation start information and operation end information of each operator into the event data structure of the CPU.

[0149] In one embodiment, the hardware time determination module 1103 is specifically configured to: for each operator, call the target management plug-in to transmit the operator name of the operator to the CPU, so that the CPU updates the name of the operator in the event data structure.

[0150] For the specific definition of the device for obtaining the operator hardware time, please refer to the definition of the method for obtaining the operator hardware time above, which will not be repeated here. The various modules in the above-mentioned device for obtaining the operator hardware time can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer processing device in the form of hardware, or can be stored in the memory of the computer processing device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0151] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: sending target data to a machine learning processor, on which a pre-designed neural network model is configured; in the process of calling multiple operators in the neural network model to process the target data, calling a target management plug-in to record the operation start information and operation end information of each operator, wherein the target management plug-in is deployed in a CPU; calling the target management plug-in to determine the operator hardware time of each operator based on the operation start information and operation end information of each operator.

[0152] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: for each operator, when the operator starts running, the operator's running start mark is generated, and the operator's original name, process number and device card number are obtained; when the operator runs, the operator's running end mark is generated; for each operator, the target management plug-in is called to obtain running start information based on the operator's running start mark, the operator's original name, process number and device card number; and the target management plug-in is called to obtain running end information based on the operator's running end mark, the operator's original name, process number and device card number.

[0153] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: calling the target management plug-in to determine the call information of each operator based on the operation start information and operation end information of each operator; calculating the operator hardware time of the operator based on the call information, wherein the call information includes the operation start information and operation end information of which both the process number and the device card number match.

[0154] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: the operator hardware information, the operation start information and the operation end information of each operator are saved in the target data structure preset by the target management plug-in, wherein the operator hardware information includes the hardware address, the work queue and the operator calculation scale information.

[0155] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: obtaining the operator name of each operator; for each operator, calling the target management plug-in to form a key-value pair corresponding to the operator based on the operator name of the operator and the data object corresponding to the operator in the target data structure, wherein the data object is a storage unit in the target data structure for storing the operator hardware information, the operator's operation start information, and the operation end information; calling the target management plug-in to save the key-value pair corresponding to the operator in a database container preset by the target management plug-in.

[0156] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: for each operator, the operator name of the operator is constructed according to the process number, device card number and original name of the operator.

[0157] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: for each operator, the target management plug-in is called to transmit the operator hardware information, the operation start information and the operation end information of each operator to the CPU, so that the CPU writes the operator hardware information, the operation start information and the operation end information of each operator into the event data structure of the CPU.

[0158] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: for each operator, calling the target management plug-in to transmit the operator name of the operator to the CPU, so that the CPU updates the name of the operator in the event data structure.

[0159] The computer-readable storage medium provided in this embodiment has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0160] In one embodiment, a computer device is provided, including a processor and a memory for storing executable instructions, wherein the processor is configured to call the instructions stored in the memory to execute the method for obtaining operator hardware time provided in the above embodiment.

[0161] In some embodiments, the present invention further discloses a chip (eg Figure 12 In one implementation, the chip is a system on chip (SoC) and integrates one or more components such as Figure 1 The chip can be connected to the external interface device (such as Figure 12 The external interface device 1206 shown in the figure is connected to other related components. The related components can be, for example, a camera, a display, a mouse, a keyboard, a network card or a wifi interface. In some application scenarios, other processing units (such as video codecs) and / or interface modules (such as DRAM interfaces) can be integrated on the chip. In some embodiments, the present invention also discloses a chip packaging structure, which includes the above-mentioned chip. In some embodiments, the present invention also discloses a board card, which includes the above-mentioned chip packaging structure. The following will be combined with Figure 12 The board is described in detail.

[0162] Figure 12 FIG. 1 is a schematic diagram showing the structure of a board 1200 according to an embodiment of the present invention. Figure 12 As shown in , the board includes a storage device 1204 for storing data, which includes one or more storage units 1210. The storage device can be connected to the control device 1208 and the chip 1202 described above and transmit data by means of, for example, a bus. Furthermore, the board also includes an external interface device 1206, which is configured for data relay or transfer function between the chip (or the chip in the chip packaging structure) and the external device 1212 (such as a server or computer, etc.). For example, the data to be processed can be passed from the external device to the chip through the external interface device. For another example, the calculation result of the chip can be transmitted back to the external device via the external interface device. According to different application scenarios, the external interface device can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0163] In one or more embodiments, the control device in the board of the present invention can be configured to regulate the state of the chip. To this end, in one application scenario, the control device can include a microcontroller unit (MCU) for regulating the working state of the chip.

[0164] Combined with the above Figure 1 and Figure 12 Based on the description, those skilled in the art can understand that the present invention also discloses an electronic device or apparatus, which may include one or more of the above-mentioned boards, one or more of the above-mentioned chips and / or one or more of the above-mentioned combined processing devices.

[0165] Depending on the application scenario, the electronic device or device of the present invention may include a server, a cloud server, a server cluster, a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a PC device, an Internet of Things terminal, a mobile terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a visual terminal, an automatic driving terminal, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric light, a gas stove, and a range hood; the medical device includes an MRI, an ultrasound machine and / or an electrocardiograph. The electronic device or device of the present invention may also be applied to the Internet, the Internet of Things, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, medical care and other fields. Furthermore, the electronic device or device of the present invention may also be used in application scenarios related to artificial intelligence, big data and / or cloud computing, such as the cloud, edge, and terminal. In one or more embodiments, electronic devices or apparatuses with high computing power according to the solution of the present invention can be applied to cloud devices (such as cloud servers), while electronic devices or apparatuses with low power consumption can be applied to terminal devices and / or edge devices (such as smartphones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0166] It should be noted that, for the purpose of simplicity, the present invention describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will understand that the scheme of the present invention is not limited by the order of the described actions. Therefore, based on the disclosure or teachings of the present invention, those skilled in the art will understand that some of the steps therein can be performed in other orders or simultaneously. Further, those skilled in the art will understand that the embodiments described in the present invention can be regarded as optional embodiments, that is, the actions or modules involved therein are not necessarily necessary for the implementation of one or more schemes of the present invention. In addition, depending on the different schemes, the present invention also has different emphases on the description of some embodiments. In view of this, those skilled in the art will understand that the parts that are not described in detail in a certain embodiment of the present invention may also refer to the relevant descriptions of other embodiments.

[0167] In terms of specific implementation, based on the disclosure and teachings of the present invention, those skilled in the art can understand that several embodiments disclosed in the present invention can also be implemented in other ways not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article divides them based on the consideration of logical functions, and there may be other ways of division in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in the units or components can be selectively disabled. With respect to the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0168] In the present invention, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network elements. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the solutions described in the embodiments of the present invention. In addition, in some scenarios, multiple units in the embodiments of the present invention may be integrated into a single unit or each unit may exist physically separately.

[0169] In some implementation scenarios, the above-mentioned integrated unit can be implemented in the form of a software program module. If implemented in the form of a software program module and sold or used as an independent product, the integrated unit can be stored in a computer-readable memory. Based on this, when the solution of the present invention is embodied in the form of a software product (such as a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to enable a computer processing device (such as a personal computer, a server or a network device, etc.) to perform some or all of the steps of the method described in the embodiment of the present invention. The aforementioned memory may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0170] In some other implementation scenarios, the above-mentioned integrated unit can also be implemented in the form of hardware, that is, a specific hardware circuit, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage unit or storage device can be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which can be, for example, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM and RAM, etc.

[0171] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The foregoing content can be better understood in accordance with the following terms:

[0174] Clause A1. A method for obtaining operator hardware time, the method comprising:

[0175] The target data is sent to a machine learning processor, which is configured with a pre-designed neural network model; in the process of calling multiple operators in the neural network model to process the target data, a target management plug-in is called to record the operation start information and the operation end information of each operator, wherein the target management plug-in is deployed in the CPU; the target management plug-in is called to determine the operator hardware time of each operator according to the operation start information and the operation end information of each operator.

[0176] Clause A2, according to the method described in Clause A1, the calling target management plug-in records the operation start information and operation end information of each operator, including:

[0177] For each of the operators, when the operator starts running, a running start mark of the operator is generated, and the original name, process number and device card number of the operator are obtained; when the operator ends running, a running end mark of the operator is generated; for each of the operators, the target management plug-in is called to obtain the running start information according to the running start mark of the operator, the original name, process number and device card number of the operator; and the target management plug-in is called to obtain the running end information according to the running end mark of the operator, the original name, process number and device card number of the operator.

[0178] Clause A3, according to the method of clause A2, wherein the calling of the target management plug-in to determine the operator hardware time of each operator according to the operation start information and the operation end information of each operator includes:

[0179] The target management plug-in is called to determine the calling information of each operator according to the running start information and the running end information of each operator; the operator hardware time of the operator is calculated according to the calling information, wherein the calling information includes the running start information and the running end information of which the process number and the device card number both match.

[0180] Clause A4. The method according to clause A1, further comprising:

[0181] The operator hardware information of each operator, the operation start information and the operation end information of each operator are saved in a target data structure preset by the target management plug-in, wherein the operator hardware information includes hardware address, work queue and operator calculation scale information.

[0182] Clause A5. The method according to clause A4, further comprising:

[0183] Obtain the operator name of each of the operators; for each of the operators, call the target management plug-in to form a key-value pair corresponding to the operator according to the operator name of the operator and the data object corresponding to the operator in the target data structure, wherein the data object is a storage unit in the target data structure for storing the operator hardware information, the operator's operation start information and the operation end information; call the target management plug-in to save the key-value pair corresponding to the operator in the database container preset by the target management plug-in.

[0184] Clause A6, according to the method described in Clause A5, obtaining the operator name of each operator includes: for each operator, constructing the operator name of the operator based on the process number, device card number and original name of the operator.

[0185] Clause A7. The method according to clause A6, further comprising:

[0186] For each of the operators, the target management plug-in is called to transmit the operator hardware information, the operation start information and the operation end information of each operator to the CPU, so that the CPU writes the operator hardware information, the operation start information and the operation end information of each operator into the event data structure of the CPU.

[0187] Clause A8. The method according to clause A7, further comprising:

[0188] For each operator, the target management plug-in is called to transmit the operator name of the operator to the CPU, so that the CPU updates the name of the operator in the event data structure.

[0189] The above-described embodiments merely represent several implementation methods of the embodiments of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art can make several modifications and improvements without departing from the concept of the embodiments of the present invention, all of which fall within the scope of protection of the embodiments of the present invention. Therefore, the scope of protection of the patents of the embodiments of the present invention shall be based on the appended claims.

Claims

1. A method for obtaining operator hardware time, characterized in that: The method comprises: Sending the target data to a machine learning processor, wherein the machine learning processor is configured with a pre-designed neural network model; In the process of calling multiple operators in the neural network model to process the target data, calling the target management plug-in to obtain the running start information of each operator according to the running start mark of each operator, the original operator name of each operator, the process number and the device card number of each operator, and calling the target management plug-in to obtain the running end information of each operator according to the running end mark of each operator, the original operator name of each operator, the process number and the device card number of each operator, wherein the target management plug-in is deployed in the CPU; The target management plug-in is called to determine the operator hardware time of each operator according to the operation start information and the operation end information of each operator.

2. The method according to claim 1, characterized in that The method further comprises: For each operator, when the operator starts running, a running start mark of the operator is generated, and the original operator name, process number and device card number of the operator are obtained; when the operator runs, an running end mark of the operator is generated.

3. The method according to claim 2, characterized in that The calling of the target management plug-in to determine the operator hardware time of each operator according to the operation start information and the operation end information of each operator includes: Calling the target management plug-in to determine the calling information of each operator according to the running start information and the running end information of each operator; The operator hardware time of the operator is calculated according to the call information, wherein the call information includes operation start information and operation end information in which both the process number and the device card number match.

4. The method according to claim 1, wherein The method further comprises: The operator hardware information of each operator, the operation start information and the operation end information of each operator are saved in a target data structure preset by the target management plug-in, wherein the operator hardware information includes hardware address, work queue and operator calculation scale information.

5. The method according to claim 4, characterized in that The method further comprises: Obtain the operator name of each operator; For each of the operators, calling the target management plug-in to compose a key-value pair corresponding to the operator according to the operator name of the operator and the data object corresponding to the operator in the target data structure, wherein the data object is a storage unit in the target data structure for storing the operator hardware information, the operator's operation start information, and the operator's operation end information; The target management plug-in is called to save the key-value pair corresponding to the operator in a database container preset by the target management plug-in.

6. The method according to claim 5, characterized in that The obtaining of the operator name of each operator includes: For each operator, the operator name of the operator is constructed according to the process number, device card number and original name of the operator.

7. The method according to claim 6, characterized in that The method further comprises: For each of the operators, the target management plug-in is called to transmit the operator hardware information, the operation start information and the operation end information of each operator to the CPU, so that the CPU writes the operator hardware information, the operation start information and the operation end information of each operator into the event data structure of the CPU.

8. The method according to claim 7, characterized in that The method further comprises: For each operator, the target management plug-in is called to transmit the operator name of the operator to the CPU, so that the CPU updates the name of the operator in the event data structure.

9. A device for obtaining operator hardware time, characterized in that: The device comprises: a sending module, configured to send target data to a machine learning processor, wherein the machine learning processor is configured with a pre-designed neural network model; an information recording module, configured to, during the process of calling multiple operators in the neural network model to process the target data, call a target management plug-in to obtain the operation start information of each operator based on the operation start mark, the original operator name, the process number, and the device card number of each operator, and call the target management plug-in to obtain the operation end information of each operator based on the operation end mark, the original operator name, the process number, and the device card number of each operator, wherein the target management plug-in is deployed in the CPU; The hardware time determination module is used to call the target management plug-in to determine the operator hardware time of each operator according to the operation start information and the operation end information of each operator.

10. A computer device, characterized in that: include: A processor and a memory for storing executable instructions, wherein the processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 8.

11. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images