Method for obtaining computing end operator data and precision verification and device thereof

Abstract

The present invention relates to a method and apparatus for obtaining compute kernel data and accuracy verification, wherein the compute device of the present invention is included in an integrated circuit device that includes a general purpose interconnect interface and other processing devices. The compute device interacts with the other processing devices to collectively perform a user-specified compute operation. The integrated circuit device can also include a storage device connected to the compute device and the other processing devices, respectively, for data storage of the compute device and the other processing devices.

Description

TECHNICAL FIELD

[0001] The present application generally relates to the field of neural networks. More specifically, the present application relates to a method for obtaining computation-end operator data in a deep learning model, a precision verification method, and a computer readable storage medium, a computer program product, a computer device, a computing device, an integrated circuit device, a board card and a system thereof. BACKGROUND

[0002] Deep learning is a research direction in the field of machine learning, and its ultimate goal is to enable machines to have analysis and learning capabilities like humans, and to recognize data such as text, images and sound. At present, quite ideal results have been achieved in computer vision, speech recognition, natural language processing and other applications.

[0003] In order to make deep learning easier to be applied, developers have created many frameworks to implement specific functions on existing platforms, thereby reducing the threshold of deep learning development. With the passage of time, the most popular deep learning frameworks at present include PaddlePaddle, Tensorflow, Caffe, Theano, MXNet, Torch and PyTorch, etc.

[0004] Each deep learning framework can support an acceleration library. Developers can build a specific function acceleration library, configure the acceleration library for running the deep learning model through the related interface of the deep learning framework, that is, add distribution logic in the deep learning framework to realize the function of distributing tasks (such as model training) to the acceleration library. The acceleration library will generate operators inside to execute the distributed tasks, and the efficiency of running the acceleration library is higher. The acceleration library developer should not modify the deep learning model.

[0005] However, the operators generated by the acceleration library are invisible to the developer. Due to the large network, complex structure and multi-layer nesting of the deep learning model, once the operators are invisible, the developer will have difficulty in quickly troubleshooting the network when locating network problems or analyzing data rules.

[0006] Furthermore, during the network training process of deep learning, there will be certain errors in the calculation accuracy when using different computing devices. These errors may come from the characteristics of parallel computing, the difference in bit width of the device, the difference in algorithm implementation, etc. When the error is not obvious, it will not affect the performance of the network, but when the error is large, it may directly cause failure.

[0007] Therefore, a scheme for obtaining data layer by layer (operator by operator) for the acceleration library, which facilitates the developer to quickly locate problems and analyze data, and reduces the difficulty of debugging the network, is urgently needed. SUMMARY

[0008] To at least partially solve the technical problems mentioned in the background, the present application provides a method for obtaining computation-end operator data in a deep learning model, a precision verification method, a computer readable storage medium, a computer program product, a computer device, a computing device, an integrated circuit device, a board card and a system.

[0009] In one aspect, the present application discloses a method for obtaining computation-end operator data in a deep learning model, comprising: adding an input interface and an output interface of an operator; receiving input data from the input interface based on a state parameter; receiving output data from the output interface based on the state parameter to perform data analysis on computation-end operator data, wherein the computation-end operator data comprises the input data and the output data.

[0010] In another aspect, the present application discloses a precision verification method, comprising: performing a task by using a processing device to obtain processing-end operator data, the processing-end operator data being reference data; performing the method as described above by using a computing device to obtain computation-end operator data; and comparing the computation-end operator data with the reference data to obtain a precision verification result.

[0011] In another aspect, the present application discloses a computer readable storage medium having stored thereon computer program codes for obtaining computation-end operator data in a deep learning model or precision verification, when the computer program codes are run by a processing device, the method as described above is performed.

[0012] In another aspect, the present application discloses a computer program product, comprising a computer program for obtaining computation-end operator data in a deep learning model or precision verification, characterized in that the computer program is executed by a processor to implement the steps of the method as described above.

[0013] In another aspect, the present application discloses a computer device, comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the steps of the method as described above.

[0014] In another aspect, the present application discloses a computing device for obtaining computation-end operator data in a deep learning model, wherein the operator adds an input interface and an output interface. The computing device comprises: a state management module and a data processing module. The state management module is used to maintain a state parameter; the data processing module is used to: receive input data from the input interface based on the state parameter; receive output data from the output interface based on the state parameter to perform data analysis on computation-end operator data. Wherein the computation-end operator data comprises the input data and the output data.

[0015] In another aspect, the present application discloses an integrated circuit device comprising the computing device according to the foregoing. The present application also discloses a board card comprising the integrated circuit device according to the foregoing.

[0016] In another aspect, the present application discloses a precision verification system comprising the computing device according to the foregoing, for performing the task to obtain the computing-end operator data; and a processing device for performing the task to obtain processing-end operator data, the processing-end operator data being reference data, comparing the computing-end operator data with the reference data to obtain a precision verification result.

[0017] The present application adds the input-end interface and the output-end interface of the operator, globally manages the state parameters in the state management module, and then obtains the computing-end operator data, so as to achieve the technical effects of obtaining the data in the network operation to locate the fault or analyze the rules, and obtaining the mapping relationship between the acceleration library operator and the upper layer of the framework. Further, after the computing-end operator data is generated, the present application can also verify the precision by comparing the computing-end operator data with the processing-end operator data. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and other objects, features and advantages of the present application will become readily apparent from the detailed description that follows, read in conjunction with the accompanying drawings. In the drawings, several embodiments of the present application are shown by way of example and not limitation. Like or corresponding reference numerals primarily designate the same or corresponding parts throughout the several views. Among others:

[0019] Figure 1 is a structural diagram of a board card of an embodiment of the present application;

[0020] Figure 2 is a structural diagram of an integrated circuit device of an embodiment of the present application;

[0021] Figure 3 is a schematic diagram showing the second distribution of tasks by an embodiment of the present application when running an acceleration library;

[0022] Figure 4 is a flowchart showing the operation of a distribution control macro of a distribution control tool of an embodiment of the present application;

[0023] Figure 5 is a schematic diagram showing the acquisition of computing-end operator data in a deep learning model by a combined processing device of an embodiment of the present application;

[0024] Figure 6 is a flowchart showing the acquisition of computing-end operator data by another embodiment of the present application; and

[0025] Figure 7 is a flowchart showing the precision verification of another embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application.

[0027] It should be understood that the terms "first", "second", "third", and "fourth" and the like in the claims, specification, and drawings of the present application are used to distinguish different objects, and are not used to describe a particular order. The terms "include" and "contain" used in the specification and claims of the present application indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or sets thereof.

[0028] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the specification and claims of the present application, the singular forms "a", "an", and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should be further understood that the term "and / or" used in the specification and claims of the present application means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0029] As used in the specification and claims of the present application, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context.

[0030] The specific embodiments of the present application will be described in detail below with reference to the drawings.

[0031] In a deep learning framework, a task (which can be a network layer in the deep learning framework, also known as an operator, but is not necessarily the same concept as the operator in the acceleration library) will be distributed to the acceleration library, which will further distribute the task, i.e., split the task into multiple operators for execution. The present application adds an in-end interface and an out-end interface to these operators to obtain computing-end operator data for developers to locate faults or analyze rules and further verify accuracy.

[0032] Figure 1 A structural schematic diagram of a board card 10 according to an embodiment of the present application is shown. As shown in FIG. 1, the board card 10 includes an in-end interface 11, an out-end interface 12, and a plurality of operators 13. Figure 1As shown, the board card 10 includes a chip 101, which is a system on chip (SoC) integrated with one or more combined processing devices, which is an artificial intelligence operation unit to support various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in cloud intelligent fields. A significant feature of cloud intelligent applications is the large amount of input data, which has high requirements for the storage capacity and computing capacity of the platform. The board card 10 of this embodiment is suitable for cloud intelligent applications and has a large off-chip storage, on-chip storage and strong computing capacity.

[0033] The chip 101 is connected with an external device 103 through an external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card or a wifi interface, etc. The data to be processed can be transmitted from the external device 103 to the chip 101 through the external interface device 102. The computing result of the chip 101 can be transmitted back to the external device 103 through the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe interface, etc.

[0034] The board card 10 further includes a storage device 104 for storing data, which includes one or more storage units 105. The storage device 104 is connected and transmits data with the control device 106 and the chip 101 through a bus. The control device 106 in the board card 10 is configured to regulate the state of the chip 101. For this purpose, in one application scenario, the control device 106 can include a micro controller unit (MCU).

[0035] Figure 2 is a structural diagram of the combined processing device in the chip 101 of this embodiment. As shown in Figure 2 The combined processing device 20 includes a computing device 201, an interface device 202, a processing device 203 and an off-chip memory 204.

[0036] The computing device 201 is configured to perform user-specified operations, mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations, which can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0037] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write the data into the storage device on the computing device 201. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write the control instructions into the control buffer on the computing device 201. Alternatively or additionally, the interface device 202 can also read data from the storage device of the computing device 201 and transmit the data to the processing device 203.

[0038] The processing device 203 is a general-purpose processing device that performs basic controls including but not limited to data transfer, starting and / or stopping of the computing device 201, etc. Depending on the implementation, the processing device 203 can be one or more types of processors including but not limited to a central processing unit (CPU), a graphics processing unit (GPU), or other general-purpose and / or special-purpose processors such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., and the number thereof can be determined according to actual needs. As mentioned above, only in terms of the computing device 201 of the present application, it can be considered as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0039] The off-chip memory 204 is used to store data to be processed and is a DDR memory, usually having a size of 16G or more, for storing data of the computing device 201 and / or the processing device 203.

[0040] In order to achieve more efficient operation, the processing device 203 will first distribute the tasks of the deep learning framework to the acceleration library. The acceleration library not only supports two different modes of inference and training, but also supports computing using intelligent programming languages. In order to make good use of the resources of the acceleration library, the tasks need to be distributed for the second time in the acceleration library, so that the deep learning framework can run in multiple modes created by the acceleration library when operating in the computing device 201.

[0041] In one case, the deep learning framework can be a deep learning framework such as PyTorch, Tensorflow, Caffe, etc., the acceleration library can be a cudnn (a GPU-specific acceleration library of the NVIDIA Corporation), an XLA (an acceleration library developed by Google based on a TPU), a CATCH acceleration library (an acceleration library developed by the Middle Kingdom Cambrian Technology Co., Ltd.), or other acceleration libraries, etc., and the intelligent programming language can be a cuda (a programming language of the NVIDIA Corporation), an opencl (a general-purpose acceleration language), a BANGC language (a language developed by the Middle Kingdom Cambrian Technology Co., Ltd.), or other intelligent programming languages, which is a set of efficient software stack tools and high-performance libraries, facilitating developers to develop and deploy on its cloud, edge, and end platforms.

[0042] Figure 3 A schematic diagram showing that the processing device 203 performs second distribution of tasks when running the acceleration library. In the acceleration library, the distribution of operators is implemented by using the following tools: an interface tool 301, an operator tool 302, a code generation tool 303, and a distribution control tool 304.

[0043] First, the acceleration library registers a pointer or proxy mode (proxy) with the deep learning framework, i.e., the interface tool 301, so that the interface tool 301 can receive the first distributed tasks from the deep learning framework, which generally include the calculation type and the tensor to be processed.

[0044] The operator tool 302 provides an operator base class, which is a virtual class and is the default operator executed by the processing device 203, i.e., in the case where the hardware of the computing device 201 does not support or fails to compute, the processing device 203 will execute the computation instead. The operator base class is derived into three backend acceleration libraries: a training operator class, an inference operator class, and an intelligent programming language operator class, wherein the training operator class includes a set of operator methods dedicated to improving the performance of the network model training phase, the inference operator class includes a set of operator methods dedicated to improving the performance of the network model inference phase, and the intelligent programming language operator class includes a set of custom operator methods.

[0045] During the running of the network, after the deep learning framework first distributes the tasks to the interface tool 301, the interface tool 301 will instantiate one of the above three subclasses or base classes as the actual operator participating in the calculation according to the runtime mode, such as the inference mode or the training mode. The interface tool 301 finally returns a class pointer, which uses the virtual class feature to select one of the four classes to run. For example, if the task is in the training mode, the interface tool 301 returns a class pointer specifying that the operator in the training operator class is used to run.

[0046] The code generation tool 303 obtains the input, output and operator information of the task from the YAML file of the acceleration library or deep learning framework, where the YAML file is a resource manifest and resource orchestration file. The code generation tool 303 then generates the basic interfaces of the four types of operators (training operators, inference operators, intelligent programming language operators and callback operators), i.e., operator in interfaces and operator out interfaces, according to the operator information, and embeds the control function of the distribution control tool 304 between the operator in interfaces and the operator out interfaces. The control function of the distribution control tool 304 is divided into three parts: a distribution control entry, a distribution control macro and a distribution control exit, so the code interface generated by the code generation tool 303 is in the order of operator in interface→distribution control entry→distribution control macro→distribution control exit→operator out interface, and in this way, the four types of operators are realized.

[0047] The distribution control tool 304 controls the calculation and interface call of the operator through the second distribution of the distribution control macro based on the base class provided by the operator tool 302, and specifically generates a training operator, an inference operator, an intelligent programming language operator or a callback operator. The training operator is extracted from the operator method in the self-training operator class, the inference operator is extracted from the operator method in the inference operator class, the intelligent programming language operator is extracted from the operator method in the intelligent programming language operator class, and the callback operator is an operator executed by the processing device 203 when the first three types of operators cannot be implemented or calculation errors occur. Since the acceleration effect of the training operator class, the inference operator class and the intelligent programming language operator class is more obvious, the distribution control tool 304 preferentially selects to split the task into these three types of operators when performing the second distribution, and generates the callback operator to be executed by the processing device 203 when these three types of operators cannot be implemented. Through the above process, the distribution control macro realizes the operator call according to the task execution operator calculation.

[0048] Figure 4A flow chart of the distribution control macro of the distribution control tool 304 is shown. In step 401, it is determined whether the operator can be executed by the computing device 201. If yes, step 402 is executed to execute the operator of the computing device 201, i.e. to execute the training operator, the inference operator or the smart programming language operator. In step 403, it is determined whether the execution is successful. If yes, step 404 is executed to determine whether the debugging mode is executed. Generally, the computing device 201 and the processing device 203 are in asynchronous computation, but in the debugging mode, the two need to be debugged to be synchronous computation for debugging. If the debugging mode is executed, step 405 is executed to perform the operator synchronization and debugging operation of the computing device 201 and the processing device 203, and in the debugging operation, the relevant data is recorded for the developer to debug. If the debugging mode is not executed or after step 405, step 406 is executed to take the result of the operation of the computing device 201 as the return value. When it is determined in step 401 that the operator is not executed by the computing device 201, or when it is determined in step 403 that the execution is not successful, step 407 is executed to execute the operator of the processing device 203, i.e. to execute the callback operator. In step 408, the result of the operation of the processing device 203 is taken as the return value. Finally, in step 409, the calculation result of the operator is returned, which is from the result of the computing device 201 or from the result of the processing device 203. Thus, the calculation of the operator is completed.

[0049] As described above, the code generation tool 303 inserts the in-end interface and the out-end interface at the entrance and the exit of each operator respectively, so that the computing device 201 can obtain the relevant data of the operator through the interfaces for the developer to evaluate the efficiency of the operator. Figure 5 A schematic diagram showing that the combined processing device 20 obtains the operator data of the computing end (i.e. the computing device 201) in the deep learning model is shown.

[0050] The neural network 501 is an executable program that has implemented operators through the acceleration library and has been compiled. Before the compilation, the neural network 501 includes a plurality of operators, and each operator 502 has an in-end interface 503 and an out-end interface 504 generated by the code generation tool 303 according to the operator list information of the acceleration library, wherein the in-end interface 503 is responsible for transmitting the input data information before entering the operator 502, and the out-end interface 504 is responsible for transmitting the output data information after the operator 502 is calculated. The existence of these in-end interfaces 503 and out-end interfaces 504 enables the computing device 201 to obtain part of the computing end operator data. It should be noted that, Figure 5 The neural network 501 is illustrated by a serial network example, and in fact, various network shapes exist in the neural network model, such as a mesh shape. This embodiment does not limit the network shape of the neural network model.

[0051] The computing device 201 comprises a user interface module 505, a state management module 506, and a data processing module 507, to achieve the acquisition of the computing-end operator data when the acceleration library of the deep learning framework is executed.

[0052] The user interface module 505 is used to generate a user interface, which can be a context manager interface, a function interface, or an environment variable interface, such as a Python context manager interface, a Python function interface, a C++ function interface, or a BANGC variable interface, etc. Developers can select or set any one or several interfaces according to actual use scenarios to match the overall running requirements. Through the user interface of the user interface module 505, developers can send instructions to determine various configurations, which carry information of the configurations, including the timing of acquiring the computing-end operator data and / or the processing method of the computing-end operator data, etc.

[0053] The state management module 506 manages the global and maintains the state parameters 508 through a state machine. The state parameters 508 are a series of information of the state machine, such as information from the preparation stage of reading data, the processing stage of collecting and processing data, to the saving stage, in other words, the state parameters 508 record the real-time running state of the state machine. The state management module 506 receives instructions from the user interface, which carries various configuration information, such as information of the data to be obtained, the processing method, and the timing of obtaining, etc. Among them, the enable flag (the timing of data acquisition) obtained by the user interface is mapped with the state parameters 508 in the state management module 506, to determine the value of the state parameters.

[0054] Based on the state parameters 508 and the information of the instructions, the state management module 506 controls the import of data of the in-end interface and the out-end interface. Further, the state management module is also used to receive the operator name and structure information from the in-end interface, and the data processing module also receives the input data and the output data based on the structure information. Specifically, the state management module receives the operator name and structure information from the in-end interface, which are part of the computing-end operator data, and these data are used to structure the operator information for the back-end. After obtaining the structure information, the state management module 506 determines whether the data processing module imports the input data or the output data from the in-end interface or the out-end interface based on the structure information, the state parameters, and other information (parameters accumulated and obtained during the running process, etc.). The following will make a detailed description of the whole process of acquiring the computing-end operator data.

[0055] In this embodiment, the control of the global process is realized by using a start function and an end function, where the start function represents the beginning of the global process, and the end function represents the end of the global process. When the state management module 506 runs to the start function, the process of collecting the computing-end operator data of a series of operators is started.

[0056] First, the state management module 506 is initialized according to the instructions to implement the configuration required by the developer, such as creating a file directory, performing preprocessing, etc. At the time of initialization, the state parameters are also reset, so that the information carried by the state parameters, such as the current number of operators, the number of inputs (or outputs) of the operator being processed, the current file directory path, how many inputs and outputs are temporarily stored, etc. Return to the initial value.

[0057] The state management module 506 then controls the input interface and collects information. Based on the state parameters 508, the state management module 506 controls the input interface to transmit the input data from the input interface to the interface of the operator execution, and obtains the operator name and structure information from the input interface, wherein the structure information includes the number of data of the input and output of the operator, the size, type, dimension, etc. of each data (tensor or numerical value).

[0058] The state management module 506 further controls the data processing module 507 to receive the input data from the input interface. Specifically, based on the state parameters 508, the structure information, and the instructions from the user interface module 505, the state management module 506 controls the data processing module 507 to execute one of a plurality of receiving modes, including: not receiving input data, receiving summary information, receiving a part of the input data, and receiving all input data. The input data is generally a tensor or a numerical value to be calculated.

[0059] After the data processing module 507 receives the input data, the state management module 506 suspends the program to wait for the completion of the operator calculation. After the calculation is completed, the operator generates output data (i.e. calculation result), and the state management module 506 controls the output interface to be opened to transmit the output data.

[0060] Then the state management module 506 controls the data processing module 507 to receive the output data. Specifically, based on the state parameters 508, the structure information, and the instructions from the user interface module 505, the state management module 506 controls the data processing module 507 to execute one of a plurality of receiving modes, including: not receiving output data, receiving a part of the output data, or receiving all output data. The output data is also a tensor or a numerical value.

[0061] After the data processing module 507 receives the output data, the input data and the output data are analyzed, such as locating faults, obtaining network structure, analyzing calculation logic, and / or analyzing rules. Specifically, the data processing module 507 mainly analyzes and sorts various data types in the deep learning framework and the acceleration library, provides a variety of processing methods such as complete reservation, partial interception, feature extraction, mean value and absolute value for tensors and their derived types, and generates data analysis results. The developer can configure through the user interface of the user interface module 505 as needed.

[0062] Then, the data processing module 507 packs the input data, output data and analysis result to generate a part of the compute-side operator data in response to the control of the state management module 506. Specifically, the data processing module 507 can pack the input data, output data and analysis result into a group of folders sorted by "running order and operator name", and each folder contains files 509 of data information such as operator input and output.

[0063] Finally, the state management module 506 controls the sending of the compute-side operator data to the off-chip. In this embodiment, the compute-side operator data includes input data, output data, analysis result, operator name and structure information, and the state management module 506 sends the operator name and structure information to the processing device 203, while controlling the data processing module 507 to send the files 509 to the processing device 203.

[0064] The state management module 506 then updates the state parameters 508 so that the state parameters 508 carry the latest information, including the current operator number, the current input (or output) number of the operator being processed, the current file directory path, how many inputs and outputs are temporarily stored, and the like.

[0065] At this point, the operation of an operator is completed, and the compute-side operator data of the operator is also obtained. Based on the updated state parameters 508, the state management module 506 performs the foregoing operations for the next operator, and repeats the operations until the compute device 201 runs to the end function, and the global flow of obtaining the compute-side operator data is completed.

[0066] When the processing device 203 receives the compute-side operator data, the structured compute-side operator data can be obtained based on the structure information, and the data of any operator in the acceleration library can be obtained, so that the developer can analyze the data to locate faults or analyze rules.

[0067] This embodiment adds the input interface and the output interface of the operator, globally manages based on the state parameters, and obtains the compute-side operator data, so as to achieve the technical effects of obtaining the data in the network running (especially through the acceleration library) to locate faults or analyze rules, and obtaining the mapping relationship between the operators in the acceleration library and the upper layer of the framework.

[0068] Another embodiment of the present application is a precision verification system having the hardware structure of the foregoing embodiment and capable of obtaining the compute-side operator data. In this embodiment, the developer configures through the interface of the deep learning framework to adjust the running mode to the precision comparison mode, and the configuration information is sent to the distribution control tool 304 and the data processing module 507.

[0069] In comparison mode, the distribution control tool 304 performs the same calculation task twice, on both the computing device 201 and the processing device 203. Upon completion of the first calculation, the task is not immediately terminated; instead, the data processing module 507 temporarily stores the results and input data before redistributing the task to the processing device 203 for the same calculation. In other words, in precision comparison mode, the main change to the distribution control tool 304 is the addition of a secondary run option. The original "distribute-calculate-obtain return value" process is modified to "distribute-calculate-temporarily store the return value from computing device 201-distribute-calculate-obtain the return value from processing device 203," allowing computing device 201 to perform the same calculation twice. Figure 5 After performing the task as shown to obtain the computational operator data, the processing device 203 continues to perform the same task based on the callback operator to obtain the processing operator data.

[0070] In this embodiment, the temporary storage results of the two sets of data are retrieved by the precision comparison module in the processing device 203, and a precision comparison is performed on the input and output respectively. More specifically, the precision comparison module uses the processing-side operator data as reference data and compares the computation-side operator data with the reference data to evaluate the precision of the computation-side operator. The precision comparison module compares not only the output data but also the input data. Its significance lies in ensuring that the content of the two calculations is consistent and avoiding inconsistencies in the input data caused by cache failure or other faults.

[0071] After the accuracy comparison is completed, the data comparison results and the data from both caches are saved to a file for developers to analyze. This embodiment utilizes the processing device 203 to perform the same task, generating processing-side operator data, which is then compared with the computation-side operator data as reference data to obtain accuracy verification results.

[0072] Another embodiment of the present invention is a method for obtaining computational operator data in a deep learning model, which can utilize... Figures 1 to 5 The hardware implementation demonstrates the operational process after the deep learning framework first distributes the task to the acceleration library. Figure 6 This embodiment shows a flowchart of the process for obtaining operator data at the computation end.

[0073] In step 601, input and output interfaces of the operator are added. Based on the operator information, basic interfaces for four types of operators (training operator, inference operator, intelligent programming language operator, and callback operator) are generated, namely, operator input interface and operator output interface.

[0074] In step 602, at least a part of the task is redistributed to the operators. Based on the base class provided by the operator tool, the second distribution is performed to control the calculation and interface call of the operator, and specifically generates a training operator, an inference operator, an intelligent programming language operator, or a callback operator, so that the generated code interface form is in sequence: operator input interface → distribution control entry → distribution control macro → distribution control exit → operator output interface. Through compilation, each operator related to the task can be implemented.

[0075] In step 603, a user interface is generated. The user interface can be a context manager interface, a function interface, or an environment variable interface. The developer can select or set any one or several interfaces according to the actual use scene to match the entire running demand.

[0076] In step 604, an instruction indicating the data to be obtained is received from the user interface. Through the user interface, the developer can send an instruction to determine various configurations, and the instruction carries the information of the configurations, including the timing of obtaining the computation-side operator data, the processing method of the computation-side operator data, and / or the structure of the computation-side operator data.

[0077] In step 605, a start function is inserted before the input interface to start the program of obtaining the computation-side operator data. The developer can determine whether or when to start the global flow through the instruction, and set the position of inserting the start function in the instruction. When the embodiment is executed to the start function, the flow of collecting the computation-side operator data of a series of operators is started.

[0078] In step 606, an end function is inserted after the output interface to end the program. Similarly, the developer can determine whether or when to end the global flow through the instruction, and set the position of inserting the end function in the instruction. When the embodiment is executed to the end function, the flow of collecting the computation-side operator data of a series of operators is ended.

[0079] In step 607, initialization is performed according to the instruction to implement the configuration. When the entire neural network is running, after the start function is executed, initialization is started to implement the configuration of the developer, including creating a file directory, performing preprocessing, resetting state parameters, and the like.

[0080] In step 608, the operator name and structure information are received from the input interface based on the state parameters. Based on the state parameters, the input data is transmitted to the interface for executing the operator by the input interface, and the operator name and structure information are obtained from the input interface, wherein the structure information includes the number of input and output data of the operator, the size, type, and dimension of each data (tensor or numerical value), and the like.

[0081] In step 609, input data is received from the input interface based on the state parameters and the structure information. Specifically, based on the state parameters and the instructions from the user interface, the control executes one of a plurality of receiving modes, including: not receiving input data, receiving summary information, receiving a portion of the input data, and receiving all of the input data. The input data is generally a tensor or a numerical value to be calculated.

[0082] After receiving the input data, the method suspends the program to wait for the completion of the operator calculation. After the completion of the calculation, the operator generates output data (i.e., the calculation result).

[0083] In step 610, output data is received from the output interface based on the state parameters and the structure information. Specifically, based on the state parameters and the instructions from the user interface, the control executes one of a plurality of receiving modes, including: not receiving output data, receiving a portion of the output data, or receiving all of the output data. The output data is also a tensor or a numerical value.

[0084] In step 611, data analysis is performed on the input data and the output data. This step mainly analyzes and organizes various data types in the deep learning framework and the acceleration library. For tensors and their derivative types, various processing methods are provided, such as complete retention, partial interception, feature extraction, mean value, and absolute value, to generate data analysis results. Developers can configure the user interface as needed.

[0085] In step 612, according to the instructions, the data is packaged to generate the calculation-end operator data. Further, the input data, the output data, and the data analysis results are packaged into a file to generate a part of the calculation-end operator data.

[0086] In step 613, the calculation-end operator data is sent off-chip. In this embodiment, the calculation-end operator data includes the input data, the output data, the analysis results, the operator name, and the structure information. The operator name and the structure information are sent off-chip using different paths from the file generated in step 612.

[0087] At this time, the state parameters are updated so that the state parameters carry the latest information, including the current operator number, the current input (or output) number of the operator being processed, the current file directory path, how many inputs and outputs are temporarily stored, and the like.

[0088] At this time, the state parameters are updated so that the state parameters carry the latest information, including the current operator number, the current input (or output) number of the operator being processed, the current file directory path, how many inputs and outputs are temporarily stored, and the like.

[0089] In step 614, it is determined whether the execution has reached the end function. If the end function has not been encountered, the process returns to step 608, and the aforementioned process is executed for the next operator based on the updated state parameters, until the end function is reached, indicating that all the computational operator data of interest to the developer has been obtained and output, and global control ends.

[0090] After receiving the operator data from the computing end, the off-chip processing device can structure the computing end operator data based on its structural information, thereby obtaining the data of any operator in the acceleration library. This data can then be analyzed by developers to locate faults or analyze patterns.

[0091] This embodiment achieves the technical effect of obtaining network operation data to locate faults or analyze patterns, as well as to obtain the mapping relationship between acceleration library operators and upper-layer framework by adding input and output interfaces of operators based on state parameters.

[0092] Another embodiment of the present invention is an accuracy verification method, which can also utilize... Figures 1 to 5 This is implemented through hardware embodiments. Figure 7 A flowchart illustrating the accuracy verification of this embodiment is shown.

[0093] In step 701, configuration information is received to adjust the running mode to precision comparison mode. Developers configure this mode via the deep learning framework's interface.

[0094] In step 702, the response accuracy comparison mode distributes the same computing task into two calculations: one by the computing device and the other by the processing device.

[0095] In step 703, a task is performed using a computing device to obtain computational operator data, i.e., based on... Figure 6 The method shown obtains the computational operator data.

[0096] In step 704, the processing device performs a task to obtain processing-end operator data, which is used as reference data.

[0097] In step 705, the computation-side operator data is compared with reference data to obtain accuracy verification results. More specifically, the computation-side operator data is compared with the reference data, using the processing-side operator data as the reference data. This step compares not only the output data but also the input data, ensuring consistency between the two calculations and preventing inconsistencies caused by cache failures or other malfunctions.

[0098] In step 706, the precision verification result is saved to a file. After the precision comparison is completed, the precision verification result and the twice cached data are saved to a file for the developer to analyze.

[0099] This embodiment generates the processing end operator data by using the processing device to perform the same task as the reference data compared with the computing end operator data to obtain the precision verification result.

[0100] Another embodiment of the present application is a computer readable storage medium, which stores computer program codes for obtaining computing end operator data or precision verification in a deep learning model, and when the computer program codes are executed by a processor, the method of each embodiment described above is executed. In some implementation scenarios, the above-mentioned integrated unit can be implemented in the form of a software program module. If implemented in the form of a software program module and sold or used as an independent product, the integrated unit can be stored in a computer readable storage medium. Based on this, when the scheme of the present application is embodied in the form of a software product (for example, a computer readable storage medium), the software product can be stored in a storage medium, which can include a plurality of instructions for causing a computer device (for example, a personal computer, a server or a network device, etc.) to execute part or all of the steps of the method described in the embodiments of the present application. The aforementioned storage medium can include, but is not limited to, a U disk, a flash disk, a read-only memory (ReadOnly Memory, ROM), a random access memory (Random Access Memory, RAM), a mobile hard disk, a magnetic disk or an optical disk, etc. various media that can store program codes.

[0101] Another embodiment of the present application is a computer program product, which includes a computer program for obtaining computing end operator data or precision verification in a deep learning model, and the computer program is executed by a processor to implement the steps of the foregoing method.

[0102] Another embodiment of the present application is a computer device, which includes a memory, a processor and a computer program stored on the memory, and the processor executes the computer program to implement the steps of the foregoing method.

[0103] One aspect of the present application is to add the input interface and the output interface of the operator and perform global management, and then obtain the computing end operator data, so as to achieve the technical effects of obtaining the data in the network operation to locate the fault or analyze the rule, and obtaining the mapping relationship between the acceleration library operator and the upper layer of the framework. On the other hand, the present application compares the computing end operator data with the processing end operator data, which is helpful to improve the convenience of precision verification.

[0104] The foregoing can be better understood in accordance with the following clauses:

[0105] Clause A1. A method for obtaining computation-end operator data in a deep learning model, comprising: adding an input-end interface and an output-end interface of an operator; receiving input data from the input-end interface based on a state parameter; receiving output data from the output-end interface based on the state parameter to perform data analysis on computation-end operator data, wherein the computation-end operator data comprises the input data and the output data.

[0106] Clause A2. A precision verification method, comprising: performing a task by a processing device to obtain processing-end operator data, the processing-end operator data being reference data; performing the method of clause A1 by a computing device to obtain computation-end operator data; and comparing the computation-end operator data with the reference data to obtain a precision verification result.

[0107] Clause A3. A computer-readable storage medium having stored thereon computer program code for obtaining computation-end operator data or precision verification in a deep learning model, which when executed by a processing device, performs the method of any one of clauses A1 to 2.

[0108] Clause A4. A computer program product comprising a computer program for obtaining computation-end operator data or precision verification in a deep learning model, wherein the computer program, when executed by a processor, implements the steps of the method of any one of clauses A1 to 2.

[0109] Clause A5. A computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the steps of the method of any one of clauses A1 to 2.

[0110] Clause A6. A computing device for obtaining computation-end operator data in a deep learning model, the operator adding an input-end interface and an output-end interface, the computing device comprising: a state management module to maintain a state parameter; and a data processing module to: receive input data from the input-end interface based on the state parameter; receive output data from the output-end interface based on the state parameter to perform data analysis on computation-end operator data; wherein the computation-end operator data comprises the input data and the output data.

[0111] Clause A7. The computing device of clause A6, wherein the data processing module performs data analysis on the input data and the output data, the data analysis being at least one of locating a fault and analyzing a regularity.

[0112] Clause A8. The computing device of any of clauses A6-A7, wherein the state management module receives an operator name and structure information from the ingress interface and sends the operator name and the structure information off-chip, and the compute operator data further comprises the operator name and the structure information, and wherein the data processing module further receives the input data and the output data based on the structure information.

[0113] Clause A9. The computing device of any of clauses A6-A8, further comprising a user interface module to generate a user interface, and wherein the state management module receives instructions from the user interface, the instructions indicating data desired to be obtained, and the data processing module packages the data to generate the compute operator data according to the instructions.

[0114] Clause A10. The computing device of any of clauses A9, wherein the instructions carry information of a configuration, the configuration comprising at least one of a timing of obtaining the compute operator data and a processing manner of the compute operator data.

[0115] Clause A11. The computing device of any of clauses A10, wherein the state management module is initialized according to the instructions to implement the configuration.

[0116] Clause A12. The computing device of any of clauses A9, wherein the user interface is a context manager interface, a function interface, or an environment variable interface.

[0117] Clause A13. The computing device of any of clauses A6-A12, wherein the computing device executes an acceleration library of a deep learning framework.

[0118] Clause A14. The computing device of any of clauses A13, wherein the ingress interface and the egress interface are generated according to operator list information of the acceleration library.

[0119] Clause A15. An integrated circuit device comprising the computing device of any of clauses A6-A14.

[0120] Clause A16. A board card comprising the integrated circuit device of any of clauses A15.

[0121] Clause A17. A precision verification system comprising: a computing device of any of clauses A6-A14 to perform a task to obtain compute operator data; and a processing device to perform a task to obtain processing operator data, the processing operator data being reference data, and to compare the compute operator data with the reference data to obtain a precision verification result.

[0122] The above has carried out the detailed introduction to the embodiment of the application, the principle and implementation mode of the application have been described by applying specific examples in this paper, the above embodiment description is only used for helping understanding the method of the application and its core idea; at the same time, for the general technical personnel in the art, according to the idea of the application, there will be changes in specific implementation mode and application range, and the above is described, the content of the specification should not be understood as the limitation of the application.

Claims

1. A computing device for acquiring data from computational operators in a deep learning model, wherein the operators are equipped with an input interface and an output interface, the input interface and the output interface being generated by a code generation tool based on the operator list information of the acceleration library of the deep learning framework and inserted at the input and output of the operator respectively; The computing device includes: The state management module is used to maintain state parameters; and The data processing module is used for: Based on the status parameters, input data is received from the input interface; Based on the state parameters, output data is received from the output interface to perform data analysis on the computational operator data; The computational operator data includes the input data and the output data.

2. The computing device according to claim 1, wherein the state management module is further configured to receive operator name and structure information from the input interface; The data processing module also receives the input data and the output data based on the structural information; in, The computational operator data also includes the operator name and the structure information.

3. The computing device according to claim 1, further comprising: The user interface module is used to generate the user interface; The state management module receives instructions from the user interface, the instructions indicating the data to be obtained, and the data processing module packages the data according to the instructions to generate the computing terminal operator data.

4. The computing device according to claim 3, wherein the instructions carry configuration information, the configuration including at least one of the timing of acquiring the computing terminal operator data and the processing method of the computing terminal operator data.

5. The computing device according to claim 4, wherein the state management module is initialized according to the instruction to implement the configuration.

6. The computing device according to claim 5, wherein the user interface is a context manager interface, a function interface, or an environment variable interface.

7. The computing device according to any one of claims 1-6, wherein the data processing module performs data analysis on the computing terminal operator data, comprising: Obtaining network structure, locating faults, and analyzing patterns are at least one of these.

8. An accuracy verification system, comprising: The computing device as described in any one of claims 1 to 7 is used to perform a task to obtain computing terminal operator data; as well as A processing device is used to perform tasks to obtain processing-end operator data, wherein the processing-end operator data is used as reference data, and the calculation-end operator data is compared with the reference data to obtain an accuracy verification result.

9. A method for obtaining computational operator data in a deep learning model, comprising: Add input and output interfaces for the operator; The input and output interfaces are generated by the code generation tool based on the operator list information of the acceleration library of the deep learning framework and inserted at the input and output of the operator, respectively. Input data is received from the input interface based on the status parameters; Based on the state parameters, output data is received from the output interface to perform data analysis on the computational operator data, wherein the computational operator data includes the input data and the output data.

10. An accuracy verification method, comprising: The processing device is used to perform a task to obtain processing-end operator data, which is used as reference data; The method described in claim 9 is performed using a computing device to obtain computational operator data; as well as The computational operator data is compared with the reference data to obtain the accuracy verification result.

11. A computer-readable storage medium having stored thereon computer program code for acquiring computational operator data or accuracy verification in a deep learning model, wherein when the computer program code is run by a processing device, it performs the method according to any one of claims 9 to 10.

12. A computer program product comprising a computer program for acquiring computational operator data or accuracy verification in a deep learning model, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 9 to 10.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 9 to 10.

14. An integrated circuit device comprising a computing device according to any one of claims 1 to 7.

15. A board comprising the integrated circuit device according to claim 14.

Images