A method, device, equipment and medium for testing a neural network model

Abstract

The present invention discloses a test method, device, equipment and medium for a neural network model. The method includes: obtaining standard parameters of each node of a standard neural network model that has passed the test on a target chip; running the neural network model to be tested on the target chip to obtain the operating parameters of each node; performing node tests on each node based on the operating parameters and the standard parameters to obtain the node test results of each node; and generating a test report of the neural network model to be tested according to the node test results of each node. The data of each node of the standard neural network model is used as the standard parameter. When the neural network model to be tested with the same structure is run on the target chip later, the operating parameters are compared with the previous standard parameters. Since both data are generated on the target chip, the accuracy will be strictly byte-aligned, and the error will not accumulate with the increase of the number of model layers, thereby improving the accuracy and reliability of the neural network model test.

Description

Technical Field

[0001] The embodiments of the present invention relate to neural network technology, and more particularly to a testing method, device, equipment and medium for a neural network model. Background Art

[0002] During the chip development process, it is usually necessary to test the software stack. Since the model layer is the top layer of the entire software stack, this layer not only needs to rely on the correct operation of the lower layers, but the model layer is also directly used by customers on the chip. Compared with the bottom layer of the software stack, the model layer has the characteristics of long running time and vague test acceptance evaluation criteria.

[0003] Currently, when testing the model layer, it is mainly through testing the neural network model running on the model layer. There are two commonly used methods: one is the model accuracy test of multiple step data sets, that is, the training or inference test completed by running the model on the complete data set, and measuring the correctness of the model operation through task accuracy indicators such as accuracy or perplexity; the other is the single step multi-chip model accuracy test, that is, the output of the fixed model and the randomness of the framework, using the same training parameters to run a step on the target chip and the correct reference standard chip respectively, transfer and store the tensors of activation values ​​or gradients of multiple model layers, and then compare whether the errors of these tensors of the target chip and the correct reference standard chip meet the preset conditions.

[0004] However, when using the above methods to test the model, although the test results of the first test method are more accurate, the test time is longer; although the test efficiency of the second test method is high, since the output error of the model layer is cumulative and the setting of the threshold depends on experience, the test results may have errors, so there is a certain recall rate. Summary of the invention

[0005] The embodiments of the present invention provide a method, device, equipment and medium for testing a neural network model to achieve reliable testing of the neural network model.

[0006] In a first aspect, an embodiment of the present invention provides a method for testing a neural network model, comprising: obtaining standard parameters of each node of a standard neural network model that has passed testing on a chip;

[0007] Running the neural network model to be tested on the chip to obtain operating parameters of each node, wherein the underlying structure of the software stack of the neural network model to be tested and the standard neural network model on the chip is different;

[0008] Performing node testing on each node based on the operating parameters and the standard parameters to obtain a node testing result of each node;

[0009] A test report of the neural network model to be tested is generated according to the node test results of each node.

[0010] In a second aspect, an embodiment of the present invention provides a testing device for a neural network model, including: a standard parameter acquisition module, used to acquire standard parameters of each node of a standard neural network model that has passed the test on a chip;

[0011] An operation parameter acquisition module, used for running the neural network model to be tested on the chip to obtain the operation parameters of each node, wherein the underlying structure of the software stack of the neural network model to be tested and the standard neural network model on the chip is different;

[0012] A node test result acquisition module, used to perform a node test on each node based on the operating parameters and the standard parameters to obtain a node test result of each node;

[0013] A test report generation module is used to generate a test report of the neural network model to be tested based on the node test results of each node.

[0014] The present invention uses the data of each node of the standard neural network model as the standard parameter under the condition that the standard neural network model has been adapted and tested on the target chip software stack and the accuracy is correct. When the neural network model to be tested with the same structure is run on the target chip at a later time, the running parameters are compared with the previous standard parameters. Since both data are generated on the target chip, the accuracy will be strictly byte-aligned, and errors will not be accumulated as the number of model layers increases, thereby improving the accuracy and reliability of the neural network model test. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of a testing method of a neural network model in Embodiment 1 of the present invention;

[0016] Figure 2 It is a structural schematic diagram of a neural network model in the first embodiment of the present invention;

[0017] Figure 3 This is a flow chart of a method for testing a neural network model in Embodiment 2 of the present invention;

[0018] Figure 4 It is a structural schematic diagram of a neural network model device in implementation three of the present invention;

[0019] Figure 5 It is a structural diagram of a computer device in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0021] Embodiment 1

[0022] Figure 1 This is a flow chart of a method for testing a neural network model provided in the first embodiment of the present invention. This embodiment is applicable to the case of testing a neural network model on a chip model layer. The method can be executed by a test device for testing a neural network of a device, and the device can be implemented in the form of hardware and / or software. Figure 1 As shown, the method includes:

[0023] Step S101, obtaining standard parameters of each node of the standard neural network model that has passed the test on the target chip.

[0024] Specifically, the target chip of this embodiment includes a Domain Specific Architecture (DSA) chip. The software stack on the DSA chip includes an upper structure and an underlying structure. The upper structure includes a model layer, and the underlying structure includes a driver layer, a compiler layer, an operator layer, and a framework layer. Therefore, the model layer is the top layer of the software stack. The model layer not only depends on the correct operation of the underlying structure, but is also the part that customers directly use on the chip. The neural network runs in the model layer. When the model test is correct, it means that the following software stack is correct.

[0025] Among them, Figure 2 The figure shows a schematic diagram of the structure of the neural network model. The standard neural network model in this embodiment includes a standard reasoning model or a standard training model; when the standard neural network model is a standard reasoning model, the neural network model to be tested is the reasoning model to be tested; when the standard neural network model is a standard training model, the neural network model to be tested is the training model to be tested. The specific type of the neural network model is not limited in this embodiment. In addition, the standard neural network model in this embodiment has been tested on the target chip, and the test of the standard neural network model can adopt the existing multiple step data set model accuracy test method, or a single step multi-chip model accuracy test, and the correct standard parameters of each node obtained through the test will be saved.

[0026] It should be noted that the running operations corresponding to different types of neural network models are different, such as Figure 2As shown, when it is determined to be an inference model, it only involves the forward calculation process forward of the left data flow, and the first output tensor of each node, that is, the activation value, will be automatically transferred and dumped during the operation; when it is determined to be a training model, it not only involves the above-mentioned forward calculation process, but also involves the reverse calculation process backward of the right data flow. During the operation, the second output tensor of each node, that is, the gradient, will be further transferred and dumped, and the above tensors will be saved to the disk with the specified name. In this implementation, the specific location where the transferred tensors are saved is not limited. As long as the data can be saved, it is within the protection scope of this application.

[0027] Step S102, running the neural network model to be tested on the target chip to obtain the operating parameters of each node.

[0028] Specifically, since the DSA chip will automatically generate the latest software version, including the latest functions and fixes, i.e., the Nightly version software stack, this version is usually used for testing and verification to ensure that the newly added functions or fixes will not introduce new errors. Therefore, when the underlying structure of the software stack on the DSA chip changes, the parameters of the standard neural network will change accordingly, but its own structure remains unchanged, for example, the number of nodes, node connection relationships, and the number of layers, etc. At this time, it is used as the neural network model to be tested and tested on the target chip. Therefore, the neural network model to be tested in this embodiment and the standard neural network model are essentially the same model, except that the underlying structure of the software stack on the target chip is different.

[0029] Optionally, the neural network model to be tested is run on the target chip to obtain the operating parameters of each node, including: obtaining preparation data of the neural network model to be tested; and running the neural network model to be tested on the target chip based on the preparation data to obtain the operating parameters of each node.

[0030] Optionally, when the neural network model to be tested is an inference model to be tested, the prepared data includes test data and a model file that has converged; based on the prepared data, the neural network model to be tested is run on the target chip to obtain the operating parameters of each node, including: loading the inference model to be tested on the target chip based on the model file that has converged; inputting the test data into the loaded inference model to be tested, and obtaining the operating parameters of each node when the inference model to be tested performs a forward calculation.

[0031] Optionally, when the neural network model to be tested is a training model to be tested, the prepared data includes training data and an unconverged model file; based on the prepared data, the neural network model to be tested is run on the target chip to obtain the operating parameters of each node, including: initializing the training model to be tested on the target chip based on the unconverged model file; inputting the training data into the initialized training model to be tested, and obtaining the operating parameters of each node when the training model to be tested performs a forward calculation and a reverse calculation.

[0032] Optionally, when the neural network model to be tested is an inference model to be tested, the types of standard parameters and operating parameters include activation values; when the neural network model to be tested is a training model to be tested, the types of standard parameters and operating parameters include activation values ​​and gradient values.

[0033] In this embodiment, when the neural network is updated due to the change of the software stack, the position of the neural network on the original target chip will be kept unchanged, and the neural network model will be compared with its own model before the version update on the target chip, so as to test the neural network model to be tested after the version update. Before the comparison test, the neural network model to be tested will be run on the target chip to obtain the operating parameters of each node, and the preparation data corresponding to different types of neural networks during operation and the obtained operating parameters are different.

[0034] In a specific implementation, when the neural network model to be tested is the inference model to be tested, the prepared data includes test data and the converged model file. In order to obtain the operating parameters of the inference model to be tested, the inference model to be tested will be first loaded onto the target chip through the converged model file, and the test data will be input into the loaded inference model to be tested. When the inference model to be tested is inferred based on the input test data, the model will obtain the operating parameters of each node in the inference model to be tested for each forward calculation step. At this time, the obtained operating parameters include the activation value to be tested. In addition, in order to further speed up the test efficiency, the user can also pre-specify nodes that are prone to errors, such as residual network nodes, etc., and only obtain the activation values ​​of the residual network nodes. Of course, this embodiment is only an example, and does not limit the specific type of the specified node.

[0035] In another specific implementation, when the neural network model to be tested is the training model to be tested, the prepared data includes training data and an unconverged model file. Since the training model to be tested has not yet converged, it corresponds to an unconverged model file. In order to obtain the operating parameters of the training model to be tested, the training model to be tested will be initialized on the target chip through the unconverged model file, and the training data will be input into the initialized training model to be tested. When the training model to be tested is trained based on the input training data, one step of model training requires running a forward calculation and a reverse calculation. At this time, the operating parameters obtained include the activation value to be tested corresponding to the forward calculation, and the gradient value to be tested corresponding to the reverse calculation. Therefore, the operating parameters of each node obtained for the training model include both the activation value to be tested and the gradient value to be tested.

[0036] Step S103: Perform node testing on each node based on the operating parameters and standard parameters to obtain the node testing results of each node.

[0037] Optionally, a node test is performed on each node based on operating parameters and standard parameters to obtain a node test result of each node, including: comparing the operating parameters of each node with the standard parameters to obtain a comparison result, and determining whether the comparison result is the same; if so, determining that the node test has passed; otherwise, determining that the node test has failed.

[0038] Optionally, the operating parameters of each node are compared with the standard parameters to obtain a comparison result, including: performing hash calculation on the operating parameters of each node to obtain the CRC32 code to be tested; performing hash calculation on the standard parameters of each node to obtain the standard CRC32 code; and comparing the CRC32 code to be tested of each node with the standard CRC32 code to obtain a comparison result.

[0039] Optionally, the operating parameters of each node are compared with the standard parameters to obtain a comparison result, including: performing regularization calculation on the operating parameters of each node to obtain a two-norm regularization term to be tested; performing regularization calculation on the standard parameters of each node to obtain a standard two-norm regularization term; and comparing the two-norm regularization term to be tested of each node with the standard two-norm regularization term to obtain a comparison result.

[0040] Specifically, in this embodiment, after obtaining the operating parameters of each node and the saved standard parameters, the operating parameters will be compared with the standard parameters. If the comparison result shows that the two are the same, it means that the node test has passed. If the comparison result shows that the two are different, it means that the node test has failed. Since both data are generated on the target chip, the accuracy will be strictly byte-aligned, and the error will not accumulate as the number of model layers increases, thereby improving the accuracy and reliability of the neural network model test.

[0041] Among them, in this implementation, when comparing two data, multiple methods can be used: one is to directly compare the operating parameters and the standard parameters; the second is to perform hash calculations on the operating parameters and the standard parameters respectively, and compare the CRC32 codes after the hash calculations; the third is to perform regularization calculations on the operating parameters and the standard parameters respectively, and compare the two-norm regularization terms of the regularization calculations. Among them, the second and third methods have significantly reduced data capacity compared to the first method, so the storage space can be significantly reduced. The following is an explanation of different scenarios for different types of neural networks:

[0042] The first scenario: When the standard neural network model is the standard inference model and the neural network model to be tested is the inference model to be tested,

[0043] Case 1: directly compare the activation value M of each node of the inference model to be tested with the activation value M' of each node of the standard inference model. When M and M' are the same, it is determined that the node test has passed;

[0044] Case 2: The activation value M of each node in the inference model to be tested is hashed to obtain the CRC32 code M1 to be tested, and the activation value M' of each node in the standard inference model is hashed to obtain the marked CRC32 code M1'. When M1 is the same as M1', it is determined that the node test has passed;

[0045] Case 3: The activation value A of each node in the inference model to be tested is regularized to obtain the two-norm regularization term M2 to be tested, and the activation value M' of each node in the standard inference model is regularized to obtain the standard two-norm regularization term M2'. When M2 is the same as M2', it is determined that the node test has passed.

[0046] The second scenario: When the standard neural network model is the standard training model and the neural network model to be tested is the training model to be tested,

[0047] Case 1: directly compare the activation value M of each node of the inference model to be tested with the activation value M' of each node of the standard inference model, and directly compare the gradient value N of each node of the inference model to be tested with the gradient value N' of each node of the standard inference model. When M is the same as M' and N is the same as N', the node test is determined to have passed.

[0048] Case 2: Perform hash calculation on the activation value M of each node in the inference model to be tested to obtain the CRC32 code M1 to be tested, and perform hash calculation on the activation value M' of each node in the standard inference model to obtain the standard CRC32 code M1'; perform hash calculation on the gradient value N of each node in the inference model to be tested to obtain the CRC32 code N1 to be tested, and perform hash calculation on the gradient value N' of each node in the standard inference model to obtain the standard CRC32 code N1'. When M1 is the same as M1' and N1 is the same as N1', it is determined that the node test has passed.

[0049] Case three: The activation value M of each node in the inference model to be tested is regularized to obtain the two-norm regularization term M2 to be tested, and the activation value M' of each node in the standard inference model is regularized to obtain the standard two-norm regularization term M2'; the gradient value N of each node in the inference model to be tested is regularized to obtain the two-norm regularization term N2 to be tested, and the gradient value N' of each node in the standard inference model is hashed to obtain the two-norm regularization term N2'. When M2 is the same as M2' and N2 is the same as N2', it is determined that the node test has passed.

[0050] Step S104, generating a test report of the neural network model to be tested according to the node test results of each node.

[0051] Optionally, a test report of the neural network model to be tested is generated based on the node test results of each node, including: obtaining model test results of the neural network model to be tested based on the node test results of each node; and generating a test report based on the node test results of each node and the model test results.

[0052] Specifically, in the present embodiment, after obtaining the node test results of each node, since the node test results specifically include the specific circumstances of whether the node test is successful or not, for example, when the neural network to be tested contains 50 nodes, if all the nodes pass the test, then the model test result of the neural network to be tested is determined to be passed; as long as there is a situation where one node fails the test, then the model test result of the neural network to be tested is determined to be failed.

[0053] Among them, in this embodiment, model repair suggestions are automatically generated according to the node test results and model test results, and a pre-imported report template is obtained, the node test results are placed in the node test block, the model test results are placed in the model test block, and the model repair suggestions are placed in the suggestion block. In addition, nodes that fail the test in the node test block will be marked with specific symbols, for example, in the form of red fonts, so that users can quickly determine the abnormal nodes based on the test report. Of course, this embodiment is only an example, and does not limit the specific generation method of the test report.

[0054] In the implementation mode of the present application, under the condition that the standard neural network model has been adapted and tested on the target chip software stack and the accuracy is correct, the data of each node of the standard neural network model is used as the standard parameter. When the neural network model to be tested with the same structure is run on the target chip later, the operating parameters are compared with the previous standard parameters. Since both data are generated on the target chip, the accuracy will be strictly byte-aligned, and errors will not accumulate with the increase in the number of model layers, thereby improving the accuracy and reliability of the neural network model test.

[0055] Embodiment 2

[0056] Figure 3 This is a flowchart of a method for testing a neural network model provided in the second embodiment of the present invention. This embodiment is based on the above embodiment. After generating a test report of the neural network model to be tested according to the node test results of each node, it also includes: visually displaying the test report on the front end; receiving an editing instruction sent by a user based on the displayed test report, and editing the test report according to the editing instruction. Figure 3 As shown, the method includes:

[0057] Step S201, obtaining standard parameters of each node of the standard neural network model that has passed the test on the target chip.

[0058] Step S202, running the neural network model to be tested on the target chip to obtain the operating parameters of each node.

[0059] Optionally, the neural network model to be tested is run on the target chip to obtain the operating parameters of each node, including: obtaining preparation data of the neural network model to be tested; and running the neural network model to be tested on the target chip based on the preparation data to obtain the operating parameters of each node.

[0060] Optionally, when the neural network model to be tested is an inference model to be tested, the prepared data includes test data and a model file that has converged; based on the prepared data, the neural network model to be tested is run on the target chip to obtain the operating parameters of each node, including: loading the inference model to be tested on the target chip based on the model file that has converged; inputting the test data into the loaded inference model to be tested, and obtaining the operating parameters of each node when the inference model to be tested performs a forward calculation.

[0061] Optionally, when the neural network model to be tested is a training model to be tested, the prepared data includes training data and an unconverged model file; based on the prepared data, the neural network model to be tested is run on the target chip to obtain the operating parameters of each node, including: initializing the training model to be tested on the target chip based on the unconverged model file; inputting the training data into the initialized training model to be tested, and obtaining the operating parameters of each node when the training model to be tested performs a forward calculation and a reverse calculation.

[0062] Optionally, when the neural network model to be tested is an inference model to be tested, the types of standard parameters and operating parameters include activation values; when the neural network model to be tested is a training model to be tested, the types of standard parameters and operating parameters include activation values ​​and gradient values.

[0063] Step S203: Perform node testing on each node based on the operating parameters and standard parameters to obtain node testing results of each node.

[0064] Optionally, a node test is performed on each node based on operating parameters and standard parameters to obtain a node test result of each node, including: comparing the operating parameters of each node with the standard parameters to obtain a comparison result, and determining whether the comparison result is the same; if so, determining that the node test has passed; otherwise, determining that the node test has failed.

[0065] Optionally, the operating parameters of each node are compared with the standard parameters to obtain a comparison result, including: performing hash calculation on the operating parameters of each node to obtain the CRC32 code to be tested; performing hash calculation on the standard parameters of each node to obtain the standard CRC32 code; and comparing the CRC32 code to be tested of each node with the standard CRC32 code to obtain a comparison result.

[0066] Optionally, the operating parameters of each node are compared with the standard parameters to obtain a comparison result, including: performing regularization calculation on the operating parameters of each node to obtain a two-norm regularization term to be tested; performing regularization calculation on the standard parameters of each node to obtain a standard two-norm regularization term; and comparing the two-norm regularization term to be tested of each node with the standard two-norm regularization term to obtain a comparison result.

[0067] Step S204, generating a test report of the neural network model to be tested according to the node test results of each node.

[0068] Optionally, a test report of the neural network model to be tested is generated based on the node test results of each node, including: obtaining model test results of the neural network model to be tested based on the node test results of each node; and generating a test report based on the node test results of each node and the model test results.

[0069] Step S205, visually displaying the test report on the front end; receiving an editing instruction sent by the user based on the displayed test report, and editing the test report according to the editing instruction.

[0070] Specifically, in this embodiment, after obtaining the test report, the test report will be visually displayed on the front end, so that the maintenance personnel can intuitively and quickly obtain the test results of the neural network model based on the displayed test report, and accurately locate the faulty node. And after the maintenance personnel repair the model of the faulty node with reference to the model repair suggestion, the repair result can be added to the test report according to the user's editing instructions. In addition, when the user detects the faulty node and determines that the cause of the fault is inconsistent with the repair suggestion in the test report, the corresponding faulty node can be marked at the location, so that subsequent maintenance personnel can efficiently obtain the historical repair record of the faulty node. Of course, in this embodiment, only the above two editing methods are used as examples for illustration, and the specific content edited by the test report is not limited in this embodiment.

[0071] It should be noted that in order to facilitate the identification of test reports and to help maintenance personnel identify whether the currently displayed test report has been edited, the edited test report will be marked with a specified symbol. For example, when a test report has been edited, an asterisk mark will be added to a specified location, such as the upper right corner, and different numbers of model marks can be used to mark the number of edits. Of course, this implementation is only an example and does not limit the specific content of the added mark.

[0072] In the implementation mode of the present application, under the condition that the standard neural network model has been adapted and tested on the target chip software stack and the accuracy is correct, the data of each node of the standard neural network model is used as the standard parameter. When the neural network model to be tested with the same structure is run on the target chip later, the operating parameters are compared with the previous standard parameters. Since both data are generated on the target chip, the accuracy will be strictly byte-aligned, and errors will not accumulate with the increase in the number of model layers, thereby improving the accuracy and reliability of the neural network model test.

[0073] Embodiment 3

[0074] Figure 4 This is a schematic diagram of the structure of a test device for a neural network model provided in Embodiment 3 of the present invention. Figure 4 As shown, the device includes: a standard parameter acquisition module 310, an operation parameter acquisition module 320, a node test result acquisition module 330 and a test report generation module 340.

[0075] A standard parameter acquisition module 310 is used to acquire standard parameters of each node of a standard neural network model that has passed the test on a target chip;

[0076] An operation parameter acquisition module 320 is used to run the neural network model to be tested on the target chip to obtain the operation parameters of each node, wherein the underlying structure of the software stack of the neural network model to be tested and the standard neural network model on the target chip is different;

[0077] The node test result acquisition module 330 is used to perform node tests on each node based on operating parameters and standard parameters to obtain node test results of each node;

[0078] The test report generation module 340 is used to generate a test report of the neural network model to be tested according to the node test results of each node.

[0079] Optionally, the target chip includes a domain-specific architecture DSA chip, and the software stack on the DSA chip includes an upper structure and an underlying structure, the upper structure includes a model layer, and the underlying structure includes a driver layer, a compiler layer, an operator layer and a framework layer.

[0080] Optionally, the standard neural network model includes a standard inference model or a standard training model;

[0081] When the standard neural network model is the standard reasoning model, the neural network model to be tested is the reasoning model to be tested;

[0082] When the standard neural network model is the standard training model, the neural network model to be tested is the training model to be tested.

[0083] Optionally, a parameter acquisition module is run to obtain preparation data of the neural network model to be tested;

[0084] Based on the prepared data, the neural network model to be tested is run on the target chip to obtain the operating parameters of each node.

[0085] Optionally, when the neural network model to be tested is an inference model to be tested, the prepared data includes test data and a converged model file; a parameter acquisition module is run to load the inference model to be tested on a target chip based on the converged model file;

[0086] The test data is input into the loaded inference model to be tested, and the operating parameters of each point are obtained when the inference model to be tested performs a forward calculation.

[0087] Optionally, when the neural network model to be tested is a training model to be tested, the prepared data includes training data and an unconverged model file; a parameter acquisition module is run to initialize the training model to be tested on the target chip based on the unconverged model file;

[0088] The training data is input into the initialized training model to be tested, and the operating parameters of each point are obtained when the training model to be tested performs one forward calculation and one reverse calculation.

[0089] Optionally, a node test result acquisition module is used to compare the operating parameters of each node with the standard parameters to obtain a comparison result.

[0090] It is determined whether the comparison results are the same. If so, it is determined that the node test has passed. Otherwise, it is determined that the node test has failed.

[0091] Optionally, a node test result acquisition module is used to perform hash calculation on the operating parameters of each node to obtain the CRC32 code to be tested;

[0092] Perform hash calculation on the standard parameters of each node to obtain the standard CRC32 code;

[0093] The CRC32 code to be tested of each node is compared with the standard CRC32 code to obtain a comparison result.

[0094] Optionally, a node test result acquisition module is used to perform regularization calculation on the operating parameters of each node to obtain a two-norm regularization term to be tested;

[0095] Regularize the standard parameters of each node to obtain the standard two-norm regularization term;

[0096] The two-norm regularization term to be tested of each node is compared with the standard two-norm regularization term to obtain a comparison result. Optionally, when the neural network model to be tested is an inference model to be tested, the types of standard parameters and operating parameters include activation values;

[0097] When the neural network model to be tested is a training model to be tested, the types of standard parameters and operating parameters include activation values ​​and gradient values.

[0098] Optionally, a test report generation module is used to obtain a model test result of the neural network model to be tested according to the node test result of each node;

[0099] Generate a test report based on the node test results and model test results of each node.

[0100] Optionally, the device further includes a test report editing module for visually displaying the test report on the front end;

[0101] An editing instruction sent by a user based on the displayed test report is received, and the test report is edited according to the editing instruction.

[0102] The testing device for the neural network model provided in the embodiment of the present invention can execute the testing method for the neural network model provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.

[0103] Embodiment 4

[0104] Figure 5 A schematic diagram of the structure of a computer device provided in Embodiment 6 of the present invention is shown in FIG. Figure 5 As shown, the computer device includes a processor 610, a memory 620, an input device 630, and an output device 640; the number of the processor 610 in the computer device can be one or more. Figure 5 A processor 610 is taken as an example; the processor 610, memory 620, input device 630 and output device 640 in the computer device can be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0105] The memory 620, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the parameter quantization method of the recurrent neural network in the embodiment of the present invention. The processor 610 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 620, that is, implements the above-mentioned test method of the neural network model.

[0106] The testing methods for neural network models include:

[0107] Obtain standard parameters of each node of the standard neural network model that has been tested on the target chip;

[0108] Running the neural network model to be tested on the target chip to obtain the operating parameters of each node, wherein the underlying structure of the software stack of the neural network model to be tested and the standard neural network model on the target chip is different;

[0109] Perform node testing on each node based on operating parameters and standard parameters to obtain node test results for each node;

[0110] Generate a test report for the neural network model to be tested based on the node test results of each node.

[0111] The memory 620 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 620 may further include a memory remotely arranged relative to the processor 610, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0112] The input device 630 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the computer device. The output device 640 may include a display device such as a display screen.

[0113] Embodiment 5

[0114] Embodiment 7 of the present invention further provides a storage medium containing computer executable instructions, which are used to execute a method for testing a neural network model when executed by a computer processor;

[0115] Testing methods for neural network models, including:

[0116] Obtain standard parameters of each node of the standard neural network model that has been tested on the target chip;

[0117] Running the neural network model to be tested on the target chip to obtain the operating parameters of each node, wherein the underlying structure of the software stack of the neural network model to be tested and the standard neural network model on the target chip is different;

[0118] Perform node testing on each node based on operating parameters and standard parameters to obtain node test results for each node;

[0119] A test report of the neural network model to be tested is generated based on the node test results of each node.

[0120] Of course, the storage medium containing computer executable instructions provided by an embodiment of the present invention, whose computer executable instructions are not limited to the above method operations, can also execute related operations in the testing method of the neural network model provided by any embodiment of the present invention.

[0121] Through the above description of the implementation mode, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation mode. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0122] It is worth noting that in the embodiment of the testing device for the above-mentioned neural network model, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0123] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for testing a neural network model, characterized in that: include: Obtain standard parameters of each node of the standard neural network model that has been tested on the target chip; Running the neural network model to be tested on the target chip to obtain operating parameters of each node, wherein the underlying structure of the software stack of the neural network model to be tested and the standard neural network model on the target chip is different; Performing node testing on each node based on the operating parameters and the standard parameters to obtain a node testing result of each node; A test report of the neural network model to be tested is generated according to the node test results of each node.

2. The method according to claim 1, characterized in that The target chip includes a domain-specific architecture (DSA) chip, and the software stack on the DSA chip includes an upper structure and an underlying structure, wherein the upper structure includes a model layer, and the underlying structure includes a driver layer, a compiler layer, an operator layer, and a framework layer.

3. The method according to claim 1, characterized in that The standard neural network model includes a standard reasoning model or a standard training model; When the standard neural network model is the standard reasoning model, the neural network model to be tested is the reasoning model to be tested; When the standard neural network model is the standard training model, the neural network model to be tested is the training model to be tested.

4. The method according to claim 3, characterized in that The step of running the neural network model to be tested on the target chip to obtain the operating parameters of each node includes: Acquiring preparation data of the neural network model to be tested; Based on the prepared data, the neural network model to be tested is run on the target chip to obtain the operating parameters of each node.

5. The method according to claim 4, characterized in that When the neural network model to be tested is an inference model to be tested, the prepared data includes test data and a converged model file; The step of running the neural network model to be tested on the target chip based on the prepared data to obtain the operating parameters of each node includes: Loading the inference model to be tested on the target chip based on the converged model file; The test data is input into the loaded inference model to be tested, and the operating parameters of each node are obtained when the inference model to be tested performs a forward calculation.

6. The method according to claim 4, characterized in that When the neural network model to be tested is a training model to be tested, the prepared data includes training data and an unconverged model file; The step of running the neural network model to be tested on the target chip based on the prepared data to obtain the operating parameters of each node includes: Initializing the training model to be tested on the target chip based on the unconverged model file; The training data is input into the initialized training model to be tested, and the operating parameters of each node are obtained when the training model to be tested performs one forward calculation and one reverse calculation.

7. The method according to claim 3, characterized in that The performing node testing on each node based on the operating parameters and the standard parameters to obtain a node test result of each node includes: Compare the operating parameters of each node with the standard parameters to obtain a comparison result, It is determined whether the comparison results are the same, if so, it is determined that the node test has passed, otherwise, it is determined that the node test has failed.

8. The method according to claim 7, characterized in that The comparing the operating parameters of each node with the standard parameters to obtain a comparison result includes: Perform hash calculation on the operating parameters of each node to obtain the CRC32 code to be tested; Perform hash calculation on the standard parameters of each node to obtain a standard CRC32 code; The CRC32 code to be tested of each node is compared with the standard CRC32 code to obtain a comparison result.

9. The method according to claim 7, characterized in that: The comparing the operating parameters of each node with the standard parameters to obtain a comparison result includes: Perform regularization calculation on the operating parameters of each node to obtain a two-norm regularization term to be tested; Perform regularization calculation on the standard parameters of each node to obtain a standard two-norm regularization term; The two-norm regularization term to be tested of each node is compared with the standard two-norm regularization term to obtain a comparison result.

10. The method according to claim 1, characterized in that When the neural network model to be tested is an inference model to be tested, the types of the standard parameters and the operating parameters include activation values; When the neural network model to be tested is a training model to be tested, the types of the standard parameters and the operating parameters include activation values ​​and gradient values.

11. The method according to claim 7, characterized in that Generating a test report of the neural network model to be tested according to the node test results of each node includes: Obtaining a model test result of the neural network model to be tested according to the node test result of each node; The test report is generated according to the node test results of each node and the model test results.

12. The method according to claim 1, characterized in that After generating the test report of the neural network model to be tested according to the node test results of each node, the method further includes: Visually display the test report on the front end; An editing instruction sent by a user based on the displayed test report is received, and the test report is edited according to the editing instruction.

13. A testing device for a neural network model, characterized in that: include: A standard parameter acquisition module is used to obtain standard parameters of each node of the standard neural network model that has passed the test on the target chip; An operation parameter acquisition module, used for running the neural network model to be tested on the target chip to obtain the operation parameters of each node, wherein the underlying structure of the software stack of the neural network model to be tested and the standard neural network model on the target chip is different; A node test result acquisition module, used to perform a node test on each node based on the operating parameters and the standard parameters to obtain a node test result of each node; A test report generation module is used to generate a test report of the neural network model to be tested based on the node test results of each node.

14. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 12 is implemented.

15. A computer executable instruction storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images