Chip verification method, device, electronic device and storage medium

Abstract

The present disclosure provides a chip verification method, device, electronic device and storage medium, the method comprising: obtaining a neural network to be verified; generating a target operator consistent with the operator type of at least one original operator included in the neural network to be verified; replacing the corresponding original operator in the neural network to be verified with the target operator to generate a replaced neural network; training the replaced neural network and the neural network to be verified respectively with training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network; generating a verification result corresponding to the chip to be verified based on the first training result and the second training result.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning technology, and in particular to a chip verification method, device, electronic device, and storage medium. Background Art

[0002] Deep learning is a computationally intensive algorithm. As the diversity and complexity of processing tasks increase, the accuracy and real-time requirements of the algorithm continue to increase, making the scale of the neural network used in deep learning larger and larger, and more computing resources and storage resources are required to run the neural network. Therefore, the designed artificial intelligence (AI) chip can be used to run the neural network.

[0003] Generally, the designed AI chip needs to be evaluated before it is taped out. Summary of the invention

[0004] In view of this, the present disclosure at least provides a chip verification method, device, electronic device and storage medium.

[0005] In a first aspect, the present disclosure provides a chip verification method, comprising:

[0006] Obtain the neural network to be verified;

[0007] For an operator type of at least one original operator included in the neural network to be verified, generating a target operator consistent with the operator type;

[0008] Using the target operator to replace the corresponding original operator in the neural network to be verified, to generate a replaced neural network;

[0009] Using the training samples, respectively training the replaced neural network and the neural network to be verified to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network;

[0010] Based on the first training result and the second training result, a verification result corresponding to the chip to be tested is generated.

[0011] In the above method, after obtaining the neural network to be verified, a target operator is generated that is consistent with the operator type of at least one original operator included in the neural network to be verified. Since the target data processing method corresponding to the target operator matches the preset data processing method corresponding to the operator type indicated by the design file of the chip to be verified, the simulation of floating-point operations in the chip to be verified is realized by using the target operator. Then, the target operator can be used to replace the corresponding original operator in the neural network to be verified to generate a replaced neural network. And using the training samples, the neural network to be verified and the replaced neural network are trained respectively to obtain the first training result and the second training result. Since the first training result can be determined as the benchmark result of the network to be verified, the second training result can be determined as the simulation result when the chip to be verified runs the neural network to be verified. Then, using the first training result and the second training result, the verification result of the chip to be verified can be generated more accurately, and the accuracy of the chip to be verified can be evaluated.

[0012] In a possible implementation, the generating a target operator consistent with the operator type of at least one original operator included in the neural network to be verified includes:

[0013] For at least one operator type of the original operator included in the neural network to be verified, generating an initial operator corresponding to the operator type;

[0014] Testing the initial operator to generate a test result corresponding to the initial operator;

[0015] When the test result indicates that the test fails, the initial operator is adjusted to obtain an adjusted initial operator; the adjusted initial operator is used as a new initial operator, and the process returns to the step of testing the initial operator to obtain a test result, until the test result of the adjusted initial operator indicates that the test passes;

[0016] When the test result indicates that the test is passed, the initial operator is determined as a target operator that is consistent with the operator type.

[0017] Here, for at least one operator type of the original operator, after generating an initial operator corresponding to the operator type, the initial operator can be tested. When the test fails, the initial operator can be adjusted until the test passes. The initial operator that passes the test is determined as the target operator consistent with the operator type, which ensures the performance of the generated target operator, so that when the replaced neural network is obtained based on the target operator later, the network performance deviation between the replaced neural network and the neural network to be verified is small. Based on the replaced neural network and the neural network to be verified with smaller deviations, the chip to be verified can be verified more accurately.

[0018] In a possible implementation manner, the testing the initial operator to generate a test result corresponding to the initial operator includes:

[0019] Inputting the preset feature data into the initial operator to obtain first detection data corresponding to the initial operator; wherein the first detection data includes: predicted output data obtained when the preset feature data includes input feature data, and / or predicted gradient data obtained when the preset feature data includes input gradient data; and

[0020] Inputting the preset feature data into an original operator matching the initial operator to obtain second detection data corresponding to the original operator;

[0021] A test result corresponding to the initial operator is generated based on the first detection data and the second detection data.

[0022] In a possible implementation manner, the using the training sample to train the replaced neural network to obtain a second training result corresponding to the replaced neural network includes:

[0023] On the selected training framework, using the training samples, and according to the first floating-point operation mode indicated by the training framework, the replaced neural network is trained to obtain a first floating-point training result; and / or,

[0024] On the selected training framework, using the training samples, and according to the second floating-point operation mode used by the chip to be tested, the replaced neural network is trained to obtain a second floating-point training result;

[0025] At least one of the first floating-point training result and the second floating-point training result is determined as a second training result corresponding to the replaced neural network.

[0026] Under the above implementation mode, the replaced neural network can be trained according to the first floating-point operation method on the selected training framework, and / or the replaced neural network can be trained according to the second floating-point operation method. The training method is more flexible, which improves the flexibility and diversity of the second training result, so that the first training result and the second training result can be used to verify the chip to be tested more accurately.

[0027] In a possible implementation manner, generating a verification result corresponding to the chip to be tested based on the first training result and the second training result includes:

[0028] If a first deviation value between a first floating-point training result in the second training result and the first training result is less than or equal to a set first threshold, and a second deviation value between the first floating-point training result and a second floating-point training result in the second training result is less than or equal to a set second threshold, generating a verification result indicating that the chip to be tested has passed verification;

[0029] When a first deviation value between the first floating-point training result and the second training result is greater than a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is less than or equal to a set second threshold, generating a verification result indicating that the design file of the chip to be tested needs to be adjusted;

[0030] When a first deviation value between the first floating-point training result and the second training result is less than or equal to a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is greater than a set second threshold, generating a verification result indicating that a floating-point operation mode used by the chip to be tested needs to be adjusted;

[0031] When a first deviation value between the first floating-point training result and the first training result is greater than a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is greater than a set second threshold, a verification result indicating the floating-point operation method used by the chip to be tested and the design file to be adjusted is generated.

[0032] Here, by setting a plurality of modes, the verification results of the chip to be tested are generated, and the verification results of the chip to be tested are more accurate and more diverse.

[0033] In a possible implementation manner, the using of training samples to respectively train the replaced neural network and the neural network to be verified to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network includes:

[0034] Initializing the replaced neural network and the neural network to be verified;

[0035] The training samples are used to train the replaced neural network and the neural network to be verified after initialization for multiple times, respectively, until the number of training times is equal to a set first number threshold, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0036] Here, by setting the first number threshold, the number of training times of the neural network can be flexibly controlled, and by reducing the first number threshold, the verification efficiency of the chip can be improved.

[0037] In a possible implementation manner, the using of training samples to respectively train the replaced neural network and the neural network to be verified to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network includes:

[0038] Initializing the neural network to be verified;

[0039] Using a first training sample in the training samples, the initialized neural network to be verified is trained multiple times to obtain a neural network to be verified including first network parameters; and determining the first network parameters as network parameters of the replaced neural network;

[0040] The neural network to be verified including the first network parameters and the replaced neural network including the first network parameters are trained multiple times by using the second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0041] In a possible implementation manner, the using of training samples to respectively train the replaced neural network and the neural network to be verified to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network includes:

[0042] Initializing the replaced neural network and the neural network to be verified;

[0043] Using the first training sample in the training samples, respectively train the replaced neural network and the neural network to be verified after the initialization processing for multiple times until the number of training times is equal to the set second number threshold, thereby obtaining the replaced neural network including the second network parameters and the neural network to be verified including the third network parameters;

[0044] The second training sample in the training samples is used to perform multiple training on the neural network to be verified including the second network parameters and the neural network to be verified including the third network parameters, respectively, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0045] The effects of the following devices, electronic devices, etc. are described in the description of the above method and will not be repeated here.

[0046] In a second aspect, the present disclosure provides a chip verification device, including:

[0047] An acquisition module, used to acquire the neural network to be verified;

[0048] A first generating module is used to generate a target operator consistent with the operator type of at least one original operator included in the neural network to be verified;

[0049] A second generation module replaces the corresponding original operator in the neural network to be verified with the target operator to generate a replaced neural network;

[0050] A training module, used to train the replaced neural network and the neural network to be verified respectively using training samples, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network;

[0051] The third generating module is used to generate a verification result corresponding to the chip to be tested based on the first training result and the second training result.

[0052] In a possible implementation manner, the first generating module, when generating a target operator consistent with the operator type of at least one original operator included in the neural network to be verified, is used to:

[0053] For at least one operator type of the original operator included in the neural network to be verified, generating an initial operator corresponding to the operator type;

[0054] Testing the initial operator to generate a test result corresponding to the initial operator;

[0055] When the test result indicates that the test fails, the initial operator is adjusted to obtain an adjusted initial operator; the adjusted initial operator is used as a new initial operator, and the process returns to the step of testing the initial operator to obtain a test result, until the test result of the adjusted initial operator indicates that the test passes;

[0056] When the test result indicates that the test is passed, the initial operator is determined as a target operator that is consistent with the operator type.

[0057] In a possible implementation manner, when testing the initial operator and generating the test result corresponding to the initial operator, the first generating module is used to:

[0058] Inputting the preset feature data into the initial operator to obtain first detection data corresponding to the initial operator; wherein the first detection data includes: predicted output data obtained when the preset feature data includes input feature data, and / or predicted gradient data obtained when the preset feature data includes input gradient data; and

[0059] Inputting the preset feature data into an original operator matching the initial operator to obtain second detection data corresponding to the original operator;

[0060] A test result corresponding to the initial operator is generated based on the first detection data and the second detection data.

[0061] In a possible implementation manner, the training module, when training the replaced neural network using the training sample to obtain a second training result corresponding to the replaced neural network, is used to:

[0062] On the selected training framework, using the training samples, and according to the first floating-point operation mode indicated by the training framework, the replaced neural network is trained to obtain a first floating-point training result; and / or,

[0063] On the selected training framework, using the training samples, and according to the second floating-point operation mode used by the chip to be tested, the replaced neural network is trained to obtain a second floating-point training result;

[0064] At least one of the first floating-point training result and the second floating-point training result is determined as a second training result corresponding to the replaced neural network.

[0065] In a possible implementation manner, when the third generating module generates the verification result corresponding to the chip to be tested based on the first training result and the second training result, it is configured to:

[0066] If a first deviation value between a first floating-point training result in the second training result and the first training result is less than or equal to a set first threshold, and a second deviation value between the first floating-point training result and a second floating-point training result in the second training result is less than or equal to a set second threshold, generating a verification result indicating that the chip to be tested has passed verification;

[0067] When a first deviation value between the first floating-point training result and the second training result is greater than a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is less than or equal to a set second threshold, generating a verification result indicating that the design file of the chip to be tested needs to be adjusted;

[0068] When a first deviation value between the first floating-point training result and the second training result is less than or equal to a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is greater than a set second threshold, generating a verification result indicating that a floating-point operation mode used by the chip to be tested needs to be adjusted;

[0069] When a first deviation value between the first floating-point training result and the first training result is greater than a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is greater than a set second threshold, a verification result indicating the floating-point operation method used by the chip to be tested and the design file to be adjusted is generated.

[0070] In a possible implementation manner, the training module, when using training samples to respectively train the replaced neural network and the neural network to be verified to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, is used to:

[0071] Initializing the replaced neural network and the neural network to be verified;

[0072] The training samples are used to train the replaced neural network and the neural network to be verified after initialization for multiple times, respectively, until the number of training times is equal to a set first number threshold, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0073] In a possible implementation manner, the training module, when using training samples to respectively train the replaced neural network and the neural network to be verified to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, is used to:

[0074] Initializing the neural network to be verified;

[0075] Using a first training sample in the training samples, the initialized neural network to be verified is trained multiple times to obtain a neural network to be verified including first network parameters; and determining the first network parameters as network parameters of the replaced neural network;

[0076] The neural network to be verified including the first network parameters and the replaced neural network including the first network parameters are trained multiple times by using the second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0077] In a possible implementation manner, the training module, when using training samples to respectively train the replaced neural network and the neural network to be verified to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, is used to:

[0078] Initializing the replaced neural network and the neural network to be verified;

[0079] Using the first training sample in the training samples, respectively train the replaced neural network and the neural network to be verified after the initialization processing for multiple times until the number of training times is equal to the set second number threshold, thereby obtaining the replaced neural network including the second network parameters and the neural network to be verified including the third network parameters;

[0080] The second training sample in the training samples is used to perform multiple training on the neural network to be verified including the second network parameters and the neural network to be verified including the third network parameters, respectively, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0081] In a third aspect, the present disclosure provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the chip verification method described in the first aspect or any one of the embodiments are performed.

[0082] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the chip verification method as described in the first aspect or any one of the embodiments are executed.

[0083] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0085] Figure 1 A schematic diagram showing a flow chart of a chip verification method provided by an embodiment of the present disclosure;

[0086] Figure 2 A schematic diagram of the architecture of a chip verification device provided by an embodiment of the present disclosure is shown;

[0087] Figure 3 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0089] Deep learning is a computationally intensive algorithm. As the diversity and complexity of processing tasks increase, the accuracy and real-time requirements of the algorithm continue to increase, making the scale of the neural network used in deep learning larger and larger, and more computing resources and storage resources are required to run the neural network. Therefore, the designed AI chip can be used to run the neural network. Generally, the designed AI chip needs to be evaluated before the AI ​​chip is taped out.

[0090] For example, the chip area, power consumption, computing power, accuracy and other indicators of the AI ​​chip can be evaluated. Since accuracy is the main indicator for measuring the performance of AI chips, it is particularly important to propose a method for evaluating the accuracy of AI chips. Based on this, the embodiments of the present disclosure provide a chip verification method, device, electronic device and storage medium.

[0091] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0092] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0093] To facilitate understanding of the embodiments of the present disclosure, a chip verification method disclosed in the embodiments of the present disclosure is first introduced in detail. The execution subject of the chip verification method provided in the embodiments of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a personal digital assistant (Personal Digital Assistant, PDA), etc. In some possible implementations, the chip verification method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0094] See also Figure 1 FIG. 1 is a flow chart of a chip verification method provided by an embodiment of the present disclosure, the method comprising S101-S105, wherein:

[0095] S101, obtaining a neural network to be verified.

[0096] S102, for an operator type of at least one original operator included in the neural network to be verified, generating a target operator consistent with the operator type;

[0097] S103, using the target operator to replace the corresponding original operator in the neural network to be verified, to generate a replaced neural network.

[0098] S104, using the training samples, respectively training the replaced neural network and the neural network to be verified, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0099] S105: Generate a verification result corresponding to the chip to be tested based on the first training result and the second training result.

[0100] In the above method, after obtaining the neural network to be verified, a target operator is generated that is consistent with the operator type of at least one original operator included in the neural network to be verified. Since the target data processing method corresponding to the target operator matches the preset data processing method corresponding to the operator type indicated by the design file of the chip to be verified, the simulation of floating-point operations in the chip to be verified is realized by using the target operator. Then, the target operator can be used to replace the corresponding original operator in the neural network to be verified to generate a replaced neural network. And using the training samples, the neural network to be verified and the replaced neural network are trained respectively to obtain the first training result and the second training result. Since the first training result can be determined as the benchmark result of the network to be verified, the second training result can be determined as the simulation result when the chip to be verified runs the neural network to be verified. Then, using the first training result and the second training result, the verification result of the chip to be verified can be generated more accurately, and the accuracy of the chip to be verified can be evaluated.

[0101] S101 to S105 are described in detail below.

[0102] For S101:

[0103] The neural network to be verified can be selected from multiple candidate neural networks that have been constructed according to the training task and / or network type. For example, the training tasks may include: object detection, semantic segmentation, image segmentation, key point detection, image classification, pose estimation, visual fashion analysis, object re-identification, etc.; the network types may include: deep convolutional neural network, self-attention network, recurrent neural network, etc.

[0104] Exemplarily, any Pytorch project can be selected from an open source project, wherein the Pytorch project includes a neural network built based on Pytorch; and then the neural network corresponding to the Pytorch project is determined as the neural network to be verified.

[0105] In order to reduce the training time of the neural network and improve the efficiency of chip verification, a neural network with a network parameter value less than a set parameter value threshold can be obtained from the multiple candidate neural networks constructed as the neural network to be verified. The parameter value threshold can be set according to actual needs.

[0106] In one implementation, a neural network to be verified may be obtained using a training task, a network type, and a network parameter amount. For example, multiple candidate neural networks of different network types included in the training task may be determined based on the training task. Each network type may include at least one candidate neural network. Then, a candidate neural network matching the determined network type may be selected from the multiple candidate neural networks. Finally, a neural network whose network parameter amount is less than a parameter amount threshold is selected from the candidate neural networks matching the determined network type as the neural network to be verified.

[0107] For S102:

[0108] After obtaining the neural network to be verified, the operator type of the original operator included in the neural network to be verified can be determined according to the network structure of the neural network to be verified. The original operator may include a convolution operator, a fully connected operator, a pooling operator, and the like. Then, for at least one operator type of the original operator, a target operator consistent with the operator type can be generated. Among them, the target data processing method corresponding to the target operator matches the preset data processing method corresponding to the operator type indicated by the design file of the chip to be verified. The preset data processing method can be: the processing method of the operation circuit that executes the operation process of the operator type on the chip to be verified.

[0109] Exemplarily, a target operator consistent with the operator type can be generated for each original operator included in the neural network to be verified, and a target operator consistent with the operator type can also be generated for some of the various original operators included in the neural network to be verified. For example, when it is necessary to verify the performance of the neural network to be verified on the chip to be verified, a corresponding target operator can be generated for each original operator type. When it is necessary to verify the performance of a local original operator in the neural network to be verified on the chip to be verified, a corresponding target operator can be generated for the local original operator.

[0110] In a possible implementation, the generating a target operator consistent with the operator type of at least one original operator included in the neural network to be verified includes:

[0111] S1021, for at least one operator type of original operator included in the neural network to be verified, generate an initial operator corresponding to the operator type.

[0112] S1022: Test the initial operator to generate a test result corresponding to the initial operator.

[0113] S1023, when the test result indicates that the test fails, the initial operator is adjusted to obtain an adjusted initial operator; the adjusted initial operator is used as a new initial operator, and the process returns to the step of testing the initial operator to obtain a test result, until the test result of the adjusted initial operator indicates that the test passes.

[0114] S1024: When the test result indicates that the test is passed, determine the initial operator as a target operator that is consistent with the operator type.

[0115] Here, for at least one operator type of the original operator, after generating an initial operator corresponding to the operator type, the initial operator can be tested. When the test fails, the initial operator can be adjusted until the test passes. The initial operator that passes the test is determined as the target operator consistent with the operator type, which ensures the performance of the generated target operator, so that when the replaced neural network is obtained based on the target operator later, the network performance deviation between the replaced neural network and the neural network to be verified is small. Based on the replaced neural network and the neural network to be verified with smaller deviations, the chip to be verified can be verified more accurately.

[0116] For at least one operator type of the original operator, an initial operator corresponding to the operator type is generated. For example, an initial operator corresponding to at least one operator type may be generated on a Compute Unified Device Architecture (CUDA), and the initial operator may be a developed CUDA operator.

[0117] After the initial operator is generated, the initial operator can be tested to generate a test result corresponding to the initial operator. For example, when the initial operator is a convolution operator, the initial operator can be used to perform a dot product operation on the preset data to generate an operation result. It is determined whether the operation result matches the true value corresponding to the preset data. If it matches, the test result corresponding to the initial operator is determined to be a test pass. If it does not match, the test result corresponding to the initial operator is determined to be a test fail.

[0118] In an optional implementation manner, the testing the initial operator and generating the test result corresponding to the initial operator may include:

[0119] Step A1, input the preset feature data into the initial operator to obtain the first detection data corresponding to the initial operator; wherein the first detection data includes: the predicted output data obtained when the preset feature data includes the input feature data, and / or the predicted gradient data obtained when the preset feature data includes the input gradient data.

[0120] Step A2: input the preset feature data into an original operator that matches the initial operator to obtain second detection data corresponding to the original operator.

[0121] Step A3: generating a test result corresponding to the initial operator based on the first detection data and the second detection data.

[0122] Acquire preset feature data, which may include input feature data and / or input gradient data. The input feature data may be a multi-channel feature map. The input gradient data may be gradient data determined during the back propagation of the neural network.

[0123] The preset characteristic data are respectively input into the initial operator and the original operator corresponding to the initial operator to obtain the first detection data corresponding to the initial operator and the second detection data corresponding to the original operator. Then the deviation between the first detection data and the second detection data is determined. When it is determined that the deviation is less than the set deviation threshold, the test result corresponding to the initial operator is determined to be a test pass; otherwise, the test result corresponding to the initial operator is determined to be a test fail.

[0124] Specifically, when the preset feature data includes input feature data, the input feature data is respectively input into the initial operator and the original operator to obtain the predicted output data corresponding to the initial operator and the predicted output data corresponding to the original operator. The output deviation between the predicted output data corresponding to the initial operator and the predicted output data corresponding to the original operator is determined, and when the output deviation is less than a first deviation threshold, the test result corresponding to the initial operator is determined to be a test pass; otherwise, the test result corresponding to the initial operator is determined to be a test fail.

[0125] When the preset feature data includes input gradient data, the input gradient data is respectively input into the initial operator and the original operator to obtain the predicted gradient data corresponding to the initial operator and the predicted gradient data corresponding to the original operator. The predicted gradient data may include the gradient value corresponding to the weight data and / or the gradient value corresponding to the input feature map. The gradient deviation between the predicted gradient data corresponding to the initial operator and the predicted gradient data corresponding to the original operator is determined. When the gradient deviation is less than the second deviation threshold, the test result corresponding to the initial operator is determined to be a passed test; otherwise, the test result corresponding to the initial operator is determined to be a failed test.

[0126] When the preset feature data includes input feature data and input gradient data, the first detection data and the second detection data include predicted output data and predicted gradient data. Determine the output deviation between the predicted output data corresponding to the initial operator and the predicted output data corresponding to the original operator, and the gradient deviation between the predicted gradient data corresponding to the initial operator and the predicted gradient data corresponding to the original operator. When the output deviation is less than the first deviation threshold, and the gradient deviation is less than the second deviation threshold, determine that the test result corresponding to the initial operator is a test pass; when the output deviation is greater than or equal to the first deviation threshold, or the gradient deviation is greater than or equal to the second deviation threshold, determine that the test result corresponding to the initial operator is a test fail.

[0127] In the above implementation, the preset feature data is input into the initial operator and the original operator respectively to obtain the first detection data corresponding to the initial operator and the second detection data corresponding to the original operator. The preset feature data may include input feature data and / or input gradient data, so as to test the forward calculation performance and backward calculation performance of the initial operator using the preset feature data.

[0128] After obtaining the test result corresponding to the initial operator, if the test result indicates that the test fails, the initial operator can be adjusted. For example, the calculation method, implementation logic, etc. of the initial operator can be adjusted to obtain an adjusted initial operator, and the adjusted initial operator can be used as the new initial operator, and the process returns to step S1022 until the test result of the adjusted initial operator indicates that the test passes.

[0129] If the test result indicates that the test passes, the initial operator is determined to be a target operator that is consistent with the operator type.

[0130] For S103:

[0131] The target operator is used to replace the corresponding original operator in the neural network to be verified to generate a replaced neural network. For example, the neural network to be verified includes original operator 1, original operator 2, and original operator 3. The target operator 1 corresponding to the original operator 1, the target operator 2 corresponding to the original operator 2, and the target operator 3 corresponding to the original operator 3 can be determined. After the original operators are replaced, the generated replaced neural network includes: target operator 1, target operator 2, and target operator 3.

[0132] For S104:

[0133] During implementation, a training sample is obtained from a data set corresponding to the neural network to be verified, and the replaced neural network and the neural network to be verified are trained using the training sample to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0134] In order to improve the efficiency of chip verification, the data set corresponding to the neural network to be verified can be screened to reduce the number of samples included in the data set. Then, the training samples included in the screened data set can be used to train the replaced neural network and the neural network to be verified.

[0135] The detection indicators included in the first training result and the second training result match. For example, the first training result and the second training result may include: loss value, loss value change trend graph, network precision, precision change trend graph, network accuracy, etc.

[0136] Exemplarily, the training samples may be used to train the replaced neural network and the neural network to be verified multiple times until the replaced neural network and the neural network to be verified converge, thereby obtaining a first training result and a second training result.

[0137] Considering that in the process of training the replaced neural network, the floating point operation method corresponding to the chip to be tested can be used, or the floating point operation method corresponding to the training framework can be used. Therefore, when using the training sample to train the replaced neural network and obtaining the second training result corresponding to the replaced neural network, it can include:

[0138] S1041, on the selected training framework, using the training samples, and according to the first floating-point operation method indicated by the training framework, training the replaced neural network to obtain a first floating-point training result.

[0139] S1042, using the training samples on the selected training framework and in accordance with the second floating-point operation method used by the chip to be tested, training the replaced neural network to obtain a second floating-point training result.

[0140] S1043: Determine at least one of the first floating-point training result and the second floating-point training result as the second training result corresponding to the replaced neural network.

[0141] The training framework may be selected from a plurality of training frameworks that have been built. For example, the training framework may be a training framework based on a graphics processing unit (GPU), etc. The training framework indicates a first floating-point operation method used in the training process.

[0142] In one method, the replaced neural network is trained on the selected training framework using the training samples and in accordance with the first floating-point operation mode indicated by the training framework to obtain a first floating-point training result. That is, in the process of training the replaced neural network, the target operator in the replaced neural network processes the input data using the first floating-point operation mode.

[0143] In another way, the replaced neural network is trained on the selected training framework using the training samples according to the second floating-point operation method used by the chip to be tested to obtain a second floating-point training result. That is, in the process of training the replaced neural network, the target operator in the replaced neural network processes the input data using the second floating-point operation method.

[0144] Exemplarily, the first floating-point operation method may be, for example, performing floating-point operations according to the channel dimension of the feature map, and the second floating-point operation method may be, for example, first dividing the feature map into multiple feature blocks according to length and width, and then performing floating-point operations on each feature block; the first floating-point operation method may also be set as a basic operation, and the second floating-point operation method may be set as a personalized operation, which is not limited in the present disclosure.

[0145] At least one of the first floating-point training result and the second floating-point training result may be determined as the second training result corresponding to the replaced neural network.

[0146] In the above implementation, the replaced neural network can be trained according to the first floating-point operation method on the selected training framework, and / or the replaced neural network can be trained according to the second floating-point operation method. The training method is more flexible, which improves the flexibility and diversity of the second training result, so that the first training result and the second training result can be used to more accurately verify the chip to be tested.

[0147] In specific implementation, at least one of the following multiple training schemes can be used to train the replaced neural network and the neural network to be verified, so as to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0148] Training scheme 1: using training samples to train the replaced neural network and the neural network to be verified respectively, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, may include:

[0149] Step B1, initializing the replaced neural network and the neural network to be verified.

[0150] Step B2: using the training samples, respectively, to train the replaced neural network and the neural network to be verified after initialization for multiple times, until the number of training times is equal to the set first number threshold, and obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0151] Here, the replaced neural network and the neural network to be verified may be initialized to ensure that the initialization parameters of the replaced neural network and the neural network to be verified are consistent.

[0152] Then, the training samples are used to train the replaced neural network and the neural network to be verified after initialization for multiple times, respectively, until the number of training times is equal to the set first number threshold, to obtain the first training result and the second training result. The first number threshold can be set as needed.

[0153] During implementation, the preset n epochs (indicating that all samples in the training set are used for training once) can be used as training samples, and the replaced neural network and the neural network to be verified can be fully trained using the training samples to obtain the first training result corresponding to the neural network to be verified and the second training result corresponding to the replaced neural network. This training method can make the replaced neural network and the neural network to be verified have the highest accuracy after training, but the training process is time-consuming and the chip verification efficiency is low.

[0154] Alternatively, the number of training iteration epochs can be reduced, that is, some epochs are selected from n epochs to train the replaced neural network and the neural network to be verified, and the first training result corresponding to the neural network to be verified and the second training result corresponding to the replaced neural network after the first numerical threshold is trained are obtained. The first training result and the second training result are subsequently used to determine the verification result of the chip to be verified. Under this training method, the accuracy of the replaced neural network and the neural network to be verified after the training is completed cannot reach the highest accuracy, but the training process is time-saving and the chip verification efficiency is high.

[0155] Here, by setting the first number threshold, the number of training times of the neural network can be flexibly controlled, and by reducing the first number threshold, the verification efficiency of the chip can be improved.

[0156] Training scheme 2: using training samples to train the replaced neural network and the neural network to be verified respectively, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, may include:

[0157] Step C1, initializing the neural network to be verified.

[0158] Step C2: using the first training sample in the training samples, train the initialized neural network to be verified multiple times to obtain the neural network to be verified including the first network parameters, and determine the first network parameters as the network parameters of the replaced neural network.

[0159] Step C3, using the second training sample in the training samples, respectively, to train the neural network to be verified including the first network parameters and the replaced neural network including the first network parameters for multiple times, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0160] Considering that when the number of training times is fixed, the training time of the neural network to be verified is shorter than the training time of the replaced neural network. Therefore, after the neural network to be verified is initialized, the neural network to be verified can be trained first to obtain the neural network to be verified including the first network parameters. Then, the first network parameters are used as the network parameters of the replaced neural network, and the replaced neural network is trained until the replaced neural network converges, which can reduce the time taken for the replaced neural network to reach the highest accuracy and improve the training efficiency of the replaced neural network.

[0161] In a specific implementation, the first training sample in the training samples is first used to train the initialized neural network to be verified for multiple times until the pre-training cutoff condition is met, and a first model file corresponding to the neural network to be verified is obtained. The first model file indicates that there is a neural network to be verified containing the first network parameters. The pre-training cutoff condition can be set according to actual needs.

[0162] Exemplarily, the pre-training cutoff condition may include at least one of the following: 1. The set number of training times, for example, after training 100 epochs, it is determined that the pre-training cutoff condition is met. 2. The loss value of the neural network to be verified after training is less than a first preset value. 3. The accuracy of the neural network to be verified after training is greater than a second preset value, etc.

[0163] During implementation, the target operator is used to replace the initial operator in the neural network to be verified that contains the first network parameters indicated by the first model file, and the replaced neural network contains the first network parameters. Then, the second training sample in the training sample can be used to train the neural network to be verified that contains the first network parameters and the replaced neural network that contains the first network parameters for multiple times until the neural network to be verified and the replaced neural network converge, or until the number of training times is equal to a preset number of times, and the first training result corresponding to the neural network to be verified and the second training result corresponding to the replaced neural network are obtained. Among them, the first training sample and the second training sample can be at least part of the samples in the training sample.

[0164] In the above implementation, after obtaining the neural network to be verified including the first network parameters, the first network parameters can be used as the network parameters of the replaced neural network, and then the replaced neural network including the first network parameters can be trained multiple times using the training samples to obtain the second training result corresponding to the replaced neural network. In the above manner, when the number of training times is fixed, the accuracy of the second training result can be improved. When the accuracy is fixed, the time consumed to obtain the second training result can be reduced.

[0165] Training scheme three: using training samples to train the replaced neural network and the neural network to be verified respectively, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, may include:

[0166] Step D1, initializing the replaced neural network and the neural network to be verified.

[0167] Step D2, using the first training sample in the training samples, respectively train the replaced neural network and the neural network to be verified after initialization for multiple times until the number of training times is equal to the set second number threshold, to obtain the replaced neural network containing the second network parameters, and the neural network to be verified containing the third network parameters.

[0168] Step D3, using the second training sample in the training samples, respectively, to train the neural network to be verified including the second network parameters and the neural network to be verified including the third network parameters for multiple times, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0169] During implementation, the replaced neural network and the neural network to be verified can be initialized to ensure that the network parameters of the replaced neural network and the neural network to be verified after the initialization processing are consistent. Then, the first training sample in the training samples is used to train the replaced neural network and the neural network to be verified after the initialization processing for multiple times, respectively, until the number of training times is equal to the set second number threshold, to obtain the replaced neural network containing the second network parameters and the neural network to be verified containing the third network parameters.

[0170] Considering that when the number of training times is fixed, the training time of the neural network to be verified is shorter than the training time of the replaced neural network. Therefore, a neural network to be verified including the second network parameters and a neural network to be verified including the third network parameters can be generated. Using the second training sample in the training sample, the neural network to be verified including the second network parameters and the neural network to be verified including the third network parameters are trained multiple times respectively until the neural network to be verified and the replaced neural network converge, or until the number of training times is equal to the preset number of times, and the first training result corresponding to the neural network to be verified and the second training result corresponding to the replaced neural network are obtained. Among them, the second training result corresponding to the replaced neural network is obtained by training the neural network to be verified including the second network parameters.

[0171] Since the second network parameters and the third network parameters are numerically different, there will be a deviation between the first training result and the second training result obtained in the subsequent training process. For example, the precision included in the first training result and the second training result is different. Therefore, by comparing the precision of the first training result and the precision of the second training result, a verification result corresponding to the chip to be tested can be generated.

[0172] For S105:

[0173] During implementation, the verification result corresponding to the chip to be tested can be generated based on the first training result and the second training result. For example, when the first training result and the second training result include accuracy, the difference between the accuracy in the first training result and the accuracy in the second training result can be determined. When the difference is less than or equal to the deviation threshold, the verification result corresponding to the chip to be tested can be determined as follows: the accuracy performance of the chip to be tested is good. When the difference is greater than the deviation threshold, the verification result corresponding to the chip to be tested can be determined as follows: the accuracy performance of the chip to be tested is poor.

[0174] In the case where the second training result includes the first floating-point training result and the second floating-point training result, generating a verification result corresponding to the chip to be tested based on the first training result and the second training result may include any one of the following methods:

[0175] Method 1: When the first deviation value between the first floating-point training result and the first training result in the second training result is less than or equal to the set first threshold, and the second deviation value between the first floating-point training result and the second floating-point training result in the second training result is less than or equal to the set second threshold, a verification result indicating that the chip to be tested has passed the verification is generated.

[0176] Here, when the first deviation value between the first floating-point training result and the first training result is less than or equal to the set first threshold, and the second deviation value between the first floating-point training result and the second floating-point training result is less than or equal to the set second threshold, it is determined that the floating-point operation method (such as the second floating-point operation method) used by the chip to be tested is better, and the corresponding performance of the chip to be tested is better, then the generated verification result can be: the chip to be tested has passed the verification.

[0177] Method 2: When the first deviation value between the first floating-point training result and the second training result is greater than the set first threshold, and the second deviation value between the first floating-point training result and the second floating-point training result is less than or equal to the set second threshold, a verification result indicating that the design file of the chip to be tested needs to be adjusted is generated.

[0178] Here, when the first deviation value between the first floating-point training result and the second training result is greater than the first threshold, it is determined that the corresponding performance of the chip to be tested is poor; when the second deviation value between the first floating-point training result and the second floating-point training result is less than or equal to the set second threshold, it is determined that the floating-point operation method (such as the second floating-point operation method) used by the chip to be tested is better, and the generated verification result can be: the design file of the chip to be tested needs to be adjusted.

[0179] Method three, when the first deviation value between the first floating-point training result and the second training result is less than or equal to the set first threshold, and the second deviation value between the first floating-point training result and the second floating-point training result is greater than the set second threshold, a verification result is generated indicating that the floating-point operation method used by the chip to be tested needs to be adjusted.

[0180] Here, when the first deviation value between the first floating-point training result and the second training result is less than or equal to the first threshold, it is determined that the corresponding performance of the chip to be tested is better; when the second deviation value between the first floating-point training result and the second floating-point training result is greater than the second threshold, it is determined that the floating-point operation method (such as the second floating-point operation method) used by the chip to be tested is poor, and the generated verification result can be: the floating-point operation method used by the chip to be tested needs to be adjusted.

[0181] Method 4: When the first deviation value between the first floating-point training result and the first training result is greater than the set first threshold, and the second deviation value between the first floating-point training result and the second floating-point training result is greater than the set second threshold, a verification result indicating the floating-point operation method used by the chip to be tested and the design file to be adjusted is generated.

[0182] Here, when the first deviation value between the first floating-point training result and the first training result is greater than the first threshold, and the second deviation value between the first floating-point training result and the second floating-point training result is greater than the set second threshold, it is determined that the floating-point operation method used by the chip to be tested is poor, and the corresponding performance of the chip to be tested is poor, then the generated verification result can be: the floating-point operation method and design file used by the chip to be tested need to be adjusted.

[0183] Here, by setting a plurality of modes, the verification results of the chip to be tested are generated, and the verification results of the chip to be tested are more accurate and more diverse.

[0184] In specific implementation, in the process of verifying the chip to be tested, various training details of the training process of the neural network to be tested and various training details of the training process of the replaced neural network can also be recorded. For example, the training details can include: training project introduction, operator list, training method, training process, etc. So that after the chip to be tested is taped out, the chip obtained by taping out can be tested according to the recorded training details to obtain the test results, and then the test results can be compared with the first training results and the second training results to determine the problems existing in the chip and the performance it has.

[0185] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0186] Based on the same concept, the present disclosure also provides a chip verification device, see Figure 2 As shown, it is a schematic diagram of the architecture of the chip verification device provided by the embodiment of the present disclosure, including an acquisition module 201, a first generation module 202, a second generation module 203, a training module 204, and a third generation module 205. Specifically:

[0187] An acquisition module 201 is used to acquire a neural network to be verified;

[0188] A first generating module 202 is used to generate a target operator consistent with the operator type of at least one original operator included in the neural network to be verified;

[0189] A second generating module 203 is used to replace the corresponding original operator in the neural network to be verified with the target operator to generate a replaced neural network;

[0190] A training module 204 is used to train the replaced neural network and the neural network to be verified respectively using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network;

[0191] The third generating module 205 is used to generate a verification result corresponding to the chip to be tested based on the first training result and the second training result.

[0192] In a possible implementation manner, the first generating module 202, when generating a target operator consistent with the operator type of at least one original operator included in the neural network to be verified, is configured to:

[0193] For at least one operator type of the original operator included in the neural network to be verified, generating an initial operator corresponding to the operator type;

[0194] Testing the initial operator to generate a test result corresponding to the initial operator;

[0195] When the test result indicates that the test fails, the initial operator is adjusted to obtain an adjusted initial operator; the adjusted initial operator is used as a new initial operator, and the process returns to the step of testing the initial operator to obtain a test result, until the test result of the adjusted initial operator indicates that the test passes;

[0196] When the test result indicates that the test is passed, the initial operator is determined as a target operator that is consistent with the operator type.

[0197] In a possible implementation manner, when testing the initial operator and generating the test result corresponding to the initial operator, the first generating module 202 is used to:

[0198] Inputting the preset feature data into the initial operator to obtain first detection data corresponding to the initial operator; wherein the first detection data includes: predicted output data obtained when the preset feature data includes input feature data, and / or predicted gradient data obtained when the preset feature data includes input gradient data; and

[0199] Inputting the preset feature data into an original operator matching the initial operator to obtain second detection data corresponding to the original operator;

[0200] A test result corresponding to the initial operator is generated based on the first detection data and the second detection data.

[0201] In a possible implementation manner, the training module 204, when training the replaced neural network using the training sample to obtain a second training result corresponding to the replaced neural network, is used to:

[0202] On the selected training framework, using the training samples, and according to the first floating-point operation mode indicated by the training framework, the replaced neural network is trained to obtain a first floating-point training result; and / or,

[0203] On the selected training framework, using the training samples, and according to the second floating-point operation mode used by the chip to be tested, the replaced neural network is trained to obtain a second floating-point training result;

[0204] At least one of the first floating-point training result and the second floating-point training result is determined as a second training result corresponding to the replaced neural network.

[0205] In a possible implementation manner, the third generating module 205, when generating the verification result corresponding to the chip to be tested based on the first training result and the second training result, is used to:

[0206] If a first deviation value between a first floating-point training result in the second training result and the first training result is less than or equal to a set first threshold, and a second deviation value between the first floating-point training result and a second floating-point training result in the second training result is less than or equal to a set second threshold, generating a verification result indicating that the chip to be tested has passed verification;

[0207] When a first deviation value between the first floating-point training result and the second training result is greater than a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is less than or equal to a set second threshold, generating a verification result indicating that the design file of the chip to be tested needs to be adjusted;

[0208] When a first deviation value between the first floating-point training result and the second training result is less than or equal to a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is greater than a set second threshold, generating a verification result indicating that a floating-point operation mode used by the chip to be tested needs to be adjusted;

[0209] When a first deviation value between the first floating-point training result and the first training result is greater than a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is greater than a set second threshold, a verification result indicating the floating-point operation method used by the chip to be tested and the design file to be adjusted is generated.

[0210] In a possible implementation manner, the training module 204, when respectively training the replaced neural network and the neural network to be verified using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, is used to:

[0211] Initializing the replaced neural network and the neural network to be verified;

[0212] The training samples are used to train the replaced neural network and the neural network to be verified after initialization for multiple times, respectively, until the number of training times is equal to a set first number threshold, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0213] In a possible implementation manner, the training module 204, when respectively training the replaced neural network and the neural network to be verified using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, is used to:

[0214] Initializing the neural network to be verified;

[0215] Using a first training sample in the training samples, the initialized neural network to be verified is trained multiple times to obtain a neural network to be verified including first network parameters; and determining the first network parameters as network parameters of the replaced neural network;

[0216] The neural network to be verified including the first network parameters and the replaced neural network including the first network parameters are trained multiple times by using the second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0217] In a possible implementation manner, the training module 204, when respectively training the replaced neural network and the neural network to be verified using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, is used to:

[0218] Initializing the replaced neural network and the neural network to be verified;

[0219] Using the first training sample in the training samples, respectively train the replaced neural network and the neural network to be verified after the initialization processing for multiple times until the number of training times is equal to the set second number threshold, thereby obtaining the replaced neural network including the second network parameters and the neural network to be verified including the third network parameters;

[0220] The second training sample in the training samples is used to perform multiple training on the neural network to be verified including the second network parameters and the neural network to be verified including the third network parameters, respectively, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

[0221] In some embodiments, the functions or templates contained in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0222] Based on the same technical concept, the embodiment of the present disclosure also provides an electronic device. Figure 3 As shown, it is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, including a processor 301, a memory 302, and a bus 303. Among them, the memory 302 is used to store execution instructions, including a memory 3021 and an external memory 3022; the memory 3021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 301, and the data exchanged with the external memory 3022 such as a hard disk. The processor 301 exchanges data with the external memory 3022 through the memory 3021. When the electronic device 300 is running, the processor 301 communicates with the memory 302 through the bus 303, so that the processor 301 executes the following instructions:

[0223] Obtain the neural network to be verified;

[0224] For an operator type of at least one original operator included in the neural network to be verified, generating a target operator consistent with the operator type;

[0225] Using the target operator to replace the corresponding original operator in the neural network to be verified, to generate a replaced neural network;

[0226] Using the training samples, respectively training the replaced neural network and the neural network to be verified to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network;

[0227] Based on the first training result and the second training result, a verification result corresponding to the chip to be tested is generated.

[0228] In addition, the embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the chip verification method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0229] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the chip verification method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0230] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0231] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0232] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0233] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0234] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0235] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A chip verification method, characterized in that: include: Obtain the neural network to be verified; For at least one operator type of a primitive operator included in the neural network to be verified, generating an initial operator corresponding to the operator type; Testing the initial operator to generate a test result corresponding to the initial operator; if the test result indicates that the test fails, adjusting the initial operator to obtain an adjusted initial operator; The adjusted initial operator is used as a new initial operator, and the process returns to the step of testing the initial operator to obtain a test result, until the test result of the adjusted initial operator indicates that the test has passed; if the test result indicates that the test has passed, the initial operator is determined to be a target operator consistent with the operator type; the target data processing mode corresponding to the target operator matches the preset data processing mode corresponding to the operator type indicated in the design file of the chip to be tested; the preset data processing mode is the processing mode of the operation circuit that performs the operation process of the operator type on the chip to be tested; Replacing the corresponding original operator in the neural network to be verified with the target operator to generate a replaced neural network; Using the training samples, respectively training the replaced neural network and the neural network to be verified to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network; Based on the first training result and the second training result, a verification result corresponding to the chip to be tested is generated.

2. The method according to claim 1, characterized in that The testing of the initial operator to generate a test result corresponding to the initial operator includes: Inputting preset feature data into the initial operator to obtain first detection data corresponding to the initial operator; wherein the first detection data includes: predicted output data obtained when the preset feature data includes input feature data, and / or predicted gradient data obtained when the preset feature data includes input gradient data; and Inputting the preset feature data into an original operator that matches the initial operator to obtain second detection data corresponding to the original operator; A test result corresponding to the initial operator is generated based on the first detection data and the second detection data.

3. The method according to claim 1 or 2, characterized in that The step of training the replaced neural network using the training sample to obtain a second training result corresponding to the replaced neural network includes: On the selected training framework, using the training samples, and according to the first floating-point operation mode indicated by the training framework, the replaced neural network is trained to obtain a first floating-point training result; and / or, On the selected training framework, using the training samples, and according to the second floating-point operation mode used by the chip to be tested, the replaced neural network is trained to obtain a second floating-point training result; At least one of the first floating-point training result and the second floating-point training result is determined as the second training result corresponding to the replaced neural network.

4. The method according to any one of claims 1 to 3, characterized in that: Generating a verification result corresponding to the chip to be tested based on the first training result and the second training result includes: If a first deviation value between a first floating-point training result in the second training result and the first training result is less than or equal to a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result in the second training result is less than or equal to a set second threshold, generating a verification result indicating that the chip to be tested has passed verification; If a first deviation value between the first floating-point training result and the second training result is greater than a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is less than or equal to a set second threshold, generating a verification result indicating that the design file of the chip to be tested needs to be adjusted; If a first deviation value between the first floating-point training result and the second training result is less than or equal to a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is greater than a set second threshold, generating a verification result indicating that a floating-point operation mode used by the chip to be tested needs to be adjusted; When a first deviation value between the first floating-point training result and the second training result is greater than a set first threshold, and a second deviation value between the first floating-point training result and the second floating-point training result is greater than a set second threshold, a verification result indicating the floating-point operation method used by the chip to be tested and the design file to be adjusted is generated.

5. The method according to any one of claims 1 to 4, characterized in that: The step of training the replaced neural network and the neural network to be verified respectively using the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network includes: Initializing the replaced neural network and the neural network to be verified; Using the training samples, the replaced neural network and the neural network to be verified after initialization are trained multiple times respectively until the number of training times is equal to a set first threshold, thereby obtaining a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

6. The method according to any one of claims 1 to 4, characterized in that: The step of training the replaced neural network and the neural network to be verified respectively using the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network includes: Initializing the neural network to be verified; Using a first training sample from the training samples, the initialized neural network to be verified is trained multiple times to obtain a neural network to be verified including first network parameters; and determining the first network parameters as network parameters of the replaced neural network; Using the second training sample in the training samples, the neural network to be verified including the first network parameters and the replaced neural network including the first network parameters are trained multiple times respectively to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

7. The method according to any one of claims 1 to 4, characterized in that: The step of training the replaced neural network and the neural network to be verified respectively using the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network includes: Initializing the replaced neural network and the neural network to be verified; Using the first training sample among the training samples, the replaced neural network and the neural network to be verified after the initialization processing are trained multiple times, respectively, until the number of training times is equal to a set second number threshold, thereby obtaining the replaced neural network including the second network parameters and the neural network to be verified including the third network parameters; Using the second training sample in the training samples, the neural network to be verified containing the second network parameters and the neural network to be verified containing the third network parameters are trained multiple times respectively to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.

8. A chip verification device, characterized in that: include: An acquisition module, used to obtain the neural network to be verified; A first generating module is configured to generate an initial operator corresponding to at least one operator type of the original operator included in the neural network to be verified; Testing the initial operator to generate a test result corresponding to the initial operator; if the test result indicates that the test fails, adjusting the initial operator to obtain an adjusted initial operator; The adjusted initial operator is used as a new initial operator, and the process returns to the step of testing the initial operator to obtain a test result, until the test result of the adjusted initial operator indicates that the test has passed; if the test result indicates that the test has passed, the initial operator is determined to be a target operator consistent with the operator type; the target data processing mode corresponding to the target operator matches the preset data processing mode corresponding to the operator type indicated in the design file of the chip to be tested; the preset data processing mode is the processing mode of the operation circuit that performs the operation process of the operator type on the chip to be tested; A second generating module is configured to replace the corresponding original operator in the neural network to be verified with the target operator to generate a replaced neural network; A training module, configured to train the replaced neural network and the neural network to be verified respectively using training samples, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network; The third generating module is configured to generate a verification result corresponding to the chip to be tested based on the first training result and the second training result.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the chip verification method according to any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the chip verification method according to any one of claims 1 to 7 are executed.

Images