A chip verification result checking method, device, equipment and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG YUNHAI GUOCHUANG CLOUD COMPUTING EQUIP IND INNOVATION CENT CO LTD
- Filing Date
- 2023-04-14
- Publication Date
- 2026-06-19
Smart Images
Figure CN116450516B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of chip verification technology, and in particular to a chip verification result verification method, apparatus, device, and storage medium. Background Technology
[0002] The purpose of chip verification is to check for bugs in the Design Under Test (DUT). Chip verification first requires building a verification platform, which mainly consists of components such as a Driver, Monitor, Sequencer, Reference Model, and Scoreboard. After the verification platform is built, during chip verification simulation, verification stimuli are input into two paths: one for the verification platform and the other for the DUT. In the verification platform, the stimuli flow through the Reference Model, and the execution result is input into the Scoreboard. In the Scoreboard, the verification stimuli are input into the DUT, and the execution result is also input into the Scoreboard. The Scoreboard then compares the execution results from both paths, and the comparison results are used to check and analyze whether bugs exist in the DUT.
[0003] Generally, if the execution results of the reference model and the design under test (DUT) are correctly compared, the DUT is considered to be free of vulnerabilities. However, there are cases where the comparison is correct, but both the reference model and the DUT produce incorrect execution results. Furthermore, the algorithm module of the DUT contains complex algorithmic calculations, further increasing the possibility of errors.
[0004] In summary, improving the reliability of chip verification and more effectively discovering vulnerabilities in the design under test are problems that need to be solved. Summary of the Invention
[0005] In view of this, the purpose of this invention is to provide a chip verification result verification method, apparatus, device, and storage medium, which can improve the reliability of chip verification and more effectively discover vulnerabilities in the design under test. The specific solution is as follows:
[0006] In a first aspect, this application discloses a chip verification result verification method, applied to a preset chip verification platform, comprising:
[0007] Obtain test cases used to verify the target functionality of the design under test during chip verification;
[0008] Determine the verification stimulus from the test cases;
[0009] Based on the verification stimulus, a model training sample library is determined, and the preset neural network is trained using the samples in the model training sample library to obtain the target neural network model.
[0010] The verification stimulus is input into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test;
[0011] The first execution result is compared with the second execution result, and the correct results with the same result are verified by comparing the third execution result.
[0012] Optionally, determining the model training sample library based on the verification stimulus includes:
[0013] A model training sample library is generated based on the verification stimulus and the random constraints of the data in the verification stimulus.
[0014] Optionally, before training the preset neural network using samples from the model training sample library to obtain the neural network model, the method further includes:
[0015] Each sample in the training sample library of the model is labeled with a corresponding tag to obtain sample labels;
[0016] Accordingly, the step of training a preset neural network using samples from the model training sample library to obtain a target neural network model includes:
[0017] The samples carrying the sample labels in the model training sample library are input into the preset neural network to obtain the prediction output corresponding to the sample;
[0018] Based on the predicted output and the sample labels, the prediction error of the target neural network model during the training process is determined. Based on the prediction error, the preset neural network is trained using a neural network optimization algorithm. When the neural network model converges, the target neural network model is determined.
[0019] Optionally, the step of training a preset neural network using samples from the model training sample library to obtain a target neural network model includes:
[0020] The target neural network model is obtained by training any one of the fully connected neural networks, convolutional neural networks, and recurrent neural networks using samples from the model training sample library.
[0021] Optionally, the step of inputting the verification stimulus into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result includes:
[0022] The verification stimulus is input into the design under test to obtain a first execution result;
[0023] The verification stimulus is input into the reference model to obtain the second execution result;
[0024] The verification stimulus is converted to obtain a data format that the target neural network model can process. Then, the verification stimulus converted to the data format is input into the target neural network model to obtain a third execution result.
[0025] Optionally, the step of inputting the verification stimulus into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result includes:
[0026] Based on the parallel processing mechanism, the verification stimulus is input into the design under test, the reference model and the target neural network model respectively to obtain the corresponding first execution result, second execution result and third execution result.
[0027] Optionally, comparing the first execution result with the second execution result and using the third execution result to verify correct results that have the same result includes:
[0028] If the third execution result is different from the correct result, and the third execution result is greater than a preset threshold, then it is determined that there is a vulnerability in the design under test.
[0029] If the third execution result is different from the correct result, and the third execution result is not greater than the preset threshold, then it is determined that the vulnerability does not exist in the design under test.
[0030] If the third execution result is the same as the correct result, then it is determined that the vulnerability does not exist in the design under test.
[0031] Secondly, this application discloses a chip verification result verification device, applied to a preset chip verification platform, comprising:
[0032] The test case acquisition module is used to acquire test cases for verifying the target function of the design under test during chip verification.
[0033] The verification stimulus determination module is used to determine the verification stimulus from the test cases;
[0034] The neural network model training module is used to determine the model training sample library based on the verification stimulus, and to train the preset neural network using the samples in the model training sample library to obtain the target neural network model.
[0035] The execution result acquisition module is used to input the verification stimulus into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test;
[0036] The result verification module is used to compare the first execution result with the second execution result, and to verify the correct results that have the same result by comparing the third execution result.
[0037] Thirdly, this application discloses an electronic device, which includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the chip verification result verification method as described above.
[0038] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the chip verification result verification method as described above.
[0039] This application provides a chip verification result verification method applied to a preset chip verification platform, comprising: acquiring test cases used to verify the target function of the design under test during chip verification; determining verification stimuli from the test cases; determining a model training sample library based on the verification stimuli, and training a preset neural network using samples in the model training sample library to obtain a target neural network model; inputting the verification stimuli into the design under test, a reference model, and the target neural network model respectively to obtain corresponding first execution results, second execution results, and third execution results; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test; comparing the first execution result with the second execution result, and using the third execution result to verify correct results with the same result. It can be seen that, from the perspective of chip verification, a trained target neural network model is added to a traditional chip verification platform. This neural network model, through self-learning, has the same function as the design under test and the reference model, thus forming a new chip verification platform composed of the design under test, the reference model, and the target neural network model. Compared to traditional chip verification methods that input verification stimuli into two paths, this method adds a third path, simultaneously inputting the verification stimuli into the neural network model. The execution results of the neural network model are used to further verify the comparison results between the reference model and the design under test (DUT) during simulation, improving the reliability of chip verification and more effectively identifying potential but potentially overlooked vulnerabilities in the DUT. Furthermore, this neural network model operates independently of the pre-defined chip verification platform, without compilation or simulation, and can be executed in parallel with the chip verification simulation between the DUT and the reference model, without adding compilation and simulation time, thus saving valuable chip verification time.
[0040] In addition, the chip verification result verification device, equipment and storage medium provided in this application correspond to the above-mentioned chip verification result verification method and have the same effect. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0042] Figure 1 This is a flowchart of a chip verification result verification method disclosed in this application;
[0043] Figure 2 This is a schematic diagram of a pre-defined chip verification platform disclosed in this application;
[0044] Figure 3 This is a flowchart of a specific chip verification result verification method disclosed in this application;
[0045] Figure 4 This is a schematic diagram of a fully connected neural network model disclosed in this application;
[0046] Figure 5 This is a schematic diagram of the structure of a chip verification result verification device disclosed in this application;
[0047] Figure 6 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0049] Currently, during chip verification, there are instances where the reference model and the design under test (DUT) match correctly, but the execution results are still incorrect. Furthermore, the algorithm module within the DUT contains complex algorithmic calculations, further increasing the possibility of errors.
[0050] Therefore, this application provides a chip verification result verification scheme, which can improve the reliability of verification during chip verification and more effectively discover vulnerabilities in the design under test.
[0051] This invention discloses a method for verifying chip verification results. See [link to relevant documentation]. Figure 1 As shown, applied to a preset chip verification platform, the method includes:
[0052] Step S11: Obtain test cases used to verify the target function of the design under test during chip verification.
[0053] In this embodiment of the application, in the entire chip verification process, the first step is to extract the functional points of the design under test to be verified based on the design and architecture documents of the DUT. Each functional point will have a corresponding test case.
[0054] Step S12: Determine the verification stimulus from the test cases.
[0055] During chip simulation and verification, for each test case, there will be multiple verification stimuli as inputs, and one or more execution results will be obtained.
[0056] Step S13: Determine the model training sample library based on the verification stimulus, and use the samples in the model training sample library to train the preset neural network to obtain the target neural network model.
[0057] In this embodiment, a model training sample library for training a preset neural network is generated using verification stimuli determined from test cases. Based on the current chip verification scenario and the characteristics of the samples in the training sample library, a neural network model is constructed. Samples from the sample library are input into the neural network model, allowing the model to learn from the samples. In this way, through self-learning, the neural network model acquires the same functionality as the design-under-test and reference models in traditional verification platforms.
[0058] It should be noted that neural network models exist in two states: an untrained neural network model and a trained neural network model. Integrating the trained neural network model into a chip verification platform forms a pre-defined chip verification platform. In contrast to traditional chip verification methods that input verification stimuli into two paths, this adds a third path, simultaneously inputting the verification stimuli into the target neural network model.
[0059] When constructing the target neural network model, the preset neural network for training can be selected from various neural network models, including fully connected neural networks, convolutional neural networks, and recurrent neural networks. Because neural networks possess powerful self-learning and data fitting capabilities, in this embodiment, a model training sample library determined based on validation stimuli is used to train the preset neural network, enabling the neural network model to learn the functional characteristics of the design under test and achieve performance close to that of the design under test.
[0060] Step S14: Input the verification stimulus into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test.
[0061] During chip simulation verification, verification stimuli are input into both the chip verification platform and the design under test (DUT). Within the chip verification platform, these stimuli are input into a reference model. Then, the execution results of the DUT are compared with the execution results of the reference model on the chip verification platform's scoring board. It should be noted that the reference model is a chip verification platform component with the same functionality, constructed by chip verification personnel based on the principles of the DUT.
[0062] Typically, the presence of vulnerabilities in the design under test (DUT) is analyzed based on the presence of errors in the execution results from the scoreboard. However, there are situations where the scoreboard's execution results are correct, but the DUT's execution results differ from the reference model's. Therefore, it is necessary to check the correctness of the execution results obtained after comparison from the scoreboard.
[0063] In this embodiment, during chip verification simulation, verification stimuli are simultaneously input into the target neural network model obtained after training. Because the input formats of the verification stimuli and the neural network model are different, the verification stimuli must first be converted into the input format of the neural network model before being input into the neural network model. Specifically, the verification stimuli undergo conversion processing to obtain a data format that the target neural network model can process. Then, the verification stimuli converted to the data format are input into the target neural network model to obtain a third execution result. Additionally, the verification stimuli are input into the design under test to obtain a first execution result; and the verification stimuli are input into the reference model to obtain a second execution result.
[0064] In this embodiment, the neural network model is not compiled or simulated, and can be executed in parallel with the chip verification simulation. It does not participate in the compilation and simulation process of the chip verification environment, thus achieving dual verification of the simulation results. Therefore, no compilation and simulation time is added, saving valuable time in chip verification. Specifically, based on the parallel processing mechanism, the verification stimulus is input into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result.
[0065] Step S15: Compare the first execution result with the second execution result, and use the third execution result to verify the correct results that have the same result.
[0066] In this embodiment, during chip verification simulation, verification stimuli are input into the design under test (DUT), a reference model, and a target neural network model, and execution results are obtained for each. The first execution result of the DUT is compared with the second execution result of the reference model to determine the correct result, which is then double-verified. That is, only the correct execution result in the scoring board is checked for correctness. If the first execution result is the same as the second execution result, it is then compared with the third execution result of the target neural network model. If the third execution result differs from the first two and the execution result of the neural network model exceeds a set threshold, a potential vulnerability exists in the DUT, requiring further analysis and confirmation. Conversely, if the third execution result differs from the correct comparison result, but the third execution result of the target neural network model is less than the set threshold, no vulnerability exists in the chip. Similarly, if the execution result of the neural network model is the same as the correct comparison result, no vulnerability exists in the chip.
[0067] Specifically, if the third execution result is different from the correct result and the third execution result is greater than a preset threshold, then it is determined that there is a vulnerability in the design under test; if the third execution result is different from the correct result and the third execution result is not greater than the preset threshold, then it is determined that there is no vulnerability in the design under test; if the third execution result is the same as the correct result, then it is determined that there is no vulnerability in the design under test.
[0068] like Figure 2The illustration shows a novel chip verification platform provided in this application embodiment. Independent of traditional chip verification frameworks, it incorporates a trained neural network model into the traditional UVM (Universal Verification Methodology) verification platform. This model provides dual verification of the chip simulation verification results, further checking their correctness. Neural network models excel at learning algorithmic computation tasks, making them suitable for learning the functional characteristics of the algorithm module DUT, achieving the same functionality as the DUT. Verification stimuli are used as input to the neural network model, allowing it to learn and continuously improve. During chip verification simulation, the execution results of the neural network model are used to re-verify the chip simulation verification results, uncovering potential but potentially overlooked bugs in the DUT, thus improving the reliability of chip simulation verification. Furthermore, the trained neural network model can be executed in parallel with the chip verification simulation process, without consuming compilation and simulation time, saving valuable simulation time. It is understood that the construction of the neural network model and the quantity and quality of samples directly affect the accuracy of task prediction. For designs under test with relatively simple functions, ordinary neural network models can achieve ideal results. However, for designs under test with complex functions, it is necessary to build specific neural network models suitable for the current task, or even build multiple neural network models. Moreover, complex tasks often require more high-quality training samples, which places high demands on the quantity and quality of the samples.
[0069] This application provides a chip verification result verification method applied to a preset chip verification platform, comprising: acquiring test cases used to verify the target function of the design under test during chip verification; determining verification stimuli from the test cases; determining a model training sample library based on the verification stimuli, and training a preset neural network using samples in the model training sample library to obtain a target neural network model; inputting the verification stimuli into the design under test, a reference model, and the target neural network model respectively to obtain corresponding first execution results, second execution results, and third execution results; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test; comparing the first execution result with the second execution result, and using the third execution result to verify correct results with the same result. It can be seen that, from the chip verification perspective, a trained target neural network model is added to the traditional chip verification platform. The neural network model is adept at learning algorithmic computation tasks. Through self-learning, this neural network model has the same function as the design under test and the reference model, thus forming a new chip verification platform composed of the design under test, the reference model, and the target neural network model. Compared to traditional chip verification methods that input verification stimuli into two paths, this method adds a third path, simultaneously inputting the verification stimuli into the neural network model. The execution results of the neural network model are used to further verify the comparison results between the reference model and the design under test (DUT) during simulation, improving the reliability of chip verification and more effectively identifying potential but potentially overlooked vulnerabilities in the DUT. Furthermore, this neural network model operates independently of the pre-defined chip verification platform, without compilation or simulation, and can be executed in parallel with the chip verification simulation between the DUT and the reference model, without adding compilation and simulation time, thus saving valuable chip verification time.
[0070] This application discloses a specific method for verifying chip verification results. (See also...) Figure 3 As shown, applied to a preset chip verification platform, the method includes:
[0071] Step S21: Obtain test cases used to verify the target function of the design under test during chip verification.
[0072] Step S22: Determine the verification stimulus from the test cases.
[0073] For more detailed processing procedures regarding steps S21 and S22, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.
[0074] Step S23: Generate a model training sample library based on the verification stimulus and the random constraints of the data in the verification stimulus.
[0075] Step S24: Label each sample in the model training sample library with a corresponding label to obtain sample labels.
[0076] In this embodiment, samples are generated based on the verification stimulus and the random constraints of the data in the verification stimulus. The generated samples are then manually labeled to produce labeled samples. To ensure the performance of the neural network model, the number of generated samples should be appropriately large, and the number of samples in each category should be kept balanced.
[0077] Step S25: Input the samples carrying the sample labels from the model training sample library into the preset neural network to obtain the prediction output corresponding to the samples.
[0078] For example, such as Figure 4 The diagram shows a common fully connected neural network model, consisting of three layers: an input layer, hidden layers, and an output layer. The input layers (input 1, input 2, ..., input n) serve as validation stimuli. The output layers (output 1, output 2, ..., output m) represent the execution results of the current test case.
[0079] Step S26: Based on the predicted output and the sample label, determine the prediction error of the target neural network model during the training process, and according to the prediction error, use a neural network optimization algorithm to train the preset neural network. When the neural network model converges, determine the target neural network model.
[0080] In this embodiment, the generated samples are input into the constructed neural network model, the error between the predicted value of the neural network model and the sample label is calculated, and the neural network model is trained using a neural network optimization algorithm until the neural network model converges. Convergence indicates that the neural network model training is complete.
[0081] Step S27: Input the verification stimulus into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test.
[0082] Step S28: Compare the first execution result with the second execution result, and use the third execution result comparison to verify the correct results that have the same result.
[0083] For more detailed processing procedures regarding steps S27 and S28, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.
[0084] This application provides a chip verification result verification method applied to a preset chip verification platform, comprising: acquiring test cases used to verify the target function of the design under test during chip verification; determining verification stimuli from the test cases; generating a model training sample library based on the verification stimuli and random constraints of the data in the verification stimuli; labeling each sample in the model training sample library with a corresponding label to obtain sample labels; inputting samples carrying the sample labels in the model training sample library into the preset neural network to obtain a prediction output corresponding to the sample; and determining the target neural network model during the training process based on the prediction output and the sample labels. The prediction error is calculated, and based on the prediction error, the preset neural network is trained using a neural network optimization algorithm. When the neural network model converges, the target neural network model is determined. The verification stimulus is input into the design under test, the reference model, and the target neural network model respectively to obtain corresponding first execution results, second execution results, and third execution results. The reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test. The first execution result is compared with the second execution result, and the third execution result is used to verify correct results that are identical. Therefore, from the perspective of chip verification, a trained target neural network model is added to the traditional chip verification platform. The neural network model is adept at learning algorithmic tasks. Through self-learning, this neural network model has the same functions as the design under test and the reference model. Thus, the design under test, the reference model, and the target neural network model form a new chip verification platform. Compared to traditional chip verification methods that input verification stimuli into two paths, this method adds a third path, simultaneously inputting the verification stimuli into the neural network model. The execution results of the neural network model are used to further verify the comparison results between the reference model and the design under test (DUT) during simulation, improving the reliability of chip verification and more effectively identifying potential but potentially overlooked vulnerabilities in the DUT. Furthermore, this neural network model operates independently of the pre-defined chip verification platform, without compilation or simulation, and can be executed in parallel with the chip verification simulation between the DUT and the reference model, without adding compilation and simulation time, thus saving valuable chip verification time.
[0085] The technical solution in this application will be illustrated by a specific example below.
[0086] Suppose there exists an algorithm module DUT (Design Under Test) with a single parity check function. Odd parity means that the parity bit is 0 when the number of 1s in the data bits is odd, and 1 otherwise; even parity means that the parity bit is 0 when the number of 1s in the data bits is even, and 1 otherwise. The verification stimulus is input to the DUT, containing the configured odd or even parity function and the data to be verified. The DUT outputs a parity bit, which contains only 0 or 1. The register length and the data length in the verification stimulus are both 32 bits, and the parity bit output by the DUT and the reference model is 1 bit. An example is as follows:
[0087] There exists a 32-bit data set: 1111_1111_1111_1111_1110_0000_0000_0000, where the number of 1s is odd. If the odd parity check is selected, the parity bit is 0 because the number of 1s is odd; if the even parity check is selected, the parity bit is 1 because the number of 1s is odd.
[0088] There is a test case used to verify the parity check function of the design under test. Its execution flow is as follows:
[0089] 1. Configure the clock enable register of the design under test (DUT) to enable the DUT;
[0090] 2. Configure the parity register of the design under test to select the parity check function;
[0091] 3. Send the verification data to the register in the design under test used to store the verification data;
[0092] 4. Obtain the execution results of the design under test;
[0093] 5. Obtain the execution results of the reference model;
[0094] 6. Compare the execution results of the design under test with the execution results of the reference model.
[0095] First, samples are generated based on the validation stimuli in the test cases and the random constraints of the data in the validation stimuli. There are three validation stimuli in the above test cases, as follows:
[0096] Verify stimulus 1 by configuring the clock enable register of the design under test;
[0097] Verification stimulus 2: Configure the parity check register of the design under test;
[0098] Verification stimulus 3 sends the data to be verified to the verification data register of the design under test.
[0099] Of these, only the data in verification stimulus 3 has a random constraint: the data in verification stimulus 3 is 32 bits long, and each bit can only be 0 or 1. Therefore, there are 232 data points that satisfy this constraint. Because 232 data points is a large number, it is necessary to sample from these 232 data points, sampling a total of n data points, in the following order: verification stimulus 3_1, verification stimulus 3_2, ..., verification stimulus 3_n. Finally, the following n samples will be generated:
[0100] Sample 1: [Validation stimulus 1, Validation stimulus 2, Validation stimulus 3_1],
[0101] Sample 2: [Validation stimulus 1, Validation stimulus 2, Validation stimulus 3_2],
[0102] …
[0103] Sample n: [Validation stimulus 1, Validation stimulus 2, Validation stimulus 3_n].
[0104] However, labeled samples are used when training the neural network model, so labels need to be added to n samples. A label refers to the correct execution result obtained by performing the same operation on a single sample as the design under test. Therefore, the n labeled samples are as follows:
[0105] Sample 1: [Validation stimulus 1, Validation stimulus 2, Validation stimulus 3_1, Label 1]
[0106] Sample 2: [Validation stimulus 1, Validation stimulus 2, Validation stimulus 3_2, Label 2]
[0107] …
[0108] Sample n: [Validation stimulus 1, Validation stimulus 2, Validation stimulus 3_n, Label n].
[0109] Example of a sample:
[0110] Sample 1: Odd check, the number of 1s in the data is odd, so the label is 0.
[0111] [0000_0000_0000_0000_0000_0000_0000_0001,
[0112] 0000_0000_0000_0000_0000_0000_0001,
[0113] 0000_0000_0000_0000_0000_0001_1111, 0];
[0115] Sample 2: Odd check, the number of 1s in the data is even, so the label is 1.
[0116] [0000_0000_0000_0000_0000_0000_0000_0001,
[0117] 0000_0000_0000_0000_0000_0000_0001,
[0118] 0000_0000_0000_0000_0000_0000_1111, 1];
[0120] Sample 3: Even parity, the number of 1s in the data is even, so the label is 0.
[0121] [0000_0000_0000_0000_0000_0000_0000_0001,
[0122] 0000_0000_0000_0000_0000_0000_0000,
[0123] 0000_0000_0000_0000_0000_0000_1111, 0];
[0125] Sample 4: Even parity, the number of 1s in the data is odd, so the label is 1.
[0126] [0000_0000_0000_0000_0000_0000_0000_0001,
[0127] 0000_0000_0000_0000_0000_0000_0000,
[0128] 0000_0000_0000_0000_0000_0001_1111,
[0129] 1).
[0130] Next, we construct the neural network model. Here, we use a three-layer fully connected neural network model, such as... Figure 4 As shown. The fully connected neural network model has an input layer dimension of 32, which is the same as the length of the registers in the design under test and the length of the data in the verification stimulus. The hidden layer dimension is N, where N is a positive integer. The output layer dimension is 2, representing the parity bits as 0 and 1 respectively.
[0131] Next, the generated samples are input into the neural network model to train the model until it converges. When inputting the samples into the neural network model, the data in the samples needs to be converted into a data format that the neural network model can process. This is done by summing the data in the samples: validation stimulus 1 + validation stimulus 2 + validation stimulus 3. The converted sample data is a vector with a dimension of 32, denoted as X. Vector X is used as the input to the neural network model.
[0132] Finally, the trained neural network model is used... Figure 2 The chip verification process is shown below. During chip verification simulation, the following verification stimuli are sequentially input to the design under test:
[0133] Verification stimulus 1: Configure the clock enable register of the design under test.
[0134] Verification stimulus 2: Configure the parity check register of the design under test.
[0135] Verification stimulus 3: Send the data to be verified to the verification data register of the design under test.
[0136] The input to the neural network model (referring to the trained neural network model here and below) has the same format as the vector X mentioned above, denoted as X1. Vector X1 undergoes the following operations in the neural network model:
[0137] H = f(X1W1 + b1), where H is the hidden layer vector with dimension N; W1 is the weight matrix with dimension 32×N; b1 is the offset; and f is the activation function.
[0138] Then, the following operation is performed on vector H to obtain the output vector Y: Y(y1,y2)=SoftMax(HW2+b2); where W2 is the weight matrix with dimension N×2; b2 is the offset; SoftMax is the activation function, which maps each number in the vector to a value between 0 and 1, representing the probability of prediction, and the sum of all values is 1; Y is the output vector of the output layer with dimension 2; y1 and y2 represent the probabilities of the model's execution result being 0 and 1, respectively, and y1+y2=1. If y1>y2, the execution result of the neural network model is the label value corresponding to the position of y1; otherwise, it is the label value corresponding to the position of y2.
[0139] Simultaneously, the execution results of the design under test and the reference model will also output a checksum, and the two execution results will be compared. If the comparison is correct, that is, the two execution results are the same, then the following execution results are obtained based on the output vector Y of the neural network model:
[0140] Assuming the threshold is set to 0.8,
[0141] 1. The execution result of the neural network model differs from the execution result of the design under test (or the execution result of the reference model). Assume the execution result of the neural network model is 1, while the execution result of the design under test is 0. Assume Y = (0.4, 0.6), where 0.4 represents the probability of a value of 0 and 0.6 represents the probability of a value of 1. Since 0.6 < 0.8, it is assumed that there are no bugs in the design under test.
[0142] 2. The execution result of the neural network model differs from the execution result of the design under test (or the reference model). Assume the neural network model's execution result is 1, while the DUT's execution result is 0. Assume Y = (0.1, 0.9), where 0.1 represents the probability of a value of 0 and 0.9 represents the probability of a value of 1. Because 0.9 > 0.8, it is considered that there may be a bug in the design under test, requiring further analysis.
[0143] 3. If the execution result of the neural network model is the same as the execution result of the design under test (or the execution result of the reference model), then it is considered that there is no bug in the design under test.
[0144] Accordingly, this application also discloses a chip verification result verification device, see [link to relevant documentation]. Figure 5 As shown, the device, applied to a pre-defined chip verification platform, includes:
[0145] Test case acquisition module 11 is used to acquire test cases used to verify the target function of the design under test during chip verification.
[0146] The verification stimulus determination module 12 is used to determine the verification stimulus from the test cases;
[0147] The neural network model training module 13 is used to determine the model training sample library based on the verification stimulus, and use the samples in the model training sample library to train the preset neural network to obtain the target neural network model.
[0148] The execution result acquisition module 14 is used to input the verification stimulus into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test;
[0149] The result verification module 15 is used to compare the first execution result with the second execution result, and to verify the correct results that have the same result by comparing the third execution result.
[0150] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.
[0151] Therefore, the above-described scheme of this embodiment, applied to a preset chip verification platform, includes: acquiring test cases used to verify the target function of the design under test during chip verification; determining verification stimuli from the test cases; determining a model training sample library based on the verification stimuli, and training a preset neural network using samples in the model training sample library to obtain a target neural network model; inputting the verification stimuli into the design under test, the reference model, and the target neural network model respectively to obtain corresponding first execution results, second execution results, and third execution results; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test; comparing the first execution result with the second execution result, and using the third execution result to verify correct results with the same result. It can be seen that, from the perspective of chip verification, a trained target neural network model is added to the traditional chip verification platform. The neural network model is adept at learning algorithmic computation tasks. Through self-learning, this neural network model has the same function as the design under test and the reference model, thus forming a new chip verification platform composed of the design under test, the reference model, and the target neural network model. Compared to traditional chip verification methods that input verification stimuli into two paths, this method adds a third path, simultaneously inputting the verification stimuli into the neural network model. The execution results of the neural network model are used to further verify the comparison results between the reference model and the design under test (DUT) during simulation, improving the reliability of chip verification and more effectively identifying potential but potentially overlooked vulnerabilities in the DUT. Furthermore, this neural network model operates independently of the pre-defined chip verification platform, without compilation or simulation, and can be executed in parallel with the chip verification simulation between the DUT and the reference model, without adding compilation and simulation time, thus saving valuable chip verification time.
[0152] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.
[0153] Figure 6 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the chip verification result verification method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.
[0154] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0155] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it can include an operating system 221, computer programs 222, and data 223, etc. The data 223 can include various types of data. The storage method can be temporary storage or permanent storage.
[0156] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the chip verification result verification method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.
[0157] Furthermore, this application also discloses a computer-readable storage medium, which includes random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, magnetic disks, optical disks, or any other form of storage medium known in the art. The computer program, when executed by a processor, implements the aforementioned chip verification result verification method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0158] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0159] The steps of chip verification result verification or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0160] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0161] The present invention has provided a detailed description of a chip verification result verification method, apparatus, device, and storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method of checking a chip verification result, characterized by, Applied to pre-defined chip verification platforms, including: Obtain test cases used to verify the target functionality of the design under test during chip verification; Determine the verification stimulus from the test cases; Based on the verification stimulus, a model training sample library is determined, and the preset neural network is trained using the samples in the model training sample library to obtain the target neural network model. The verification stimulus is input into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test; The first execution result is compared with the second execution result, and the correct results with the same result are verified by comparing the third execution result.
2. The chip verification result checking method according to claim 1, wherein The process of determining the model training sample library based on the verification stimulus includes: A model training sample library is generated based on the verification stimulus and the random constraints of the data in the verification stimulus.
3. The chip verification result verification method according to claim 1, characterized in that, Before training the preset neural network using samples from the model training sample library to obtain the neural network model, the method further includes: Each sample in the training sample library of the model is labeled with a corresponding tag to obtain sample labels; Accordingly, the step of training a preset neural network using samples from the model training sample library to obtain a target neural network model includes: The samples carrying the sample labels in the model training sample library are input into the preset neural network to obtain the prediction output corresponding to the sample; Based on the predicted output and the sample labels, the prediction error of the target neural network model during the training process is determined. Based on the prediction error, the preset neural network is trained using a neural network optimization algorithm. When the neural network model converges, the target neural network model is determined.
4. The chip verification result checking method of claim 1, wherein The step of training a preset neural network using samples from the model training sample library to obtain a target neural network model includes: The target neural network model is obtained by training any one of the fully connected neural networks, convolutional neural networks, and recurrent neural networks using samples from the model training sample library.
5. The chip verification result checking method of claim 1, wherein The step of inputting the verification stimulus into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result includes: The verification stimulus is input into the design under test to obtain a first execution result; The verification stimulus is input into the reference model to obtain the second execution result; The verification stimulus is converted to obtain a data format that the target neural network model can process. Then, the verification stimulus converted to the data format is input into the target neural network model to obtain a third execution result.
6. The chip verification result checking method of claim 1, wherein The step of inputting the verification stimulus into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result includes: Based on the parallel processing mechanism, the verification stimulus is input into the design under test, the reference model and the target neural network model respectively to obtain the corresponding first execution result, second execution result and third execution result.
7. The chip verification result verification method according to any one of claims 1 to 6, characterized in that, The step of comparing the first execution result with the second execution result and using the third execution result to verify the correct results that have the same result includes: If the third execution result is different from the correct result, and the third execution result is greater than a preset threshold, then it is determined that there is a vulnerability in the design under test. If the third execution result is different from the correct result, and the third execution result is not greater than the preset threshold, then it is determined that the vulnerability does not exist in the design under test. If the third execution result is the same as the correct result, then it is determined that the vulnerability does not exist in the design under test.
8. A chip verification result verification device, characterized in that, Applied to pre-defined chip verification platforms, including: The test case acquisition module is used to acquire test cases for verifying the target function of the design under test during chip verification. The verification stimulus determination module is used to determine the verification stimulus from the test cases; The neural network model training module is used to determine the model training sample library based on the verification stimulus, and to train the preset neural network using the samples in the model training sample library to obtain the target neural network model. The execution result acquisition module is used to input the verification stimulus into the design under test, the reference model, and the target neural network model respectively to obtain the corresponding first execution result, second execution result, and third execution result; wherein, the reference model is a chip verification platform component with the same function constructed in the preset chip verification platform according to the principle of the design under test; The result verification module is used to compare the first execution result with the second execution result, and to verify the correct results that have the same result by comparing the third execution result.
9. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the chip verification result verification method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein the computer programs, when executed by a processor, implement the chip verification result verification method as described in any one of claims 1 to 7.