Chip verification method, system, device and storage medium

Abstract

The application provides a chip verification method, system, device and storage medium, the method comprises the following steps: generating excitation data according to model configuration parameters; establishing a software reference model and a hardware simulation reference model; inputting the excitation data into the software reference model and the hardware simulation reference model respectively, and obtaining software reference output and hardware simulation reference output; in response to comparison of the software reference output and the hardware simulation reference output, determining whether the hardware simulation reference model passes the reference model evaluation; according to the excitation data, performing simulation verification to obtain simulation output; in response to the hardware simulation reference model passing the reference model evaluation, comparing the hardware simulation reference output and the simulation output to obtain a chip verification result. The application is applied to simulation and verification of a neural network model inference chip, and improves verification efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a chip verification method, system, device and storage medium. BACKGROUND

[0002] Chip verification is an important part of chip development. Through chip verification, design defects and errors can be found in a timely manner to ensure that the design of the chip meets the expected functions. Due to the complexity of the system and the tediousness of the work, chip verification has always occupied a large amount of manpower and time. In recent years, with the landing of artificial intelligence algorithms, some chips have appeared that are specifically designed for accelerating the inference of artificial intelligence (AI) models. In order to make this chip product suitable for complex and diverse models and application scenarios, and to improve the product's market competitiveness, the chip itself is also designed to be more flexible and have a faster iteration speed, so a lightweight, highly expandable customized chip verification system is essential to shorten the chip development cycle.

[0003] The mainstream framework of chip verification at present is the universal verification methodology (UVM). UVM is a universal verification methodology. In this traditional verification structure, even if only the function of one processor is verified, all components in the verification structure need to be involved. This makes the verification resource consumption large and the time spent relatively long. AI inference algorithms have more variable and diverse characteristics depending on the scene. In order to cope with this characteristic, AI chips need a more flexible verification platform to adapt to product development and iteration. The complex modeling basis of the UVM library and the abstract level verification language based on System Verilog (SV) cannot well match the deep customization of AI chip verification requirements.

[0004] "Chip verification method and system, and storage medium", CN111859831A, for the artificial intelligence core module (AI Core) of the convolutional neural network, needs to actively move data features such as features, weights, and offsets from the core external storage unit. A chip verification system based on the UVM verification methodology is adopted, which makes the verification system highly reusable and configurable. However, this verification system relies on many UVM components itself, and verification personnel need to understand the chip architecture, the protocols involved, and the interface of each component is relatively complex to build.

[0005] Method, system and verification platform for system on chip verification, CN113051855A, it is observed that the complex verification platform is built through UVM library, and the high consumption of resource verification, by establishing a variety of component library including interface protocol component, bus protocol component for system on chip, software library for each processor running of system on chip and excitation library of corresponding component, an improved method for system on chip verification is proposed. Although the method alleviates the dependence of the verification platform based on UVM library, it also introduces some relatively low-level chip design concepts irrelevant to chip verification, and the whole verification process is relatively complex, which is not conducive to the phased verification needs of chip development.

[0006] Chip verification system and method, CN113220518A, proposes to encapsulate the chip verification process into several independent modules such as excitation module and configuration module according to the special format of the chip to be verified, to ensure the accuracy of chip function verification. This verification method can only meet the functional verification needs of the chip, and the function is relatively simple.

[0007] Currently, AI algorithm models in the industry have the characteristics of deep customization: for different application scenarios and data, algorithm engineers will design different model architectures, and these models can be composed of different operators, such as convolutional neural networks based on computer vision, which can be divided into convolution operators and pooling operators. Chip developers can initially develop some acceleration cards that can cover simple inference models by using operators as units, and then follow the expansion of business and the evolution of AI models to continuously iterate products with more supported operator libraries to support more inference models. Chip verification is an important part of chip development, which determines the development cycle of the chip. For such AI chip development characteristics, the current general chip verification method or platform involves many chip underlying concepts, which is not conducive to the quick start of verification personnel; the verification process is complex, and a small change requires global adjustment, verification resource consumption is large, and it limits the later function expansion and code reuse.

[0008] In addition, AI chips are diverse and changeable, and often have clever designs at the algorithm level to optimize model performance. If the hardware design cannot fully support the implementation, some adjustments will be made, and the adjusted scheme needs to be backtracked to evaluate the impact on the overall model performance. Therefore, the process of AI chip product development from scheme design, development to chip verification is iterative and interactive. Most of the abstract level verification languages based on System Verilog (SV) and C compilation platform, object-oriented programming features and verification platform design, ignore the relevance between each link, to some extent, increase the workload of chip development.

[0009] Therefore, how to optimize chip verification to reduce chip verification complexity, improve the flexibility and expansibility of chip verification, and reduce the redundancy of chip verification is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0010] To solve the problems in the prior art, the purpose of the present application is to provide a chip verification method, system, device and storage medium, which optimizes chip verification to reduce chip verification complexity, improve the flexibility and expansibility of chip verification, and reduce the redundancy of chip verification.

[0011] The chip verification method provided by the embodiment of the present application comprises the following steps:

[0012] Generating excitation data according to model configuration parameters;

[0013] Establishing a software reference model and a hardware simulation reference model;

[0014] Inputting the excitation data into the software reference model and the hardware simulation reference model respectively, and obtaining software reference output and hardware simulation reference output, and determining whether the hardware simulation reference model passes the reference model evaluation based on the comparison of the software reference output and the hardware simulation reference output;

[0015] According to the excitation data, performing simulation verification to obtain simulation output;

[0016] In response to the hardware simulation reference model passing the reference model evaluation, comparing the hardware simulation reference output and the simulation output to obtain chip verification results.

[0017] In some embodiments of the present application, the generating excitation data according to model configuration parameters comprises:

[0018] Generating first excitation data and second excitation data, wherein the first excitation data is generated based on a trained model, and the second excitation data is generated based on a random function,

[0019] Wherein, the software reference model, the hardware simulation reference model and the simulation verification all use the second excitation data, and the first excitation data is used for software development performance verification of the chip.

[0020] In some embodiments of the present application, the inputting the excitation data into the software reference model and the hardware simulation reference model respectively, and obtaining software reference output and hardware simulation reference output comprises:

[0021] Inputting the excitation data into the software reference model and the hardware simulation reference model respectively, and obtaining software intermediate layer output of the software reference model and hardware simulation intermediate layer output of the hardware simulation reference model;

[0022] obtaining an intermediate output difference between the software intermediate layer output and the hardware-in-the-loop intermediate layer output;

[0023] obtaining a software performance parameter of the software reference model and a hardware-in-the-loop performance parameter of the hardware-in-the-loop reference model;

[0024] obtaining a performance difference between the software performance parameter and the hardware-in-the-loop performance parameter;

[0025] determining whether the hardware-in-the-loop reference model passes the reference model evaluation according to a functional relationship between the intermediate output difference and the performance difference.

[0026] In some embodiments of the present application, the simulation verification according to the stimulus data comprises:

[0027] register transfer level simulation verification according to the stimulus data.

[0028] In some embodiments of the present application, the simulation verification according to the stimulus data comprises:

[0029] field programmable gate array prototype simulation verification according to the stimulus data.

[0030] In some embodiments of the present application, in response to the chip verification result being a second verification evaluation failure, the code of the simulation verification is modified for regression testing;

[0031] In response to the chip verification result being a second verification evaluation pass, the code of the simulation verification is saved for regression testing.

[0032] In some embodiments of the present application, after the generation of the stimulus data according to the model configuration parameters, the method further comprises:

[0033] performing parameter configuration of the model according to the stimulus data, and performing data compilation according to the parameter configuration, so as to perform software development of the chip.

[0034] According to still another aspect of the present application, a chip verification system is also provided, comprising:

[0035] a stimulus generation module configured to generate stimulus data according to model configuration parameters;

[0036] a reference model module configured to establish a software reference model and a hardware-in-the-loop reference model;

[0037] The reference model output module is configured to input the excitation data into the software reference model and the hardware-in-silico reference model respectively, and obtain software reference output and hardware-in-silico reference output.

[0038] The simulation module is configured to perform simulation verification according to the excitation data, and obtain simulation output.

[0039] The verification result module is configured to compare the hardware-in-silico reference output and the simulation output to obtain a chip verification result in response to the hardware-in-silico reference model passing the reference model evaluation.

[0040] Embodiments of the present application also provide a chip verification device, comprising:

[0041] A processor;

[0042] A memory having executable instructions of the processor stored therein;

[0043] The processor is configured to execute the steps of the chip verification method by executing the executable instructions.

[0044] Embodiments of the present application also provide a computer readable storage medium for storing a program, wherein the program is executed by a processor to implement the steps of the chip verification method.

[0045] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure.

[0046] The chip verification method, system, device and storage medium of the present application have the following beneficial effects:

[0047] The present application establishes a software reference model and a hardware-in-silico reference model, and inputs excitation data into the software reference model and the hardware-in-silico reference model respectively, to determine whether the hardware-in-silico reference model passes the reference model evaluation based on the comparison between the software reference output and the hardware-in-silico reference output. The present application can realize automatic evaluation of the hardware-in-silico reference model. When the hardware-in-silico reference model fails to pass the evaluation, the hardware-in-silico reference model can be directly adjusted without performing subsequent simulation verification. When the hardware-in-silico reference model passes the evaluation, subsequent simulation verification can be performed based on the hardware-in-silico reference model. Thus, the chip verification is optimized to reduce the complexity of chip verification, improve the flexibility and expansibility of chip verification, and reduce the redundancy of chip verification. BRIEF DESCRIPTION OF DRAWINGS

[0048] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments with reference to the attached drawings.

[0049] Figure 1 is a flowchart of a chip verification method according to an embodiment of the present application;

[0050] Figure 2 is a flowchart of model evaluation according to an embodiment of the present application;

[0051] Figure 3 is a flowchart of a chip verification method according to an embodiment of the present application;

[0052] Figure 4 is a schematic diagram of a chip development life cycle according to an embodiment of the present application;

[0053] Figure 5 is a schematic diagram of a chip verification system according to an embodiment of the present application;

[0054] Figure 6 is a schematic diagram of the interaction of various modules of a chip verification system according to an embodiment of the present application;

[0055] Figure 7 is a schematic diagram of a chip verification device according to an embodiment of the present application;

[0056] Figure 8 is a schematic diagram of a computer readable storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive aspects to those skilled in the art. Features described in the description and / or shown in the figures can be combined in any suitable manner without departing from the scope of the present disclosure.

[0058] In addition, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments and, as such, should not be considered limiting in any way. Identical reference numerals in the figures indicate identical or functionally similar elements, and thus their description need not be repeated. Some of the blocks in the drawings are functional blocks that represent one or more functions implemented by a processor, software, or combination thereof. These functional blocks may

[0059] The flowcharts shown in the figures are merely examples and do not necessarily include all steps. For example, some steps can be further divided, and some steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0060] As Figure 1 shown in the figure, an embodiment of the present application provides a chip verification method applied to simulation and verification of a neural network model inference chip, and the method comprises the following steps:

[0061] Step S110: generating excitation data according to model configuration parameters.

[0062] Specifically, the artificial intelligence algorithm (AI) is different from the traditional algorithm, and is variable in form and fast in version iteration, which is manifested as more layers and more types of operators. Different combinations of operators are different algorithm models. The operator is composed of some hyperparameters, and a large number of test cases can be generated by randomly combining these hyperparameters, which requires the chip verification framework of the AI algorithm to be more lightweight and highly flexible.

[0063] Therefore, in the present application, the model configuration parameters can include the number of neural network layers, the type of each layer operator, the shape of input data, the shape of weight data, the parameters of the operator, etc. Specifically, the operator is a single functional unit in the neural network model. Taking a convolutional neural network model as an example, it includes convolution operators, pooling operators, fully connected operators, etc.

[0064] In some embodiments, the model configuration parameters can define the operator parameters and test parameters of at least one test case, and the test parameters further include a test case number. The test case can be a functional test for the operator, or a test for a model. A model includes multiple operators, and the operators in the model are connected in a certain order. Different operator parameters can form different operator test cases. Taking a general convolution operator as an example, the operator parameters include the length, width, channel number, convolution kernel number, convolution kernel size, convolution step, and zero padding mode. When a test case is for operator testing, the running mode is single operator testing, thereby realizing the testing of the function of a single operator, and facilitating problem tracking and positioning when verification problems occur.

[0065] Step S120: establishing a software reference model and a hardware reference model.

[0066] Specifically, in the present application, the software reference model established in step S120 is implemented based on the pure software idea of algorithm, which does not consider the more underlying hardware logic, directly calls the general algorithm library of software at each step, is more focused on the algorithm idea itself, and outputs data as a real reference standard. The hardware-imitating reference model established in step S120 is implemented based on the software simulation of the hardware idea. This part is loyal to the hardware design idea, and each step is completed according to the chip bottom layer design idea to realize the function of the algorithm. The output of the hardware-imitating reference model will be compared with the subsequent simulation output to realize the verification of the chip.

[0067] In some specific implementations, the intermediate position observation points (such as the set intermediate layer / intermediate operator observation points) can be set in advance for the hardware-imitating reference model, so that when the sub-modules of the chip are combined and debugged, the problem can be directly located.

[0068] Step S130: inputting the excitation data into the software reference model and the hardware-imitating reference model respectively, and obtaining software reference output and hardware-imitating reference output, determining whether the hardware-imitating reference model passes the reference model evaluation based on the comparison between the software reference output and the hardware-imitating reference output.

[0069] Specifically, due to the flexible and variable design of AI operators, which is limited by the complexity of hardware logic, sometimes it cannot be implemented completely in software steps. Therefore, step S130 evaluates the influence of the overall performance of the model by comparing the pure software output of the software reference model and the hardware-imitating output of the hardware-imitating reference model. For example, how much influence will a small change in the output of an intermediate layer of hardware-imitating and pure software have on the performance of the entire model? The evaluation standard is used to judge whether it is within an acceptable range (set range). If so, the hardware-imitating reference model constructed in step S120 is used as the reference standard for chip function verification, and the next function verification process is performed. If the evaluation result is large, it indicates that there is a defect in the chip design idea, which needs to be further adjusted.

[0070] Specifically, the standard of model evaluation can be measured from two dimensions, one is the real value deviation, and the other is the model precision deviation. For example, the real value deviation can be the absolute value of the difference between the software reference output value and the hardware reference output value, and the percentage of the absolute value of the software reference output. If it exceeds the set threshold (such as 10%), it is considered that the error is large, and the hardware implementation scheme needs to be adjusted. The model precision deviation can be considered as a performance evaluation standard commonly used for the performance of different task models, such as the mean intersection over union (mIOU) of the segmentation task model. According to the model output, the mIOU of the software reference scheme and the hardware reference scheme is calculated respectively. If the difference between the two values is greater than the set threshold (such as 0.02), it is considered that the error is large, and the hardware implementation scheme needs to be adjusted.

[0071] Step S140: according to the excitation data, performing simulation verification to obtain simulation output.

[0072] Specifically, one or more simulation verification methods can be implemented in step S140, such as register transfer level simulation verification, FPGA prototype verification, etc., and the application is not limited thereto.

[0073] Step S150: in response to the fact that the simulation hardware reference model passes the reference model evaluation, comparing the simulation hardware reference output and the simulation output to obtain chip verification results.

[0074] The application establishes a software reference model and a simulation hardware reference model, and inputs excitation data into the software reference model and the simulation hardware reference model respectively, so as to determine whether the simulation hardware reference model passes the reference model evaluation based on the comparison between the software reference output and the simulation hardware reference output. The application can realize automatic evaluation of the simulation hardware reference model. When the simulation hardware reference model does not pass the evaluation, the simulation hardware reference model can be directly adjusted without performing subsequent simulation verification. When the evaluation is passed, subsequent simulation verification can be performed based on the simulation hardware reference model. Thus, the chip verification is optimized to reduce the complexity of chip verification, improve the flexibility and expansibility of chip verification, and reduce the redundancy of chip verification.

[0075] The following will be described with reference to Figure 2 , Figure 2 is a flowchart of model evaluation of an embodiment of the application. Figure 2 The following steps are shown:

[0076] Step S131: inputting the excitation data into the software reference model and the simulation hardware reference model respectively to obtain software intermediate layer output of the software reference model and simulation hardware intermediate layer output of the simulation hardware reference model.

[0077] Step S132: obtaining intermediate output difference of the software intermediate layer output and the simulation hardware intermediate layer output.

[0078] Step S133: obtaining software performance parameters of the software reference model and simulation hardware performance parameters of the simulation hardware reference model.

[0079] Step S134: obtaining performance difference of the software performance parameters and the simulation hardware performance parameters.

[0080] Step S135: judging whether the intermediate layer of the simulation hardware reference model passes the reference model evaluation according to the function relationship between the intermediate output difference and the performance difference.

[0081] Specifically, in this embodiment, the software intermediate layer output and the hardware-imitating intermediate layer output are the same layer of the AI model. The intermediate layer may, for example, be the aforementioned observation point. In some variations of the present application, the intermediate layer of the above steps S131 to S135 may also be an arbitrary intermediate operator. Thus, the present application can achieve flexible evaluation of different types of reference models.

[0082] Specifically, in the above steps, the intermediate output difference may, for example, be the difference between the software intermediate layer output and the hardware-imitating intermediate layer output. In some variations, the intermediate output difference may also be a difference value obtained after normalization based on the difference between the software intermediate layer output and the hardware-imitating intermediate layer output.

[0083] Specifically, in the above steps, the performance parameter may, for example, be one or more of the accuracy, recall rate, and execution speed of the model, without being limited thereto.

[0084] Specifically, in the above steps, the intermediate output difference of a plurality of intermediate layer performance differences can be obtained, so that a functional relationship between each of the intermediate output differences and the performance difference can be fitted, thereby locating the intermediate layer (intermediate operator) that has the greatest impact on the performance difference, so as to facilitate the adjustment of the intermediate layer (intermediate operator) by the designer.

[0085] In some other variations of the present application, the software reference model and the hardware-imitating reference model can also be evaluated as a whole. For example, when the difference between the overall output of the software reference model and the overall output of the hardware-imitating reference model is greater than a set threshold range, it can be considered that the hardware-imitating reference model fails the reference evaluation; when the difference between the overall output of the software reference model and the overall output of the hardware-imitating reference model falls within the set threshold range, it can be considered that the hardware-imitating reference model passes the reference evaluation.

[0086] Specifically, the evaluation of the intermediate layer (intermediate operator) of the software reference model and the hardware-in-the-loop reference model, and the overall evaluation of the model can be set as needed to realize combined evaluation. For example, in some specific implementations, when the performance difference between the software performance parameter and the hardware-in-the-loop performance parameter is greater than a set threshold range, it is considered that the hardware-in-the-loop reference model fails the reference evaluation; when the performance difference between the software performance parameter and the hardware-in-the-loop performance parameter falls within the set threshold range, it is further judged whether the difference value between the overall output of the software reference model and the overall output of the hardware-in-the-loop reference model, when the difference value between the overall output of the software reference model and the overall output of the hardware-in-the-loop reference model is greater than a set threshold range, it is considered that the hardware-in-the-loop reference model fails the reference evaluation; when the difference value between the overall output of the software reference model and the overall output of the hardware-in-the-loop reference model falls within the set threshold range, it is considered that the hardware-in-the-loop reference model passes the reference evaluation. For example, in some other specific implementations, when the performance difference between the software performance parameter and the hardware-in-the-loop performance parameter is greater than a set threshold range, it is considered that the hardware-in-the-loop reference model fails the reference evaluation, and the steps S131 to S135 are executed to locate the intermediate layer (intermediate operator) that has the greatest impact on performance for reference by chip setting personnel. The present application is not limited thereto, and the specific evaluation method can be set as needed.

[0087] Referring to the following Figure 3 , Figure 3 is a flowchart of a chip verification method according to an embodiment of the present application;

[0088] Step S210: generating first excitation data and second excitation data according to model configuration parameters.

[0089] Specifically, the generation of excitation data has two interfaces, one is a trained real model data interface for providing first excitation data, and the other is a random number simulation interface for providing second excitation data. The random number interface uses the Numpy library (a mathematical function library based on python) to simulate different excitation data each time according to model parameter information, thereby meeting the boundary value and coverage test requirements in the simulation verification stage; and the real model data interface is the existing trained excitation data. The real model data interface can be connected to the network overall performance verification of the corresponding software development of the late-stage product in step S280, and the performance of the real network is more intuitive in evaluating the computing power performance of other products.

[0090] Step S220: establishing a software reference model and a hardware-in-the-loop reference model.

[0091] Specifically, the software reference model and the hardware emulation reference model are established based on the second stimulus data. The software reference model established in step S220 is a pure software idea based on an algorithm. This part does not need to consider the underlying hardware logic, directly calls the general algorithm library of software at each step, is more focused on the algorithm idea itself, and outputs data as a real reference standard. The hardware emulation reference model established in step S220 is a software simulation based on a hardware idea. This part is loyal to the hardware design idea, and each step is implemented according to the chip bottom layer design idea to complete the function implementation of the algorithm. The output of the hardware emulation reference model will be compared with the subsequent simulation output result to realize the verification of the chip.

[0092] Step S231: input the stimulus data into the software reference model and the hardware emulation reference model respectively, and obtain software reference output and hardware emulation reference output. Based on the comparison of the software reference output and the hardware emulation reference output, it is determined whether the hardware emulation reference model passes the reference model evaluation.

[0093] Step S232: determine whether the hardware emulation reference model passes the reference model evaluation.

[0094] If step S232 is determined to be no, return to step S220. If step S232 is determined to be yes, execute step S253.

[0095] Specifically, while establishing the software reference model and the hardware emulation reference model, step S240: parameter configuration can be executed synchronously.

[0096] Specifically, for the AI model inference acceleration chip design, the configurations required for each task start include: register information, model weight data, and input data. The register information refers to the data register, index register, pointer register, etc. designed in advance, which is used to tell the chip task information; the model weight data is the self-owned data of the AI inference model, and each customized model has a set of different data. The input data is the variable of the customized model, and each customized model is to realize the desired inference result on different input data. An AI acceleration chip can deploy multiple sets of different inference task models through register information, and a set of inference model can realize the inference result of different input data. In step S240, the data configuration preparation of a set of model can be realized.

[0097] Step S251: according to the stimulus data, execute register transfer level simulation verification, and obtain register transfer level simulation output.

[0098] Specifically, the stimulus data used in step S251 is the second stimulus data.

[0099] Specifically, when the chip design is completed, the developer will implement the logic design of the module according to the functional and interface requirements using a hardware description language (HDL), form a register transfer level (RTL) code, and then the step S251 will perform functional verification, also known as RTL simulation verification. Specifically, the step S251 first implements the driving preparation of functional simulation: according to the interface design of simulation, the format conversion of the second excitation data is implemented, and the text form is saved. Then, based on the second excitation data which has been converted by the format, the simulation program is started to perform register transfer level simulation verification, and the register transfer level simulation output is obtained.

[0100] Step S252: According to the excitation data, perform field programmable gate array prototype simulation verification, and obtain field programmable gate array prototype simulation output.

[0101] Specifically, the excitation data used in step S252 is the second excitation data.

[0102] Specifically, the field programmable gate array (FPGA) prototype verification is to transplant the RTL code to the field programmable gate array (FPGA) to verify the function of the application specific integrated circuit (ASIC), and after the basic function verification of the chip is passed, the driving and application development can be started. The FPGA verification stage is a necessary stage after the functional logic verification is completed, and the verification of this process contains the real clock delay condition, which can detect the functional defects caused by the delay problem in the actual gate level circuit. In step S252, data conversion is first implemented, which can implement the format conversion of the second excitation data according to the interface design of the FPGA prototype verification, and save it in the text form. Then, the chip working process is started by driving configuration, and thus the simulation output is obtained through the working process of the chip.

[0103] In some specific implementations, the step S252 and the step S251 can be executed synchronously or later than the step S251, and the present application can implement more change modes, which are not described here.

[0104] Step S253: Compare the simulation hardware reference output and the simulation output to determine whether the outputs are consistent.

[0105] If the step S253 determines yes, the step S261 is executed to maintain the code (register transfer level code) of the simulation verification.

[0106] If step S253 determines otherwise, then step S263 is executed: modify the simulation verification code (register transfer level code).

[0107] To ensure more efficient debugging and prevent unintentional impact on other modules or code logic, regression testing can be performed on the already tested test cases in step S262. Therefore, archiving and saving the dataset information is also necessary. This allows for the classification of test cases and the archiving and saving of corresponding stimulus data.

[0108] Specifically, for steps S261-S263, if the test cases fail verification, the defective code is located using pre-agreed observation points, and then the RTL code is fixed. For large and complex logic designs, there are many interrelationships between modules. When developers are fixing code, to prevent fixing one bug from affecting other passed test cases, regression testing is necessary. This application can use the controlled variable method to assist in code debugging and fixing. After modifying the RTL code, the original test cases are backtested based on the archived data of the validation set. Only when all the data is consistent with the original data is the RTL considered successfully fixed, and the new code is merged into the code project. Otherwise, the fixing continues, and no new test case process is initiated.

[0109] Step S270: Compile the data according to the parameter configuration.

[0110] Specifically, step S270 corresponds to the later software development work in step S280. After the chip development is completed, a more general and automated chip software application design is needed. Therefore, step S270 can be a conversion and compilation tool for the inference model, realizing automatic conversion of model parameters and automatic configuration of register information. Step S270 can reuse the function of the parameter configuration module, add offset address allocation, and encapsulate it into a binary file containing the inference model and chip information, serving as a software calling interface to complete the preprocessing step of software development.

[0111] Step S280: Perform the later stage of chip software development.

[0112] See below. Figure 4 , Figure 4 This is a schematic diagram of the chip development lifecycle according to an embodiment of the present invention. Figure 4 The steps are shown as follows: S310: scheme design; S321: scheme verification; S322: functional simulation verification; S323: FPGA prototype verification; S324: compilation and development; and S330: software development.

[0113] Specifically, the verification platform entrance can accept two parameters, verification mode and model parameter address. By setting the verification mode, different code flows are entered to improve program running efficiency. The agreed code flow is as follows:

[0114] Step S321: scheme verification: the influence of software pure algorithm implementation and hardware simulation implementation scheme on model performance is evaluated, and chip scheme design is assisted. The program running in this mode ends with the output of the scheme evaluation, and no other flow is performed.

[0115] Step S322: function verification mode: the excitation data will pass through the construction of the reference model and the function simulation driving two branches, and then enter the comparison verification, and the simulation result is verified. If the output comparison is consistent, the test case that passes the verification is recorded and saved, which is used as the maintenance of the verification set of the modification benchmark in the debugging stage. If the output comparison is inconsistent, the RTL developer will track and repair the implementation process. The program running in this mode ends with the comparison output. No other flow is performed.

[0116] Step S323: FPGA prototype verification mode: the excitation data will pass through the construction of the reference model and the FPGA prototype verification driving two branches, and the same steps as in the function verification mode, enter the comparison verification module, and according to the verification result, enter the verification set maintenance. The program running in this mode also ends with the comparison output.

[0117] Step S324: compiler mode: in this mode, the excitation data can only pass through the parameter configuration module branch to directly enter the compiler module, and output a binary format file containing model parameters and chip register information. The program function in this mode is mainly to output the encapsulated file, and can not pass through other verification modules.

[0118] Specifically, in the present application, the entire verification platform can use python language and be built in pycharm development environment. In this way, each flow of the verification can be more intuitive and visible, without going through the complex compilation environment of the traditional compilation platform and language. In particular, for AI inference acceleration chip research and development, after completing a life cycle, steps S321 to S324 support parallel operation, at which time the project manager arranges personnel to work in different stages according to the project progress requirements, and the developer is more clear about his own task flow. Based on such a verification platform, the development rhythm is more compact, and the scheduling efficiency of personnel is higher.

[0119] Therefore, in the present application, the chip design cycle is shortened by referring to the model-based scheme verification function; the different stage configuration information function modules of the verification are reused by sharing the parameter configuration, and combined with the late software development requirements to improve the code reuse rate and reduce the development and maintenance cost; the verification platform is designed in the idea of AI product rapid iteration as the guide, so that several important processes of chip research and development can be configured in parallel operation, improve the personnel allocation efficiency, meet the interaction requirements between processes, and effectively shorten the research and development cycle; based on the pycharm integrated platform and the python language, the compilation time of the traditional method is removed, and the chip verification and debugging are more intuitive and efficient.

[0120] Further, in the present application, a design idea for the chip product iteration process is used, and the chip design, development, verification and compilation development personnel can share a set of platform, so that the cooperation between the front end and the back end of chip research and development is more smooth, the work is more efficient, and it is beneficial to shorten the chip research and development cycle; the verification platform is built based on the integrated development environment of pycharm using the python language, so that the whole chip verification platform is built more simply and presents more lightweight. The high cohesion and low coupling characteristics of software development are realized by controlling parameters through the program main entrance, and the consumption of verification process resources is reduced.

[0121] The above only describes the multiple implementation modes of the present application, and each embodiment can be realized alone or in combination, and the present application is not limited in this way.

[0122] As shown in Figure 5 The present application also provides a chip verification system for realizing the chip verification method, and the system 400 comprises:

[0123] The excitation generation module 410 is configured to generate excitation data according to the model configuration parameters;

[0124] The reference model module 420 is configured to establish a software reference model and a hardware simulation reference model;

[0125] The reference model evaluation module 430 is configured to input the excitation data into the software reference model and the hardware simulation reference model respectively, and obtain a software reference output and a hardware simulation reference output; based on the comparison between the software reference output and the hardware simulation reference output, it is determined whether the hardware simulation reference model passes the reference model evaluation;

[0126] The simulation module 440 is configured to perform simulation verification according to the excitation data, and obtain a simulation output;

[0127] The verification result module 450 is configured to compare the hardware simulation reference output and the simulation output to obtain a chip verification result in response to the hardware simulation reference model passing the reference model evaluation.

[0128] The chip verification system of the present application can be deployed on a PC or a server to build a chip verification environment. The chip verification environment establishes a software reference model and a hardware emulation reference model, and inputs excitation data into the software reference model and the hardware emulation reference model respectively, to determine whether the hardware emulation reference model passes the reference model evaluation based on comparison of the software reference output and the hardware emulation reference output. The present application can realize automatic evaluation of the hardware emulation reference model. When the hardware emulation reference model fails the evaluation, the hardware emulation reference model can be directly adjusted without performing subsequent simulation verification. When the hardware emulation reference model passes the evaluation, subsequent simulation verification can be performed based on the hardware emulation reference model. In this way, chip verification is optimized to reduce chip verification complexity, improve the flexibility and expansibility of chip verification, and reduce the redundancy of chip verification.

[0129] The functions of each module in the chip verification system of the present application can be realized by the specific embodiments of each step described above, which will not be repeated here.

[0130] As shown in FIG. 4, it is an interaction diagram of each module of the specific embodiment of the present application: Figure 6

[0131] The excitation generation module 410 is configured to generate first excitation data and second excitation data according to model configuration parameters.

[0132] The reference model module 420 is configured to establish a software reference model and a hardware emulation reference model.

[0133] The reference model output module 430 is configured to input the excitation data into the software reference model and the hardware emulation reference model respectively, and obtain a software reference output and a hardware emulation reference output. Based on comparison of the software reference output and the hardware emulation reference output, it is determined whether the hardware emulation reference model passes the reference model evaluation.

[0134] Then, it is determined whether the hardware emulation reference model passes the reference model evaluation. If it does not pass the reference model evaluation, the reference model module 420 is returned to reestablish the model. If it passes the reference model evaluation, the next step is comparison and verification.

[0135] Specifically, while the reference model module 420 establishes the software reference model and the hardware emulation reference model, the parameter configuration module 460 can perform parameter configuration simultaneously.

[0136] The functional simulation verification module 441 is configured to perform register transfer level simulation verification according to the excitation data, and obtain a register transfer level simulation output.

[0137] ​An FPGA prototype verification module 442 is configured to perform FPGA prototype simulation verification according to the excitation data, and obtain FPGA prototype simulation output.

[0138] Then, the functional simulation verification module 441 and the functional simulation verification module 441 are compared with the output of the hardware reference model. If consistent, the code (register transfer level code) of the simulation verification is maintained by the verification set maintenance module 491. If inconsistent, the code (register transfer level code) of the simulation verification is modified by the simulation verification modification module 493. Then, regression testing 492 is performed.

[0139] The compiler module 480 is configured to perform data compilation according to the parameter configuration.

[0140] Figure 5 And Figure 6 The chip verification system provided by the present application is only illustrative, and the splitting, merging and adding of the modules are within the protection scope of the present application without departing from the concept of the present application. The chip verification system provided by the present application can be realized by software, hardware, firmware, plug-ins and any combination thereof, and the present application is not limited thereto.

[0141] The present application also provides a chip verification device, comprising a processor, a memory having executable instructions of the processor stored therein, wherein the processor is configured to execute the steps of the chip verification method by executing the executable instructions.

[0142] Those skilled in the art can understand that each aspect of the present application can be implemented as a system, a method or a program product. Therefore, each aspect of the present application can be specifically implemented as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software, which can be collectively referred to as "circuitry", "module" or "platform".

[0143] The electronic device 600 according to this embodiment of the present application will be described below with reference to Figure 7 Figure 7 The electronic device 600 shown is merely an example and should not limit the function and use range of the embodiments of the present application.

[0144] As shown in Figure 7 The electronic device 600 is in the form of a general computing device. The components of the electronic device 600 can include, but are not limited to, at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc. ​

[0145] The storage unit stores program codes which can be executed by the processing unit 610, so that the processing unit 610 performs the steps described in the chip verification method part of the present specification according to various exemplary embodiments of the present application. For example, the processing unit 610 can perform the steps as shown in the flowchart of FIG. 6. Figure 1

[0146] The storage unit 620 can include readable media in the form of volatile storage units, such as random access memory (RAM) 6201 and / or cache memory 6202, and can further include a read-only memory (ROM) 6203.

[0147] The storage unit 620 can further include program / utility 6204 having a set of programs / modules 6205, including an operating system, one or more application programs, other programs, and programmatic data, each or some combination thereof, which can include implementation of a network environment.

[0148] The bus 630 can be representative of one or more of several types of bus structures, including a storage unit bus or bus controller, a peripheral bus, a graphics acceleration port, a processing unit bus, or a local bus using any of a variety of bus architectures.

[0149] The electronic device 600 can also communicate with one or more external devices 700 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices that enable a user to interact with the electronic device 600; and / or one or more devices that enable the electronic device 600 to communicate with one or more other computing devices. Such communication can be facilitated by an Input / Output (I / O) interface 650. The electronic device 600 can also communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the public network, such as the Internet, via a network adapter 660. The network adapter 660 can be communicatively coupled to the other components of the electronic device 600 via the bus 630. It should be appreciated that other hardware and / or software modules can be used in conjunction with the electronic device 600, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0150] In the chip verification device, the program in the memory is executed by the processor to implement the steps of the chip verification method, so that the device can also obtain the technical effects of the chip verification method.

[0151] ​The embodiments of the present application also provide a computer readable storage medium for storing a program, which, when executed by a processor, implements the steps of the chip verification method. In some possible implementation manners, various aspects of the present application can also be implemented as a program product in the form of a computer program code, which, when executed on a terminal device, causes the terminal device to perform the steps of the chip verification method according to various exemplary embodiments of the present application described in the foregoing method part of the specification.

[0152] Reference Figure 8 As shown, a program product 800 for implementing the above method according to the embodiments of the present application is described, which can adopt a portable compact disc read-only memory (CD-ROM) and include program code, and can be executed on a terminal device, such as a personal computer. However, the program product of the present application is not limited to this, and in the present document, the readable storage medium can be any tangible medium containing or storing a program, which can be used or combined with an instruction execution system, device or apparatus.

[0153] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0154] The computer readable storage medium can include a data signal carried in the baseband or as a part of a carrier wave, in which readable program codes are borne. Such a propagated data signal can take on multiple forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium can also be any readable medium that is not a readable storage medium, which can send, propagate or transmit programs for use by or in connection with an instruction execution system, device or apparatus. The program codes contained on the readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0155] The program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0156] The program in the computer storage medium is executed by the processor to realize the steps of the chip verification method, and therefore, the computer storage medium can also obtain the technical effects of the chip verification method.

[0157] The chip verification method, system, device and storage medium of the present application have the following beneficial effects:

[0158] The present application can realize automatic evaluation of the hardware simulation reference model. When the hardware simulation reference model fails to pass the evaluation, the hardware simulation reference model can be directly adjusted without performing subsequent simulation verification. When the hardware simulation reference model passes the evaluation, subsequent simulation verification can be performed based on the hardware simulation reference model. In this way, chip verification is optimized to reduce chip verification complexity, improve chip verification flexibility and expansibility, and reduce chip verification redundancy.

[0159] The above is a further detailed description of the present application in combination with specific preferred embodiments, and the specific implementation of the present application should not be limited to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, several simple deductions or replacements can be made without departing from the concept of the present application, and all of them should be considered as falling within the protection scope of the present application.

Claims

1. A chip verification method characterized by, The method comprises the following steps: According to the model configuration parameters, the excitation data is generated, which comprises generating first excitation data and second excitation data, wherein the first excitation data is generated based on the trained model, and the second excitation data is generated based on a random function; A software reference model and a hardware-in-the-loop reference model are established, and the software reference model, the hardware-in-the-loop reference model and the simulation verification all use the second excitation data, and the first excitation data is used for software development performance verification of the chip; the parameter configuration of the model is performed according to the excitation data, and data compilation is performed according to the parameter configuration, so as to execute software development of the chip; The excitation data is input into the software reference model and the hardware-in-the-loop reference model respectively, the software intermediate layer output of the software reference model and the hardware-in-the-loop intermediate layer output of the hardware-in-the-loop reference model are obtained, the intermediate output difference of the software intermediate layer output and the hardware-in-the-loop intermediate layer output is obtained, the software performance parameter of the software reference model and the hardware-in-the-loop performance parameter of the hardware-in-the-loop reference model are obtained, the performance difference of the software performance parameter and the hardware-in-the-loop performance parameter is obtained, and the intermediate layer with the greatest impact on the performance difference is located according to the functional relationship of the intermediate output difference and the performance difference, whether the intermediate layer of the hardware-in-the-loop reference model passes the reference model evaluation is judged, when the difference value of the overall output of the software reference model and the overall output of the hardware-in-the-loop reference model is greater than a set threshold range, the hardware-in-the-loop reference model fails the reference evaluation, and when the difference value of the overall output of the software reference model and the overall output of the hardware-in-the-loop reference model falls within the set threshold range, the hardware-in-the-loop reference model passes the reference evaluation; According to the excitation data, field programmable gate array prototype simulation verification is performed, and field programmable gate array prototype simulation output is obtained; In response to the hardware-in-the-loop reference model passing the reference model evaluation, the hardware-in-the-loop reference output and the simulation output are compared, and a chip verification result is obtained.

2. The chip verification method according to claim 1, wherein In response to the chip verification result being verification failure, the code of the simulation verification is modified for regression testing; In response to the chip verification result being verification pass, the code of the simulation verification is saved for regression testing.

3. A chip verification system, characterized by, It comprises: The excitation generation module generates excitation data according to model configuration parameters, which comprises generating first excitation data and second excitation data, wherein the first excitation data is generated based on the trained model, and the second excitation data is generated based on a random function; The reference model module establishes a software reference model and a hardware-in-the-loop reference model, and the software reference model, the hardware-in-the-loop reference model and the simulation verification all use the second excitation data, and the first excitation data is used for software development performance verification of the chip; the parameter configuration of the model is performed according to the excitation data, and data compilation is performed according to the parameter configuration, so as to execute software development of the chip; The reference model evaluation module inputs the excitation data into the software reference model and the hardware-in-the-loop reference model respectively, obtains software intermediate layer output of the software reference model and hardware-in-the-loop intermediate layer output of the hardware-in-the-loop reference model, obtains intermediate output difference of the software intermediate layer output and the hardware-in-the-loop intermediate layer output, obtains software performance parameter of the software reference model and hardware-in-the-loop performance parameter of the hardware-in-the-loop reference model, obtains performance difference of the software performance parameter and the hardware-in-the-loop performance parameter, locates to an intermediate layer with the greatest impact on performance difference according to a functional relationship between the intermediate output difference and the performance difference, judges whether the intermediate layer of the hardware-in-the-loop reference model passes the reference model evaluation, and when a difference value between overall output of the software reference model and overall output of the hardware-in-the-loop reference model is greater than a set threshold range, the hardware-in-the-loop reference model does not pass the reference evaluation, and when the difference value falls within the set threshold range, the hardware-in-the-loop reference model passes the reference evaluation. The simulation module performs field programmable gate array prototype simulation verification according to the excitation data, and obtains field programmable gate array prototype simulation output. The verification result module compares the hardware-in-the-loop reference output and the simulation output to obtain a chip verification result in response to the hardware-in-the-loop reference model passing the reference model evaluation.

4. A chip verification apparatus characterized by comprising: The chip verification method comprises the following steps: a processor; a memory having executable instructions of the processor stored therein; wherein the processor is configured to execute the executable instructions to perform the steps of the chip verification method of claim 1 or 2.

5. A computer readable storage medium for storing a program, characterized in that, The program is executed by the processor to implement the steps of the chip verification method of claim 1 or 2.

Images