Chip automatic verification method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides an automatic chip verification method and device, a storage medium and electronic equipment, and relates to the technical field of computers.The method comprises the steps that a research and development code of a to-be-verified chip is obtained, a module corresponding to the research and development code is determined, and a simulation path of the module is determined; determining a simulation folder of the module according to the simulation path; according to the at least one test case code and the verification environment file in the simulation folder, parallel compiling is carried out, and at least one compiling file used for chip simulation is obtained and stored respectively; based on the regression simulation command, simulating the at least one compiled file in parallel on the server cluster to obtain a simulation result file; and performing coverage rate analysis and regression result analysis according to the simulation result file to obtain an analysis result and generate an analysis result report file. In this way, compiling and simulation are fully automatically completed, dynamic distribution of the test cases is conducted on server cluster resources, and verification efficiency is improved on the basis that verification comprehensiveness is guaranteed.

Description

Technical Field

[0001] The present disclosure relates to the field of chip automatic verification technology, and in particular to a chip automatic verification method, device, storage medium and electronic device. Background Art

[0002] Chip verification is a core step in the integrated circuit design and manufacturing process, crucial for ensuring design correctness, functional integrity, and final product reliability. Chip verification requires in-depth analysis of key design test scenarios through large-scale simulation and simulation, making regression testing an essential component of this process. In the middle and late stages of the verification phase, running a massive number of test cases across diverse verification scenarios effectively identifies potential defects in the design. At the same time, the increasing complexity of chips presents unprecedented challenges to verification, placing immense pressure on regression testing of these test cases.

[0003] With the increasing complexity of circuit designs and advancements in process nodes, the time and resources consumed by the verification phase are increasing. This not only prolongs chip development cycles but also increases the difficulty of regression testing and coverage convergence. Throughout the chip verification process, regression simulation of test cases is highly dependent on human resources. Furthermore, due to the difficulty in automatically balancing simulation resources, this not only consumes a significant amount of engineers' time and energy, but also occupies and wastes a significant amount of simulation resources.

[0004] On the one hand, regression testing generates a large number of logs, requiring engineers to expend considerable time and effort to locate error messages and their causes. Furthermore, complex scenarios typically require thousands of test cases for full coverage. Manually completing regression tests requires not only manual code management and execution of every simulation command, but also a balanced allocation of server resources. Finally, the collection, merging, and analysis of coverage for different versions of design code consumes significant time and effort.

[0005] Therefore, how to efficiently complete the regression simulation testing and coverage collection of test cases while ensuring the comprehensiveness of verification, and thus accelerate the convergence of verification, is a key issue that needs to be solved urgently. Summary of the Invention

[0006] The present disclosure provides a chip automatic verification method, device, storage medium and electronic device to at least solve the above technical problems existing in the prior art.

[0007] The technical solution of the embodiment of the present disclosure is implemented as follows: In a first aspect, an embodiment of the present disclosure provides a chip automatic verification method, which is applied to a verification platform. The method includes: Obtaining the R&D code of the chip to be verified, determining the module corresponding to the R&D code and determining the simulation path of the module; Determining a simulation folder of the module according to the simulation path, wherein the simulation folder includes at least one test case code and a verification environment file for the module; Compile in parallel according to at least one test case code and verification environment file in the simulation folder to obtain at least one compiled file for chip simulation and store them separately; Based on the regression simulation command, simulating the at least one compiled file in parallel on the server cluster to obtain a simulation result file; Perform coverage analysis and regression result analysis based on the simulation result file to obtain analysis results and generate an analysis result report file.

[0008] In a second aspect, an embodiment of the present disclosure provides a chip automatic verification device, which is applied to a verification platform and includes: An acquisition module is used to obtain the R&D code of the chip to be verified, determine the module corresponding to the R&D code, and determine the simulation path of the module; A first processing module is configured to determine a simulation folder of the module according to the simulation path, wherein the simulation folder includes at least one test case code and a verification environment file for the module; Compile in parallel according to at least one test case code and verification environment file in the simulation folder to obtain at least one compiled file for chip simulation and store them separately; A second processing module is configured to simulate the at least one compiled file in parallel on the server cluster based on the regression simulation command to obtain a simulation result file; The third processing module is used to perform coverage analysis and regression result analysis according to the simulation result file, obtain analysis results and generate an analysis result report file.

[0009] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the chip automatic verification methods described.

[0010] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute any one of the chip automatic verification methods.

[0011] The embodiments of the present disclosure have the following beneficial effects: The chip automatic verification method, device, storage medium and electronic device provided by the embodiment of the present disclosure are applied. The method includes: obtaining the R&D code of the chip to be verified, determining the module corresponding to the R&D code and determining the simulation path of the module; determining the simulation folder of the module according to the simulation path, the simulation folder including at least one test case code and verification environment file for the module; performing parallel compilation according to the at least one test case code and verification environment file in the simulation folder to obtain at least one compiled file for chip simulation and store them separately; based on the regression simulation command, simulating the at least one compiled file in parallel on the server cluster to obtain a simulation result file; performing coverage analysis and regression result analysis according to the simulation result file to obtain analysis results and generate an analysis result report file. In this way, compilation and simulation are completed fully automatically, and the results are merged and analyzed. At the same time, test cases are dynamically allocated to server cluster resources, which improves verification efficiency while ensuring the comprehensiveness of verification and greatly saves labor costs and time.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of a process flow of a chip automatic verification method provided by an embodiment of the present disclosure; Figure 2 A schematic diagram of a process flow of a fully automatic chip parallel verification method based on continuous integration provided in an application embodiment of the present disclosure; Figure 3 A schematic diagram of the structure of a chip automatic verification device provided by an embodiment of the present disclosure; Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0014] To make the purposes, features, and advantages of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative work shall fall within the scope of protection of the present disclosure.

[0015] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0016] If similar descriptions of "first / second" appear in the application documents, the following explanation is added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0018] Figure 1 A schematic diagram of a chip automatic verification method provided by an embodiment of the present disclosure is shown in FIG. Figure 1 As shown, the method is applied to a verification platform, and the chip automatic verification method includes: Step 101: Obtain the R&D code of the chip to be verified, determine the module corresponding to the R&D code, and determine the simulation path of the module; Step 102: determining a simulation folder of the module according to the simulation path, wherein the simulation folder includes at least one test case code and a verification environment file for the module; Step 103: Compile in parallel according to the at least one test case code and the verification environment file in the simulation folder to obtain at least one compiled file for chip simulation and store them separately; Step 104: Based on the regression simulation command, simulate the at least one compiled file in parallel on the server cluster to obtain a simulation result file; Step 105: Perform coverage analysis and regression result analysis based on the simulation result file to obtain analysis results and generate an analysis result report file.

[0019] In some embodiments, the R&D code is the source code of the chip design, which is usually written in a hardware description language (such as Verilog, VHDL, etc.) and is used to define various functions and behaviors of the chip.

[0020] The R&D code can correspond to one or more modules. Specifically, the chip design code can be divided into different modules. For example, a chip may contain a processor module, a memory module, an interface module, etc., and each module can have different R&D code in the design.

[0021] Test case code is a verification script written to test whether a module's functionality works as expected. Test cases are verifications performed on a module and typically include input data, expected results, and verification logic.

[0022] A module can have one or more test cases (i.e., at least one test case code), each used to check a different function or behavior of the module. For example, for a processor module, test cases might include checking that the adder works correctly, that the multiplier handles large numbers correctly, and that the interrupt mechanism is functioning properly. Therefore, a module can have multiple test cases, and the test case code is placed in the simulation folder.

[0023] Test case code and the corresponding verification environment files can be organized into a simulation folder to facilitate efficient testing during simulation. Specifically, the simulation folder can include at least one test case code and verification environment file. If there are multiple test case codes, they can be compiled in parallel to generate multiple compiled files that are stored separately.

[0024] Verification environment files are configuration files or support files used with test cases. They provide a runtime environment for the test cases, allowing simulation to proceed under the correct conditions. For example, verification environment files can include simulation clock settings, inter-module interface configuration, and test activation conditions.

[0025] In actual application, engineers can develop test case code and optimize the verification environment locally according to the verification plan and functional requirements. After compiling and simulating the test case code locally to confirm that it is correct, they obtain the project's R&D code and submit the R&D code to a remote repository, such as the Gerrit (a code version control platform) remote repository, to complete the submission of the R&D code.

[0026] To ensure the efficiency of the entire development, testing, and verification process, an automated code detection function is provided. Based on this, in some embodiments, before obtaining the R&D code of the chip to be verified, the method further includes: Performing automated testing on the R&D code; If the detection fails, determining simulation error information, performing error analysis on the simulation error information to obtain the error cause, and feeding back and / or presenting the simulation error information and the error cause; If the detection passes, parallel compilation is performed according to the project code to obtain at least one compiled file.

[0027] Here, an automated inspection method is provided, which is used to automatically trigger Jenkins (a continuous integration tool) to check Sanity test (quick test) cases after the engineer submits the project code to the remote warehouse. The inspection content includes syntax checking, basic smoke testing, basic use case testing, etc. If the inspection fails, the simulation error information of the test case can be automatically analyzed, the specific reason can be given, and the error information and the cause of the error can be fed back to the engineer in time so that the engineer can re-build and optimize the verification environment. Here, the simulation error information is analyzed to obtain the cause of the error, and the error keywords can be extracted using regular expressions. The error cause can be determined by analysis based on the error keywords.

[0028] Sanity test cases can be used to test whether certain basic functions and core functions are normal, eliminate some common simple errors or problems, and thus improve the efficiency of subsequent verification.

[0029] In some embodiments, obtaining the R&D code of the chip to be verified, determining the module corresponding to the R&D code, and determining the simulation path of the module include: The verification platform obtains a project code identifier (such as an ID); the project code identifier is used to identify the R&D code; The R&D code is obtained from the remote warehouse according to the project code identifier, and the module and the simulation path of the module corresponding to the obtained R&D code are identified.

[0030] After confirming the module corresponding to the R&D code, switch to the corresponding simulation path and execute the simulation folder (Makefile).

[0031] Here, the verification platform can be a fully automated verification platform built based on an integrated tool (Jenkins) and Python. The verification platform can obtain the code identifier in any manner, such as receiving user input through an interactive interface, or receiving instructions from other channels (the instructions carry the code identifier).

[0032] In some embodiments, there is at least one module, and each module corresponds to a different verification environment; the simulation folder also includes compilation options; Compiling in parallel according to the at least one test case code and the verification environment file includes: Based on at least one compilation option, parallel compilation is performed according to the at least one test case code and the verification environment file; wherein each of the compilation options is used to instruct the module to perform a different operation; The compilation includes at least one of the following: compilation of design code, compilation of verification IP code, and compilation of verification environment.

[0033] Here, a module can have multiple compilation options, each compilation option is used to instruct the module to perform a different operation, for example, compilation option 1 is to perform the data moving function, compilation option 2 is to move data first and then read data, and compilation option 3 is the reading function.

[0034] Each of the compilations includes at least one of the following: compilation of design code, compilation of verification IP (Verification IP) code, and compilation of verification environment. Specifically, after switching to the corresponding simulation path, the subprocess library function of the verification platform can be called to automatically perform compilation.

[0035] By executing these compilation tasks in parallel, you can generate the compilation files (such as simv files) required for chip simulation. For chip automatic verification simulation, different compilation options will generate different compilation files. Using storage variables to distinguish and store these compilation files separately ensures smooth parallel compilation and simulation execution.

[0036] In some embodiments, the method further comprises: Obtain a target file, the target file being used to store at least one set of command information; each set of command information including at least one of the following: a test case name, a number of simulation cycles, simulation options, compilation options, a timeout setting, and a coverage option; Parsing each set of command information in the target file, and generating a regression simulation command corresponding to each set of command information according to a command generation rule; Add the command for generating the regression simulation to the regression test file.

[0037] Here, the information of the simulation commands of each test case is managed using a target table (such as a table in Excel format); in this way, the information of each simulation command can be presented intuitively based on the target format, and it is convenient for engineering personnel to perform screening and statistics, as well as dynamically add and / or delete test cases and / or simulation commands according to verification requirements.

[0038] The information of the simulation command is an option for configuring simulation behavior and test parameters when executing the simulation.

[0039] The test case name is the name of the test script for a function or module to be verified. It specifies the specific test case to be executed. Each test case tests a specific module function or behavior. The simulation command must specify which test case to run.

[0040] The number of simulation cycles indicates how many cycles the simulation will run. A simulation cycle is usually one cycle of the simulation clock and determines how long the simulation will run.

[0041] Simulation options refer to some settings and configurations during simulation, which may include simulation mode (such as forward simulation, regression simulation), simulation accuracy, etc.

[0042] Compilation options refer to the configuration and parameters used when compiling the design code before simulation. For example, you can specify the compiler optimization level, debug information generation, and compile path settings.

[0043] The timeout setting refers to the maximum allowed running time during the simulation. If the simulation exceeds this time limit, the simulation tool will automatically terminate the simulation to prevent the simulation from running indefinitely.

[0044] Coverage options are used to configure code coverage analysis during simulation. For example, you can configure the simulation tool to check which lines of code or conditions were tested and which were not. Coverage analysis allows you to assess the comprehensiveness of your tests and ensure that all functionality has been verified.

[0045] After obtaining the target table, the pandas library of the verification platform can be used to parse the information of each simulation command to obtain the corresponding test case name, simulation command, compilation options, etc.; these parameters are placed in a dictionary and assembled into executable regression simulation commands according to command generation rules (such as according to a fixed format), and the executable regression simulation commands are added to the command list (cmd_list), and the command list is randomized and output to the regression test file (recfg.ini).

[0046] In some embodiments, the verification platform has a server cluster, and the parallel simulation of the at least one compiled file on the server cluster includes: Distributing the regression simulation commands contained in the regression test file to the servers in the server cluster; The servers in the server cluster execute corresponding regression simulation commands in parallel.

[0047] Here, the verification platform has a server cluster, which includes multiple server nodes. In order to achieve efficient simulation, a parallel compilation method is provided, which can be achieved by managing the resources of the server cluster and distributing regression simulation commands.

[0048] Here, resource management of the server cluster may include: pooling the server nodes (hereinafter referred to as servers) of the server cluster, supporting dynamic addition and deletion of servers, and automatically allocating and recycling resources as needed.

[0049] The distribution of regression simulation commands can include: using Jenkins (continuous integration tool) for dynamic distribution, assigning the regression simulation commands for each test case in the regression test file to the corresponding server to achieve load balancing and workload isolation, and recording the status information of each server and feeding it back to the engineer. For example, the status information that can be recorded includes: server IP, server owner, running time, etc.

[0050] In some embodiments, distributing the regression simulation command contained in the regression test file to the servers in the server cluster includes: Determine an end time corresponding to each regression simulation command, and distribute each regression simulation command to different servers according to the end time with the goal of load balancing and / or minimizing simulation time; Each of the servers distributes one or more regression simulation commands.

[0051] Here, considering that the same module may have multiple test case codes, multiple test case codes may be simulated in parallel according to different regression simulation commands.

[0052] Each regression simulation command may correspond to a latest end time (such as the above-mentioned timeout setting), and the end time of the regression simulation task corresponding to each regression simulation command may also be predicted based on factors such as the complexity of the simulation task and computing resource requirements.

[0053] Load balancing means that the goal of distribution is to make the workload of each server as even as possible, avoiding some servers being overloaded while others are idle.

[0054] Minimum simulation time means shortening the total simulation completion time as much as possible while taking load balancing into consideration. That is, when running tasks in parallel on multiple servers, the total time of the entire simulation process is minimized.

[0055] In this way, the task allocation strategy is determined based on the end time (i.e., estimated completion time) of each regression simulation task. Furthermore, to achieve load balancing, simulation commands are distributed to different servers based on task end time. This ensures a more balanced load (i.e., the number of simulation tasks) on each server, improving resource utilization and reducing overall simulation time.

[0056] In some embodiments, the servers in the server cluster execute corresponding regression simulation commands in parallel, including: The servers in the server cluster execute at least one regression simulation command corresponding to each module in parallel according to the compiled file corresponding to the server; The parallel execution of at least one regression simulation command corresponding to each module includes: Each server in the server cluster executes in parallel at least one regression simulation command corresponding to the first module; If at least one regression simulation command corresponding to the first module is executed, at least one regression simulation command corresponding to the second module is executed in parallel; wherein, the second module is any unverified module except the first module.

[0057] Here, a server cluster includes multiple servers for distributed computing and task processing. The parallel processing of multiple servers in the cluster can accelerate the execution of simulation tasks.

[0058] The verification task for each module can include multiple regression simulation commands, which are used to verify the functionality of a specific module and ensure that the module still works properly after changes are made.

[0059] If multiple modules require verification, they are verified sequentially, with the first module being the one requiring verification and the second module being the one after which verification has not yet occurred. Multiple regression simulation commands for each module are executed sequentially, following the module order. During execution, each server, based on task allocation, will simultaneously execute multiple regression simulation commands for the first module. Once the regression simulation command for the first module completes, the server will continue to execute the regression simulation command for the second module in parallel. Other unverified modules (i.e., modules other than the first module) will continue to be simulated following a similar process.

[0060] In some embodiments, distributing the regression simulation command contained in the regression test file to the servers in the server cluster includes: The number of the modules is determined, the server cluster is grouped according to the number of the modules, a server sub-cluster corresponding to each module is determined, and the regression simulation command corresponding to each module is distributed to the server sub-cluster corresponding to each module.

[0061] Here, it is considered that there may be multiple modules that need to be tested during the regression simulation process, and each module may have one or more regression simulation commands.

[0062] A server cluster composed of multiple servers needs to be reasonably allocated as a resource pool for executing assigned simulation tasks. Therefore, a distribution method is proposed here.

[0063] According to the number of modules and the regression simulation commands of each module, the regression simulation commands of each module are assigned to a specific server sub-cluster. This ensures that the regression simulation commands of each module are executed by a dedicated server sub-cluster, avoiding interference and resource competition between different modules.

[0064] This allocation process can be adjusted based on the server load, performance, and module complexity to balance the workload of each sub-cluster. It should be noted that, for each module, the method of distributing the regression simulation command corresponding to each module to the server sub-cluster corresponding to each module may also include: determining the end time corresponding to each regression simulation command, and distributing each regression simulation command to different servers in the server sub-cluster according to the end time with the goal of load balancing and / or minimizing simulation time.

[0065] In some embodiments, the servers in the server cluster execute corresponding regression simulation commands in parallel, including: The servers in the server cluster execute one or more regression simulation commands corresponding to multiple modules in parallel according to the compiled files corresponding to the servers; The step of executing one or more regression simulation commands corresponding to multiple modules in parallel includes: Each server in the server sub-cluster executes the 1st to Nth regression simulation commands corresponding to the module in parallel, and the server sub-cluster includes N servers, where N is greater than or equal to 1.

[0066] After the Nth regression simulation command is executed, the N+1th to 2Nth regression simulation commands are executed in parallel; The process loops sequentially until all regression simulation commands are executed.

[0067] Each server sub-cluster includes one or more servers, and multiple regression simulation commands of a module are distributed to the servers in the corresponding server sub-cluster, and these servers process different regression simulation commands simultaneously.

[0068] Assume that each server subcluster consists of N servers. These N servers will execute regression simulation commands 1 through 2 for a module in parallel to ensure efficient task execution. After the 1 through 2 regression simulation commands are completed, the N+1 through 2N regression simulation commands will be executed in parallel. This execution loop continues, executing N regression simulation commands at a time until all commands are completed. This loop ensures that each batch of regression simulation commands is executed, and as each batch of tasks completes, new commands continue to be executed in parallel.

[0069] In this way, multiple server subclusters can simultaneously and concurrently process the regression simulation commands of multiple modules, which can significantly improve the execution efficiency of the regression simulation and shorten the overall testing time.

[0070] It should be noted that the time required to execute each regression simulation command is different. The above-mentioned loop or continuing to execute the next regression simulation command means that each server can execute the next regression simulation command after the execution is completed. It does not mean that each server needs to wait for a batch of regression simulation commands (such as the 1st to Nth regression simulation commands) to be fully executed before parallel processing of the next batch of regression simulation commands.

[0071] In some embodiments, the servers in the server cluster execute corresponding regression simulation commands in parallel, including: Each server determines a compilation option according to the regression simulation command, and enters a corresponding simulation path according to the compilation option to load a stored compilation file; Each server executes the corresponding regression simulation command in parallel according to the compiled file, and monitors the simulation status in real time according to the simulation output pipeline information during the simulation process and records it until the simulation ends, thereby obtaining a simulation result file.

[0072] Specifically, for the parallel execution of regression simulation commands, after the regression test is ready, the platform enters the corresponding simulation folder and loads the pre-generated simulation compilation file (such as a simv format file).

[0073] Here, depending on the compilation options, we load simv files stored in different locations and use Python's subprocess library to execute regression simulation commands in parallel. By monitoring the simulation output in real time, if any problems arise during the simulation, the problematic simulation is automatically terminated to ensure the accuracy of the test process.

[0074] In some embodiments, the simulation result file includes: a simulation log; The coverage analysis and regression result analysis are performed according to the simulation result file to obtain the analysis results and generate an analysis result report file, including: Determine the test case name, seed number, and / or simulation time based on the simulation log; Match error information according to the simulation log and determine the error information matching result; compare the error information matching result with the error threshold and determine whether the simulation passes according to the comparison result; Generate an analysis result report file based on the test case name, seed number, simulation time, simulation result and / or the matching error information; The simulation result includes simulation pass (pass), simulation failure (fail), simulation timeout (timeout) or active exit (quit).

[0075] Here, the error information may include UVM_ERROR, UVM_FATAL, etc., and may also include user-defined error information such as ERROR, SVA ERROR, etc.

[0076] After identifying relevant information and error messages in the simulation result file, the system uses error thresholds to determine whether the simulation passed. The simulation results are collated and output to the simulation log, and the simulation analysis results are also fed back to the engineer. After the simulation is complete, Python automatically analyzes the generated simulation log, searching for specific error messages (such as UVM_ERROR and UVM_FATAL) or user-defined error messages (such as ERROR and SVA ERROR). By determining the error threshold, the system determines whether the simulation passed. The analysis results are collated and output to a report file for feedback to the engineer for subsequent review and analysis to identify any issues.

[0077] In addition, the platform can also generate a result report based on the analysis result report file, and count the simulation results of each test case. This data provides a basis for the subsequent verification process, ensuring that the passing status of each test case is recorded and analyzed in a timely manner.

[0078] In some embodiments, the simulation result file includes: a coverage file, the coverage file including: a coverage file obtained by at least one simulation for each module; Perform coverage analysis and regression result analysis based on the simulation result file, obtain analysis results and generate an analysis result report file, and further include: Obtain at least one coverage file of each module, and merge the at least one coverage file to obtain a merged file; Storing the merged file at a first address, reading coverage data from the merged file, and generating a coverage data file; The coverage data includes at least one of the following: module name, line coverage, state machine (FSM) coverage, flip coverage, condition coverage, and assertion coverage.

[0079] Here, the verification platform's subprocess and OS libraries can be used to parse the user-defined coverage urg option file (containing coverage-related options for regression testing and simulation), generate the coverage urg command, and execute the merge command to merge at least one coverage file. Simultaneously, the verification platform's simulation tool (such as the VCS (Verilog Compiler Simulator)) is called to read coverage data, such as module name, line coverage, FSM coverage, flip coverage, condition coverage, and assertion coverage, and output it to a coverage data file. The module name indicates the module for which the coverage data corresponds. Line coverage indicates whether a line of code has been executed, measuring the degree of code coverage. FSM coverage indicates coverage of the state machine. Flip coverage checks whether test cases trigger state transitions. Condition coverage checks whether different results of conditional expressions in the program are covered. Assertion coverage checks whether assertions are triggered.

[0080] At the same time, the platform supports multiple functions, such as whether to merge in parallel (support parallel merging of multiple coverage data files to improve processing speed), whether to run the program in 64-bit (support running in a 64-bit system environment to process larger data sets), whether to output reports (support generating coverage report files to summarize coverage results during the test process), etc. In some embodiments, the method further comprises: The reporting tool determines the report template and notification method selected by the user; generating a target report based on the report template and the coverage data; The relevant files of the target report are sent to the receiving end in the selected notification mode; the relevant files of the target report include at least one of the following: project name, target report, analysis result report file, coverage data file, and project status.

[0081] Here, the project status can be the simulation status of the entire regression use case, which can include the total number of test cases, the number of passes, the number of fails, the number of cases being simulated, etc. Specifically, the verification platform is used to extract the analysis result information, organize and summarize it, and output it into a target report. The target report includes simulation information for each test case and summarizes the simulation status of the entire regression use case in real time.

[0082] Here, the notification method can be email, SMS, notification to check messages, etc., and the relevant files of the target report are generated in a unified format based on the analysis result report file and the coverage data file.

[0083] Taking email as an example, build a report distribution task and set the email address, subject, project build time, project name, data report path, etc. in Jenkins. After the regression simulation is completed, the data report will be automatically sent to the relevant engineers through the built-in server email system.

[0084] The method provided by the disclosed embodiments requires users to simply provide the corresponding files and utilize the verification platform to complete fully automatic regression compilation. Through highly automated processes and intelligent analysis tools, the efficiency and quality of chip verification are significantly improved, human errors are reduced, the smooth progress of the project is ensured, verification convergence is accelerated, and the chip development cycle is significantly shortened.

[0085] Figure 2 A flow chart of a fully automatic chip parallel verification method based on continuous integration provided in an application embodiment of the present disclosure is shown as follows: Figure 2 As shown, the method includes: a fully automatic compilation process, a fully automatic simulation and result recording process, and a fully automatic report generation process; wherein the fully automatic compilation process includes: Step 201: After the test case is developed, it is locally checked, compiled, and simulated. If it passes, the R&D code is submitted to the Gerrit repository; Step 202: Perform a sanity use case check on the R&D code; Step 203: Determine whether the inspection is passed. If not, proceed to step 204; if passed, proceed to step 205. Step 204: Automatically analyze the compilation simulation problem and feed back the problem analysis results to the engineer for rebuilding the verification environment; Here, after the engineer completes code development and completes basic compilation and simulation checks locally, and uploads the code to the Gerrit repository, the Jenkins check operation is automatically triggered to perform a sanity use case check on the R&D code, including: compilation and simulation of basic use cases. If the check fails, the compilation and simulation problems will be automatically analyzed, and the problem analysis results will be fed back to the engineer for rebuilding the verification environment.

[0086] Step 205: Trigger Jenkins operation to obtain the R&D code and automatically execute the compilation of the verification project and the storage of the compiled files; Here, Python is used to identify the simulation path of the module corresponding to the submitted code, and after identifying the corresponding directory structure, it enters the corresponding simulation folder, and automatically calls Python's subprocess library function to complete the execution of the compilation command, and stores the compiled file after obtaining it.

[0087] Fully automated simulation and results documentation process, including: Step 206: parse the regression simulation parameters and output the regression test file; Step 207: Distribute the regression simulation command to each server according to the regression test file and the server cluster; Here, the target file is obtained, the regression simulation parameters of the target file are parsed, and it is assembled into executable commands in a fixed format and output to the regression test file (recfg.ini) of the regression configuration. At the same time, the test cases are dynamically allocated to different servers according to the regression configuration file (recfg.ini) and the server load, and the server node information progress is recorded.

[0088] Step 208: Each server executes the command in parallel according to the regression simulation command and obtains a simulation result file; Here, each server, following the regression simulation command, enters the corresponding simulation folder, loads the generated compiled file (simv), and calls Python's subprocess library function to execute the simulation command. Simultaneously, Python is called to analyze the generated simulation log, searching for matching error messages. Once the corresponding information is identified, a threshold is used to determine whether the simulation has passed. The simulation results are then compiled and output to the simulation log, resulting in a simulation result file. Fully automatic report generation process, including: Step 209: Perform coverage analysis and regression result analysis based on the simulation result file, obtain analysis results and generate an analysis result report file; Regression result analysis involves obtaining and summarizing regression analysis results for each test case based on simulation result files. Using Python to automatically extract simulation result information, the results for each test case are continuously aggregated to generate a detailed analysis result report file containing simulation information for each test case. Simultaneously, the simulation status of the entire regression case is summarized in real time and integrated into automated task deployment on Jenkins.

[0089] Coverage analysis can include: using Python's subprocess and os libraries to parse the user-defined coverage urg option file, generate the coverage urg command and execute the merge command to merge the coverage file, and at the same time call the VCS simulation tool to read the coverage data, mainly including module name, line coverage, FSM coverage, flip coverage, condition coverage and assertion coverage and other values, and output them to the coverage data file.

[0090] Step 210: Generate a target report based on the analysis result report file and send it to the user.

[0091] Here, using email delivery as an example, we generate target reports (including regression results and coverage data) in a unified format based on the analysis result report files (including regression simulation data tables and coverage analysis data). We also create a report distribution task, setting up the email address, subject, project build time, project name, data report path, and other settings in Jenkins. After the regression simulation is completed, the task automatically sends the data report to the relevant users via the built-in server email system.

[0092] Figure 3 A schematic diagram of the structure of a chip automatic verification device provided by an embodiment of the present disclosure; Figure 3 As shown, the device is applied to a verification platform, and the device includes: An acquisition module is used to obtain the R&D code of the chip to be verified, determine the module corresponding to the R&D code, and determine the simulation path of the module; A first processing module is configured to determine a simulation folder of the module according to the simulation path, wherein the simulation folder includes at least one test case code and a verification environment file for the module; Compile in parallel according to at least one test case code and verification environment file in the simulation folder to obtain at least one compiled file for chip simulation and store them separately; A second processing module is configured to simulate the at least one compiled file in parallel on the server cluster based on the regression simulation command to obtain a simulation result file; The third processing module is used to perform coverage analysis and regression result analysis according to the simulation result file, obtain analysis results and generate an analysis result report file.

[0093] In some embodiments, there is at least one module, and each module corresponds to a different verification environment; the simulation folder also includes compilation options; The first processing module is configured to perform parallel compilation based on at least one compilation option and the at least one test case code and the verification environment file; wherein each compilation option is configured to instruct the module to perform a different operation; The compilation includes at least one of the following: compilation of design code, compilation of verification IP code, and compilation of verification environment.

[0094] In some embodiments, the second processing module is further configured to obtain a target file, the target file being configured to store at least one set of command information; each set of command information comprising at least one of the following: a test case name, number of simulation cycles, simulation options, compilation options, stop time, and coverage options; Parsing each set of command information in the target file, and generating a regression simulation command corresponding to each set of command information according to a command generation rule; Add the command for generating the regression simulation to the regression test file.

[0095] In some embodiments, the verification platform has a server cluster, and the second processing module is used to distribute the regression simulation commands contained in the regression test file to the servers in the server cluster; the servers in the server cluster execute the corresponding regression simulation commands in parallel.

[0096] In some embodiments, the second processing module is used to determine an end time corresponding to each of the regression simulation commands, and distribute each of the regression simulation commands to different servers according to the end time with the goal of load balancing and / or minimizing simulation time; Each of the servers distributes one or more regression simulation commands.

[0097] In some embodiments, the servers in the server cluster execute in parallel at least one regression simulation command corresponding to each module according to the compiled file corresponding to the server; The parallel execution of at least one regression simulation command corresponding to each module includes: Each server in the server cluster executes in parallel at least one regression simulation command corresponding to the first module; If at least one regression simulation command corresponding to the first module is executed, at least one regression simulation command corresponding to the second module is executed in parallel; wherein, the second module is any unverified module except the first module.

[0098] In some embodiments, the second processing module is used to determine the number of modules, group the server cluster according to the number of modules, determine the server sub-cluster corresponding to each module, and distribute the regression simulation command corresponding to each module to the server sub-cluster corresponding to each module.

[0099] In some embodiments, the servers in the server cluster execute one or more regression simulation commands corresponding to multiple modules in parallel according to the compiled files corresponding to the servers; The step of executing one or more regression simulation commands corresponding to multiple modules in parallel includes: Each server in the server sub-cluster executes the 1st to Nth regression simulation commands corresponding to the module in parallel, and the server sub-cluster includes N servers, where N is greater than or equal to 1.

[0100] After the Nth regression simulation command is executed, the N+1th to 2Nth regression simulation commands are executed in parallel; The process loops sequentially until all regression simulation commands are executed.

[0101] In some embodiments, the servers in the server cluster execute corresponding regression simulation commands in parallel, including: Each server determines a compilation option according to the regression simulation command, and enters a corresponding simulation path according to the compilation option to load a stored compilation file; Each server executes the corresponding regression simulation command in parallel according to the compiled file, and monitors the simulation status in real time according to the simulation output pipeline information during the simulation process and records it until the simulation ends, thereby obtaining a simulation result file.

[0102] In some embodiments, the simulation result file includes: a simulation log; The third processing module is used to determine the test case name, seed number and / or simulation time according to the simulation log; Match error information according to the simulation log and determine the error information matching result; compare the error information matching result with the error threshold and determine whether the simulation passes according to the comparison result; Generate an analysis result report file based on the test case name, seed number, simulation time, simulation result and / or the matched error information; The simulation results include simulation pass, simulation failure, simulation timeout or active exit.

[0103] In some embodiments, the simulation result file further includes: a coverage file, the coverage file including: a coverage file obtained from at least one simulation for each module; The third processing module is configured to obtain at least one coverage file of each module, and merge the at least one coverage file to obtain a merged file; Storing the merged file at a first address, reading coverage data from the merged file, and generating a coverage data file; The coverage data includes at least one of the following: module name, line coverage, state machine coverage, flip coverage, condition coverage, and assertion coverage.

[0104] In some embodiments, the third processing module is further configured to determine a report template and notification method selected by the user; Generate a target report based on the report template, the analysis result report file and / or the coverage data file; The relevant files of the target report are sent to the receiving end in the selected notification mode; the relevant files of the target report include at least one of the following: project name, target report, analysis result report file, coverage data file, and project status.

[0105] It is understood that when implementing the corresponding chip automatic verification method, the chip automatic verification device provided in the above embodiment can, as needed, assign the above-described processing to different program modules to complete all or part of the above-described processing. In addition, the device provided in the above embodiment and the corresponding method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0106] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a chip automatic verification method.

[0107] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the chip automatic verification method provided by the embodiment of the present application.

[0108] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0109] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0110] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0111] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0112] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure; Figure 4 As shown, the electronic device 40 includes: a processor 401, and a memory 402 in communication with the processor 401; the memory 402 stores instructions that can be executed by the processor 401. The instructions are executed by the processor 401, so that the processor 401 can perform: Obtaining the R&D code of the chip to be verified, determining the module corresponding to the R&D code and determining the simulation path of the module; Determining a simulation folder of the module according to the simulation path, wherein the simulation folder includes at least one test case code and a verification environment file for the module; Compile in parallel according to at least one test case code and verification environment file in the simulation folder to obtain at least one compiled file for chip simulation and store them separately; Based on the regression simulation command, simulating the at least one compiled file in parallel on the server cluster to obtain a simulation result file; Perform coverage analysis and regression result analysis based on the simulation result file to obtain analysis results and generate an analysis result report file.

[0113] The electronic device provided in the above embodiment and the corresponding chip automatic verification method embodiment are of the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0114] In actual application, the electronic device 40 may further include: at least one network interface 403. The various components in the electronic device 40 are coupled together via a bus system 404. It is understood that the bus system 404 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 4 In the figure, various buses are labeled as bus system 404. There may be at least one processor 401 and at least one memory 402. The network interface 403 is used for wired or wireless communication between the electronic device 40 and other devices.

[0115] The memory 402 in the embodiment of the present disclosure is used to store various types of data to support the operation of the electronic device 40 .

[0116] The methods disclosed in the above embodiments of the present disclosure can be applied to or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 401 or by software instructions. Processor 401 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. Processor 401 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present disclosure can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium located in memory 402. Processor 401 reads information from memory 402 and, in conjunction with its hardware, completes the steps of the aforementioned chip automatic verification method.

[0117] In some embodiments, the electronic device 40 can be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to execute the aforementioned method.

[0118] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0119] In the above description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0120] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by those skilled in the art in the art of this disclosure. The terms used in this disclosure are only for the purpose of describing the embodiments of this disclosure and are not intended to limit this disclosure.

[0121] It should be understood that in the various embodiments of the present disclosure, the size of the serial number of each implementation process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0122] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means two or more, unless otherwise specifically defined.

[0123] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A chip automatic verification method, characterized in that: Applied to a verification platform, the method includes: Obtaining the R&D code of the chip to be verified, determining the module corresponding to the R&D code and determining the simulation path of the module; Determining a simulation folder of the module according to the simulation path, wherein the simulation folder includes at least one test case code and a verification environment file for the module; Compile in parallel according to at least one test case code and verification environment file in the simulation folder to obtain at least one compiled file for chip simulation and store them separately; Based on the regression simulation command, simulating the at least one compiled file in parallel on the server cluster to obtain a simulation result file; Perform coverage analysis and regression result analysis based on the simulation result file to obtain analysis results and generate an analysis result report file.

2. The method according to claim 1, characterized in that There is at least one module, and each module corresponds to a different verification environment; the simulation folder also includes compilation options; Compiling in parallel according to the at least one test case code and the verification environment file includes: Based on at least one compilation option, parallel compilation is performed according to the at least one test case code and the verification environment file; wherein each of the compilation options is used to instruct the module to perform a different operation; The compilation includes at least one of the following: compilation of design code, compilation of verification IP code, and compilation of verification environment.

3. The method according to claim 2, characterized in that The method further comprises: Obtain a target file, wherein the target file is used to store at least one set of command information; each set of command information includes at least one of the following: a test case name, a number of simulation cycles, a simulation option, a compilation option, a stop time, and a coverage option; Parsing each set of command information in the target file, and generating a regression simulation command corresponding to each set of command information according to a command generation rule; Add the command for generating the regression simulation to the regression test file.

4. The method according to claim 3, characterized in that The verification platform has a server cluster, and the parallel simulation of the at least one compiled file on the server cluster includes: Distributing the regression simulation commands contained in the regression test file to the servers in the server cluster; The servers in the server cluster execute corresponding regression simulation commands in parallel.

5. The method according to claim 4, characterized in that Distributing the regression simulation commands contained in the regression test file to the servers in the server cluster includes: Determine an end time corresponding to each regression simulation command, and distribute each regression simulation command to different servers according to the end time with the goal of load balancing and / or minimizing simulation time; Each of the servers distributes one or more regression simulation commands.

6. The method according to claim 5, characterized in that The servers in the server cluster execute corresponding regression simulation commands in parallel, including: The servers in the server cluster execute at least one regression simulation command corresponding to each module in parallel according to the compiled file corresponding to the server; The parallel execution of at least one regression simulation command corresponding to each module includes: Each server in the server cluster executes in parallel at least one regression simulation command corresponding to the first module; If at least one regression simulation command corresponding to the first module is executed, at least one regression simulation command corresponding to the second module is executed in parallel; wherein, the second module is any unverified module except the first module.

7. The method according to claim 4, characterized in that Distributing the regression simulation commands contained in the regression test file to the servers in the server cluster includes: The number of the modules is determined, the server cluster is grouped according to the number of the modules, a server sub-cluster corresponding to each module is determined, and the regression simulation command corresponding to each module is distributed to the server sub-cluster corresponding to each module.

8. The method according to claim 7, characterized in that The servers in the server cluster execute corresponding regression simulation commands in parallel, including: The servers in the server cluster execute one or more regression simulation commands corresponding to multiple modules in parallel according to the compiled files corresponding to the servers; The step of executing one or more regression simulation commands corresponding to multiple modules in parallel includes: Each server in the server sub-cluster executes the 1st to Nth regression simulation commands corresponding to the module in parallel, and the server sub-cluster includes N servers, where N is greater than or equal to 1; After the Nth regression simulation command is executed, the N+1th to 2Nth regression simulation commands are executed in parallel; The process loops sequentially until all regression simulation commands are executed.

9. The method according to claim 4, characterized in that The servers in the server cluster execute corresponding regression simulation commands in parallel, including: Each server determines a compilation option according to the regression simulation command, and enters a corresponding simulation path according to the compilation option to load a stored compilation file; Each server executes the corresponding regression simulation command in parallel according to the compiled file, and monitors the simulation status in real time according to the simulation output pipeline information during the simulation process and records it until the simulation ends, thereby obtaining a simulation result file.

10. The method according to claim 1, characterized in that The simulation result file includes: simulation log; The coverage analysis and regression result analysis are performed according to the simulation result file to obtain the analysis results and generate an analysis result report file, including: Determine the test case name, seed number, and / or simulation time based on the simulation log; Match error information according to the simulation log and determine the error information matching result; compare the error information matching result with the error threshold and determine whether the simulation passes according to the comparison result; Generate an analysis result report file based on the test case name, seed number, simulation time, simulation result and / or the matched error information; The simulation results include simulation pass, simulation failure, simulation timeout or active exit.

11. The method according to claim 10, characterized in that The simulation result file further includes: a coverage file, wherein the coverage file includes: a coverage file obtained by at least one simulation for each module; The performing coverage analysis and regression result analysis according to the simulation result file, obtaining analysis results and generating an analysis result report file further includes: Obtain at least one coverage file of each module, and merge the at least one coverage file to obtain a merged file; Storing the merged file at a first address, reading coverage data from the merged file, and generating a coverage data file; The coverage data includes at least one of the following: module name, line coverage, state machine FSM coverage, flip coverage, condition coverage, and assertion coverage.

12. The method according to claim 11, characterized in that The method further comprises: Determine the report template and notification method selected by the user; Generate a target report based on the report template, the analysis result report file and / or the coverage data file; The relevant files of the target report are sent to the receiving end in the selected notification mode; the relevant files of the target report include at least one of the following: project name, target report, analysis result report file, coverage data file, and project status.

13. A chip automatic verification device, characterized in that: The device is applied to a verification platform, and includes: An acquisition module is used to obtain the R&D code of the chip to be verified, determine the module corresponding to the R&D code, and determine the simulation path of the module; A first processing module is configured to determine a simulation folder of the module according to the simulation path, wherein the simulation folder includes at least one test case code and a verification environment file for the module; Compile in parallel according to at least one test case code and verification environment file in the simulation folder to obtain at least one compiled file for chip simulation and store them separately; A second processing module is configured to simulate the at least one compiled file in parallel on the server cluster based on the regression simulation command to obtain a simulation result file; The third processing module is used to perform coverage analysis and regression result analysis according to the simulation result file, obtain analysis results and generate an analysis result report file.

14. An electronic device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 12.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 12.

Images