Digital IC verification environment multi-thread operation automation method

Abstract

The invention discloses a digital IC verification environment multi-thread operation automation method, which comprises the following steps: S1, building a directory structure of a verification platform, including a project directory, a design directory and a simulation directory; s2, establishing a simulation environment according to the configurable parameter description file; s3, inputting an instruction by a verifier, and performing dual preprocessing; s4, the batch processing script is run to read the configurable parameter description file, and running of the multiple test cases is completed; and S5, performing anomaly detection, and ending verification. According to the method, the verification platform is submitted to the server cluster by using the bsub command, resources of the server cluster are fully utilized, and the probability of jamming is reduced; a directory structure is also optimized, and a batch.cfg file is utilized to indicate the number of processes started by the test cases at the same time, so that the test process and data are clearer, and analysis and statistics are facilitated; codes are updated regularly through dual preprocessing, it is guaranteed that code versions are consistent during design and verification, and verification can be conducted smoothly.

Description

Technical Field

[0001] The present invention relates to the technical field of digital IC verification, and in particular to a method for automating multi-threaded operation in a digital IC verification environment. Background Art

[0002] With the development of integrated circuits, the requirements for digital IC verification are becoming increasingly higher, and verification accounts for an increasing proportion of the chip development cycle. Therefore, how to improve the efficiency of verification work is particularly important.

[0003] In the existing multi-threaded digital IC verification process, verification personnel usually build the verification platform based on UVM verification methodology first, and then use VCS software for simulation. VCS is the supporting simulation software of Synopsys. To run VCS, you need to use the make command and Makefile file in the Linux system. The make command is the compilation command. It will manage the project according to the content of the Makefile, and the Makefile file specifies the dependencies between compile and simulation, target files, and execution commands.

[0004] However, as the chip scale becomes larger and larger, the amount of design code increases, and the amount of design code that needs to be loaded increases, which leads to an increase in the number of functional modules that need to be verified. In addition, due to the increase in the number of functional modules, the circuit scale of the functional modules themselves is also increasing, and multi-threaded testing often involves a large number of test resources. In actual testing, there are often problems such as chaotic test processes, chaotic test data, and various interruptions. In particular, when multiple verification platforms are performing regression tests on a server at the same time, a large number of verification platform operations simultaneously occupy the server, which can easily cause problems such as server freezing.

[0005] In addition, when conducting tests, if the designer changes the design code and fails to inform the verifier in a timely manner, the verifier will use the old version of the design code for the test. At this time, the result of the test cannot be used as the verification result of the latest design code. In more serious cases, the test may not be able to be carried out at all.

[0006] That is to say, the current multi-threaded digital IC verification includes at least the following three problems: 1) During the verification process, the server is under great pressure and is prone to freezing and other problems, which prevents the verification process from proceeding smoothly; 2) During the multi-threaded digital IC verification process, the design code and the code used during verification may have different versions. Lack of code updates may cause verification errors or even failure to proceed. 3) High testing pressure leads to chaotic testing process and data, making statistical analysis inconvenient. Summary of the invention

[0007] In view of the above three problems, the purpose of the present invention is to propose a method for automating multi-threaded operation in a digital IC verification environment, which uses the bsub command to submit the verification platform to the server cluster, fully utilizes the resources of the server cluster, and reduces the probability of getting stuck; at the same time, optimizes the directory structure and sets various parameters, and uses the batch.cfg file to indicate the number of processes that are simultaneously started for multiple test cases, making the multi-threaded test process and data clearer and convenient for analysis and statistics; in addition, double preprocessing is used to update the code to ensure that the design code and the verification code are consistent, so that the verification can proceed smoothly.

[0008] This is achieved through the following technical solutions: A method for automating multi-threaded operation of a digital IC verification environment comprises the following steps: S1. Build the directory structure of the verification platform: Build the directory structure of the verification platform according to the verification requirements, including the project directory, the design directory and the simulation directory; the project directory is used as the root directory of the directory structure, the design directory includes each design code, the simulation directory includes a multi-level verification platform, and each level of the verification platform includes the corresponding bench directory, uvm_tb directory, uvm_tc directory and rsim directory; wherein, the bench directory includes the top-level code for establishing the verification platform and the top-level code corresponding to the instantiation verification requirements, the uvm_tb directory includes the components corresponding to the verification requirements created according to the UVM verification methodology, the uvm_tc directory includes multiple test cases for the verification platform, and the rsim directory includes the Makefile file, the batch script batch.pl and the env_cfg directory; S2. Establish a simulation environment based on a configurable parameter description file: Establish a configurable parameter description file in the env_cfg directory of step S1, including a batch.cfg file, a dual-specification file, an flist_rtl.cfg file, and a case.lst file, and establish a simulation environment based on the configurable parameter description file; wherein the batch.cfg file is used to support the automatic operation of the batch script batch.pl; the dual-specification file is used to specify the path of each design code and the time to start regression verification; flist_rtl.cfg is used to specify the relative path of each design code corresponding file and the relative path of the verification platform environment code corresponding file; case.lst is used to specify multiple test parameters for each test case; S3, the verification personnel inputs instructions and runs the batch script for double preprocessing: the verification personnel inputs instructions to the batch script batch.pl, and performs double preprocessing of timing startup and updating code according to the instructions and the double designated files in step S2; S4, run the batch script to read the configurable parameter description file, and start verification and compilation: after the double preprocessing in step S3 is completed, run the batch script batch.pl to read the configurable parameter description file to generate multiple specific parameters, and configure the Makefile file in step S1 based on the multiple specific parameters; the batch script batch.pl uses the bsub command to execute the compilation command, and the compilation command manages the simulation environment according to the configured Makefile file, completes the operation of multiple test cases, and obtains the simulation folder and simulation file of each test case; S5. Perform anomaly detection and end verification: Determine whether there is an anomaly in the simulation based on the simulation file in step S4. If there is an anomaly, keep the simulation folder and end verification; if there is no anomaly, keep the simulation file and end verification.

[0009] The batch script batch.pl and bsub command can be used to effectively submit the verification platform to the server cluster and perform automatic testing, making full use of the resources of the server cluster and reducing the probability of freezing. At the same time, the directory structure is optimized and various parameters are set to make the test process clearer and the data more targeted, which is convenient for analysis and statistics.

[0010] Preferably, the multiple test parameters in step S1 include a name, a seed number, and a run number. The name, seed number, and run number are important for the execution and analysis of the test case, and facilitate effective control and management of the test case.

[0011] Preferably, in step S1, the multi-level verification platform includes multiple module-level verification platforms, multiple subsystem-level verification platforms and system-level verification platforms; wherein, multiple module-level verification platforms are subordinate platforms of multiple subsystem-level verification platforms, and multiple subsystem-level verification platforms are subordinate platforms of system-level verification platforms. Setting up a multi-level verification platform can improve the correctness and integrity of digital ICs at different levels, is more in line with actual scenarios, can ensure the correctness of functions at each level, and ultimately guarantee the quality and reliability of the entire system.

[0012] Preferably, in step S1, when creating each component corresponding to the verification requirement according to the UVM verification methodology, each component includes the env corresponding to the env_cfg directory and the agent, monitor and driver corresponding to the env, and also includes the reference model corresponding to each test case and the agent, monitor and driver corresponding to each reference model. Multiple components facilitate the construction of a complete verification environment, thereby ensuring the comprehensiveness and effectiveness of the verification; at the same time, the large number of components and the subdivision are also convenient for expansion and maintenance.

[0013] Preferably, in step S2, the batch.cfg file includes each file name, the number of processes opened simultaneously for multiple test cases, a coverage collection switch, and multiple UVM printing priorities, and each test case has a corresponding coverage file and a corresponding UVM printing priority.

[0014] Preferably, the level of UVM printing priority includes: UVM_NONE that does not output any information, UVM_LOW that outputs less information, UVM_MEDIUM that outputs moderate information, UVM_HIGH that outputs detailed information, and UVM_FULL that outputs complete information. Set multiple levels to be convenient to meet actual verification needs, particularly to meet the needs during multi-threaded testing.

[0015] Preferably, in step S3, when performing double preprocessing, first determine whether the timing is started, if so, wait until the timing ends and then determine whether to update the code; if not, directly determine whether to update the code; when determining whether to update the code, if so, update the part that needs to be updated in each design code, if not, prepare to proceed to step S4. Performing double processing of timing and updating code can meet the test requirements of a specific time and place the test when the server resources are idle.

[0016] Preferably, step S4 further comprises: generating a corresponding random seed number for the file corresponding to each test case, configuring each random seed number into the Makefile file, and each random seed number is different. Using different random seed numbers, the test of the corresponding test case can be reproduced, thereby ensuring the reliability and reproducibility of the test.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The technical solution of the present invention utilizes the bsub command to submit the verification platform to the server cluster, fully utilizes the resources of the server cluster, and reduces the probability of freezing; at the same time, optimizes the directory structure and sets various parameters, and utilizes the batch.cfg file to indicate the number of processes that are simultaneously started for multiple test cases, so that the multi-threaded test process and data are clearer and convenient for analysis and statistics; in addition, double preprocessing is also used to update the code to ensure that the design code and the code during verification are consistent, so that the verification can proceed smoothly. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a method for automating multi-threaded operation of a digital IC verification environment; Figure 2 A flowchart of a method for verifying a multi-threaded automated operation using a digital IC verification environment. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0020] like Figure 1 As shown, it is a flow chart of a method for automating multi-threaded operation in a digital IC verification environment; Figure 2 As shown, it is a flow chart of using the multi-threaded automated method of digital IC verification environment to perform verification; combined with Figure 1 and Figure 2 As shown, a method for automating multi-threaded operation of a digital IC verification environment specifically includes the following steps: S1. Build the directory structure of the verification platform: Build the directory structure of the verification platform according to the verification requirements, including the project directory, design directory and simulation directory; the project directory is used as the root directory of the directory structure, the design directory includes each design code, the simulation directory includes multi-level verification platforms, and each level of verification platform includes the corresponding bench directory, uvm_tb directory, uvm_tc directory and rsim directory; among them, the bench directory includes the top-level code for establishing the verification platform and the top-level code corresponding to the instantiation verification requirements, the uvm_tb directory includes various components corresponding to the verification requirements created according to the UVM verification methodology, the uvm_tc directory includes multiple test cases of the verification platform, and the rsim directory includes the Makefile file, the batch script batch.pl and the env_cfg directory.

[0021] In this embodiment, the multiple test parameters in step S1 include name, seed number and number of runs. Name, seed number and number of runs are all important for the execution and analysis of test cases, which is convenient for effective control and management of test cases. In the verification environment under the UVM framework, the parameter configuration of each test case is very important, especially the name, seed number and number of runs. These parameters are all important for the execution, debugging and analysis of test cases. The name corresponds to the three aspects of identification, recording and logging, that is, it is used to uniquely identify each test case, facilitate recording and tracking the execution of the test case, and can quickly locate a specific test case by name in the log file. The seed number corresponds to random number generation and reproduction problems. Random number generation is used to initialize the seed value of the random number generator to ensure that the random data generated by each test case is repeatable; and the reproduction problem is that when the test fails, the same problem can be reproduced by the same seed number, which is convenient for debugging. The number of runs is the number of times the specified test case needs to be executed. Through multiple executions, statistical analysis can be performed to evaluate the effect of the test case, which can also affect the coverage situation.

[0022] In this embodiment, in step S1, the multi-level verification platform includes multiple module-level verification platforms, multiple subsystem-level verification platforms and system-level verification platforms; wherein, multiple module-level verification platforms are subordinate platforms of multiple subsystem-level verification platforms, and multiple subsystem-level verification platforms are subordinate platforms of the system-level verification platform. Setting up a multi-level verification platform can improve the correctness and integrity of digital ICs at different levels, which is more in line with actual scenarios, can ensure the correctness of functions at each level, and ultimately ensure the quality and reliability of the entire system. The main purpose of setting up a multi-level verification platform is to ensure the correctness and integrity of digital ICs at different levels. This multi-level verification strategy helps to discover and correct errors in the design as early as possible, ensure the correctness of functions at each level, and ultimately ensure the quality and reliability of the entire system. Taking the verification of CPU as an example, the decoding module, cache, biu module, execution unit and MMU and other units can be verified at the module level, and multiple module-level verification platforms can be built; then, each of the module-level verification platforms with many interactions can be combined into a subsystem level for verification. For example, the cache can be combined with the biu module responsible for the connection to form a subsystem for verification; the entire CPU is verified as a system, that is, as a system-level verification platform.

[0023] In the present embodiment, in step S1, when each component corresponding to the verification requirement is created according to the UVM verification methodology, each component includes the env corresponding to the env_cfg directory and the agent, monitor and driver corresponding to env, and also includes the reference model corresponding to each test case and the agent, monitor and driver corresponding to each reference model.Env represents the environment component, which is the top-level part in the UVM verification framework, used to combine and coordinate all verification components, such as the agent component agent responsible for modularizing other components, the component monitor responsible for monitoring, and the component driver responsible for driving the test, etc., env is responsible for instantiating and configuring these verification components, and provides an interface for external test programs to access these components.Multiple components are convenient to build a complete verification environment, thereby ensuring the comprehensiveness and effectiveness of verification; At the same time, the number of components is large and subdivided, which is also convenient for expansion and maintenance.

[0024] S2. Establish a simulation environment based on the configurable parameter description file: In the env_cfg directory of step S1, a configurable parameter description file is established, including a batch.cfg file, a dual-specification file, an flist_rtl.cfg file, and a case.lst file. A simulation environment is established according to the configurable parameter description file. The description file is expressed in the form of regression_info.txt, which can be used to specify the path of each design code and the time to start regression verification. Among them, the batch.cfg file is used to support the automatic operation of the batch script batch.pl; the dual-specification file is used to specify the path of each design code and the time to start regression verification; flist_rtl.cfg is used to specify the relative path of the file corresponding to each design code and the relative path of the file corresponding to the environment code of the verification platform; case.lst is used to specify multiple test parameters for each test case.

[0025] In this embodiment, in step S2, the batch.cfg file includes each file name, the number of processes opened simultaneously for multiple test cases, a coverage collection switch, and multiple UVM printing priorities, and each test case has a corresponding coverage file and a corresponding UVM printing priority.

[0026] The coverage collection switch is used to control whether to collect the coverage of the corresponding design code during the verification of each test case, so as to form each corresponding coverage file. For each test case, after the verification is completed, each test case has a simulation folder named after the test case plus a random seed number. All simulation folders here contain corresponding coverage files. By merging these coverage files, the final coverage file can be generated, which can be put into the simulation file and wait for the verifier to view it, so as to know the integrity of the verification.

[0027] When determining the number of processes to be started simultaneously and starting regression verification, if the user configures the number of processes to be started to be 10, 10 test cases will be simulated and verified at the same time. The corresponding case.lst file also configures the number of runs for each test case to indicate the corresponding actual number of verifications.

[0028] In the present embodiment, the level of UVM printing priority includes: UVM_NONE that does not output any information, UVM_LOW that outputs less information, UVM_MEDIUM that outputs moderate information, UVM_HIGH that outputs detailed information, and UVM_FULL that outputs complete information. Various levels are set to be convenient to meet the actual verification requirements, particularly to meet the requirements during multi-threaded testing.

[0029] S3. Verifiers enter instructions and run batch scripts for double preprocessing: The verification personnel input instructions to the batch script batch.pl, and perform dual preprocessing of scheduled startup and code update according to the instructions and the dual designated files in step S2.

[0030] In this embodiment, in step S3, when double preprocessing is performed, it is first determined whether the timing is started. If so, wait until the timing ends before determining whether to update the code; if not, directly determine whether to update the code. The timing can be operated using a timer or a timing module, which can meet the test requirements at a specific time and place the test when the server resources are idle. For example, the server resources are heavily used during the day and are basically idle at night. In this way, the test can be scheduled to the evening to reduce the excessive server resources occupied during the day, and also reduce the problem of the inability to complete all tests due to the lack of idle server resources. When determining whether to update the code, if so, update the part that needs to be updated in each design code. If not, prepare to proceed to step S4. By combining timing and updating the code, three update situations can be designed, namely, immediate update, update after a period of time, and no update. Combined with batch scripts, when inputting commands, the verification personnel can also input three types of commands: input batch.pl to start verification immediately without updating the design code; input batch.pl svn_now to perform verification immediately and update the design code; input batch.pl svn to perform verification and update the design code at a fixed time. In this way, the verification personnel can select the corresponding solution according to the actual update requirements, which not only ensures the smooth test after the update, but also effectively improves the flexibility and convenience of the actual test.

[0031] When updating the code, SVN is used for the update. SVN stands for Subversion. SVN is an open source version control system that can be used to manage design code and verification code, especially to manage data that changes over time, and store this data in a central data archive repository. The central data archive is similar to a file server, which will retain every file change, so you can restore the file to an old version or browse the file change history. Based on SVN, you can update the code so that the code version during verification is consistent with the code version during design, thereby ensuring the smooth progress of the test.

[0032] S4. Run the batch script to read the configurable parameter description file and start verification and compilation: After the double preprocessing in step S3 is completed, the batch script batch.pl is run to read the configurable parameter description file to generate multiple specific parameters, and the Makefile file of step S1 is configured based on the multiple specific parameters; the batch script batch.pl uses the bsub command to execute the compilation command, and the compilation command manages the simulation environment according to the configured Makefile file, completes the operation of multiple test cases, and creates a simulation folder and simulation file for each test case.

[0033] In this embodiment, step S4 further includes: generating a corresponding random seed number for the file corresponding to each test case, configuring each random seed number into the Makefile file, and each random seed number is different. The random seed number is a seed generated by a random number, and each test of each test case will generate a corresponding random seed number and save it in the Makefile file, thereby ensuring that each test of each test case can be called and reproduced, ensuring the reproducibility and reliability of the regression test of all test cases.

[0034] S5. Perform anomaly detection and end verification: Determine whether there is any abnormality in the simulation based on the simulation file in step S4. If there is an abnormality, keep the simulation folder to let the verification personnel know the problem in a timely and clear manner. The simulation folder will be named in the form of test case name_random seed number, and then the verification will be ended; if there is no abnormality, keep the simulation file and store it in the location where the verification platform stores files, and then the verification will be ended.

[0035] In summary, the bsub command is used to submit the verification platform to the server cluster, making full use of the resources of the server cluster and reducing the probability of jamming. At the same time, the directory structure is optimized and various parameters are set, and the batch.cfg file is used to indicate the number of processes that are started simultaneously for multiple test cases, making the multi-threaded test process and data clearer and easier to analyze and count. In addition, double preprocessing is used to update the code to ensure that the design code and the verification code are consistent, so that the verification can proceed smoothly, which is a significant improvement.

[0036] The above embodiments are only for illustrating the technical idea of ​​the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for automating multi-threaded operation of a digital IC verification environment, characterized in that: The steps include: S1. Build the directory structure of the verification platform: Build the directory structure of the verification platform according to the verification requirements, including the project directory, design directory and simulation directory; the project directory is used as the root directory of the directory structure, the design directory includes each design code, and the simulation directory includes multi-level verification platforms. Each level of verification platform includes the corresponding bench directory, uvm_tb directory, uvm_tc directory and rsim directory; Among them, the bench directory includes the top-level code for establishing the verification platform and the top-level code corresponding to the instantiation verification requirements, the uvm_tb directory includes the components corresponding to the verification requirements created according to the UVM verification methodology, the uvm_tc directory includes multiple test cases of the verification platform, and the rsim directory includes the Makefile file, the batch script batch.pl and the env_cfg directory; S2. Establish a simulation environment based on the configurable parameter description file: In the env_cfg directory of step S1, a configurable parameter description file is created, including a batch.cfg file, a dual-specification file, an flist_rtl.cfg file, and a case.lst file, and a simulation environment is established according to the configurable parameter description file; Among them, the batch.cfg file is used to support the automatic operation of the batch script batch.pl; the double-specified file is used to specify the path of each design code and the time to start regression verification; flist_rtl.cfg is used to specify the relative path of each design code corresponding file and the relative path of the verification platform environment code corresponding file; case.lst is used to specify multiple test parameters for each test case; S3. Verifiers enter instructions and run batch scripts for double preprocessing: The verification personnel inputs instructions to the batch script batch.pl, and performs dual preprocessing of timing startup and updating code according to the instructions and the dual designated files in step S2; S4. Run the batch script to read the configurable parameter description file and start verification and compilation: After the double preprocessing in step S3 is completed, the batch processing script batch.pl is run to read the configurable parameter description file to generate multiple specific parameters, and the Makefile file in step S1 is configured based on the multiple specific parameters; The batch script batch.pl uses the bsub command to execute the compile command. The compile command manages the simulation environment according to the configured Makefile file, completes the operation of multiple test cases, and obtains the simulation folder and simulation file of each test case. S5. Perform anomaly detection and end verification: Determine whether there is an abnormality in the simulation based on the simulation file in step S4. If there is an abnormality, keep the simulation folder and end the verification; if there is no abnormality, keep the simulation file and end the verification.

2. The method for automating multi-threaded operation of a digital IC verification environment according to claim 1, characterized in that: The multiple test parameters in step S1 include a name, a seed number, and a running number.

3. The method for automating multi-threaded operation of a digital IC verification environment according to claim 1, characterized in that: In step S1, the multi-level verification platform includes multiple module-level verification platforms, multiple subsystem-level verification platforms and a system-level verification platform; wherein the multiple module-level verification platforms are subordinate platforms of the multiple subsystem-level verification platforms, and the multiple subsystem-level verification platforms are subordinate platforms of the system-level verification platform.

4. The method for automating multi-threaded operation of a digital IC verification environment according to claim 1, characterized in that: In step S1, when creating various components corresponding to the verification requirements according to the UVM verification methodology, each component includes the env corresponding to the env_cfg directory and the agent, monitor and driver corresponding to the env, and also includes the reference model corresponding to each test case and the agent, monitor and driver corresponding to each reference model.

5. The method for automating multi-threaded operation of a digital IC verification environment according to claim 1, characterized in that: In step S2, the batch.cfg file includes each file name, the number of processes opened simultaneously by multiple test cases, a collection coverage switch, and multiple UVM printing priorities. Each test case has a corresponding coverage file and a corresponding UVM printing priority.

6. The method for automating multi-threaded operation of a digital IC verification environment according to claim 5, characterized in that: The levels of UVM printing priority include: UVM_NONE which does not output any information, UVM_LOW which outputs less information, UVM_MEDIUM which outputs moderate information, UVM_HIGH which outputs detailed information, and UVM_FULL which outputs complete information.

7. The method for automating multi-threaded operation of a digital IC verification environment according to claim 1, characterized in that: In step S3, when performing double preprocessing, first determine whether the timing is started, if so, wait until the timing ends and then determine whether to update the code; if not, directly determine whether to update the code; When determining whether to update the code, if yes, then update the part of each design code that needs to be updated, if no, then prepare to proceed to step S4.

8. The method for automating multi-threaded operation of a digital IC verification environment according to claim 1, characterized in that: Step S4 also includes: generating a corresponding random seed number for the file corresponding to each test case, configuring each random seed number into the Makefile file, and each random seed number is different.

Images