A defect detection method for advanced synthesis tools based on equivalent modulus testing

By adding code variation operations and emulator simulation in equivalent mold extraction tests, the problem of advanced comprehensive tool detection is solved, the efficiency of test case generation and defect detection accuracy is improved, automated bug report generation is realized, and existing technology is solved The problem of inefficient detection of defects in medium and advanced comprehensive tools.

CN113010427BActive Publication Date: 2024-12-06DALIAN UNIV OF TECH
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202110301741.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-22
Publication Date
2024-12-06
Estimated Expiration
2041-03-22

AI Technical Summary

Technical Problem

The prior art is difficult to effectively detect and fix defects caused by advanced synthesis tools in FPGA design, and equivalent molding testing is inefficient in generating test cases and verifying output results.

Method used

Using the method based on equivalent modulo test, by deleting or inserting code mutation operations, and using the emulator to simulate the simulation output of seed code and post-mutated code, compare whether the two are consistent to judge the defects of advanced comprehensive tools, and automatically generate bug reports. .

Benefits of technology

Improves the efficiency of test case generation of advanced comprehensive tools, can effectively detect defects in the tool and automatically generate repair reports, improving the speed of developers fixing defects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113010427B_ABST
    Figure CN113010427B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of detection technology and software testing technology, and in particular, relates to a high-level synthesis tool defect detection method based on equivalent modulo testing. The present invention adopts the idea of ​​equivalent modulo testing, but in view of the characteristics of high-level synthesis tools, based on the mutation operation of deleting unexecuted code or inserting code in the seed code of the equivalent modulo test, a mutation operation of converting the seed code is added, and a simulator is used to simulate the two HDL codes obtained by synthesizing the seed code and the mutated code by the high-level synthesis tool to be tested, and compare whether the two HDL codes have the same simulation output results to determine whether the high-level synthesis tool to be tested has defects. The method of the present invention can generate effective test cases for high-level synthesis tool testing, can effectively detect defects in high-level synthesis tools, and can automatically generate bug reports for developers to quickly fix defects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of detection technology and software testing technology, and in particular relates to an advanced synthesis tool defect detection method based on equivalent modulus testing. Background Art

[0002] In recent years, with the development of digitalization, networking and intelligence, the application field of FPGA (Field Programmable Gate Array) has expanded rapidly. As a key core device in the fields of communications, aerospace, military industry, etc., FPGA is an important support foundation for ensuring national strategic security. Therefore, it is crucial to complete FPGA design and development work efficiently.

[0003] In the FPGA design and development process, high-level synthesis (HLS) technology is crucial. High-level synthesis technology is a method and process for converting the algorithm-level or behavioral-level description of circuit design specifications into circuit structure description under certain constraints. Compared with the traditional RTL (Register Transfer Level) design process, the high-level synthesis design process reduces the development cost of FPGA developers, allowing software or algorithm engineers who are not proficient in micro-architecture and digital circuits to efficiently complete FPGA design work, thereby improving the efficiency of FPGA design and development work. High-level synthesis tools play an important supporting role in the high-level synthesis design process. High-level synthesis tools can convert algorithm codes written in high-level languages ​​(such as C / C++ codes) into hardware description language (HDL) codes, such as Verilog codes, VHDL codes, etc. Whether high-level synthesis tools can run correctly affects the accuracy of FPGA design. Some existing invention patents, such as the method for implementing SystemC verification and the verification platform component architecture (patent application number: 202011129206.6), use the TLI interface to connect the SystemC design to be verified with the existing VerilogRTL verification platform to complete the SystemC verification work, thereby ensuring the correctness of the final circuit. However, this method does not test the circuit design defects caused by the defects of the high-level synthesis tool. Another example is a method and system for supporting automated testing and recording of EDA software (patent number: CN202010130642.9), which mainly tests FPGA logic synthesis software. In the FPGA design process, FPGA logic synthesis is the design process after the high-level synthesis is completed, and the output of the high-level synthesis can be used as the input of FPGA logic synthesis. Testing of FPGA logic synthesis software still cannot guarantee the correctness of the FPGA design after being processed by the high-level synthesis tool. Based on the above existing invention patents, it can be found that there are few invention patents for defect detection of high-level synthesis tools. At present, the mainstream commercial high-level synthesis tools include VivadoHLS, IntelHLSCompiler, etc., and the open source high-level synthesis tools include LegUp, Bambu, etc. These tools are still being iterated to fix the missed defects. Therefore, it is urgent and necessary to conduct rigorous testing on high-level synthesis tools.

[0004] The equivalence modulo inputs method plays a pivotal role in the field of compiler testing. The equivalence modulo test discovers defects in the compiler by constructing a mutant program and comparing whether the seed program and the mutant program have the same output under the same test input. For high-level synthesis tools, their input is high-level language code, such as C / C++ code, and their output is HDL code, such as Verilog, VHDL and other codes. Randomly generated high-level language code as a test case generates a small number of valid test cases, while the equivalence modulo test generates a small number of test cases by deleting unexecuted code or inserting code in the seed program. It depends on the number of seed programs and it is difficult to construct test cases that can be optimized by high-level synthesis tools, resulting in low efficiency in testing high-level synthesis tools. So how to efficiently generate test cases for high-level synthesis tools is an urgent problem to be solved. At the same time, since the output of high-level synthesis tools is HDL code, how to judge the correctness of the test output results and then judge whether the high-level synthesis tools have defects is also an urgent problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide a high-level synthesis tool defect detection method based on equivalent modulo test, so as to solve the problems of effective test case generation and test output result verification in the process of testing the high-level synthesis tool. The present invention adopts the idea of ​​equivalent modulo test, but in view of the characteristics of the high-level synthesis tool, based on the mutation operation of deleting unexecuted code or inserting code in the seed code in the equivalent modulo test, a mutation operation of converting the seed code is added, and a simulator is used to simulate the two HDL codes obtained after the seed code and the mutated code are synthesized by the high-level synthesis tool to be tested, and the two HDL codes are compared to see whether they have the same simulation output results to determine whether the high-level synthesis tool to be tested has defects.

[0006] The technical solution of the present invention:

[0007] A defect detection method for advanced synthesis tools based on equivalent modulus testing, the specific steps are as follows:

[0008] Step 1: Collect existing C language test case programs and build a test case seed pool;

[0009] Use crawler technology to obtain test case programs from the AnghaBench library and CHStoneHLS use case library, filter out invalid programs, and build a test case seed pool.

[0010] Step 2: Check whether the test times have been reached; if not, randomly select a program file from the seed pool constructed in step 1 as the test case seed P, and generate the corresponding TestBench file based on the test case seed; if it has been reached, stop the test.

[0011] Among them, the TestBench file contains test input, design to be tested and test output; the test input provides input data for the module to be tested, the design to be tested is the module to be tested, and the test output is the output result of the module to be tested under the current test input.

[0012] Step 3: The script generation program generates and stores a TCL (Tool Command Language) script according to the seed code P and the location where the TestBench file is stored. The script execution program calls the high-level synthesis tool to be tested through the terminal command line to perform high-level synthesis on the test case seed P according to the TCL script to obtain the HDL code HP;

[0013] Specifically, first, the script generation program generates and stores a TCL script according to the location where the seed code P and the TestBench file are stored, so as to specify a new project name, a test case seed P and a TestBench file for the high-level synthesis tool to be tested, set the top-level function, set the solution name, define the technology and clock rate, specify the synthesis process, set the optimization command, and run the high-level synthesis tool to be tested for synthesis through the TCL script; then, the script execution program calls the high-level synthesis tool to be tested through the Linux terminal to execute the TCL script for high-level synthesis to obtain the HDL code HP.

[0014] Step 4: Use a high-level language compiler and a digital circuit simulator to perform co-simulation, and obtain and save the coverage C of P, the output result HLO of P, and the output result O of HP under the same TestBench;

[0015] Specifically: use a high-level language compiler to compile and run program P, use a digital circuit simulator to simulate program HP, use TestBench to test program P, use the TestBench file synthesized by a high-level synthesis tool to test program HP, obtain and save P's coverage information C, P's output result HLO, and HP's output result O.

[0016] Step 5: Check whether the test case seed P and the code HP obtained after high-level synthesis pass the simulation;

[0017] Specifically, compare the output result HLO of P and the output result O of HP under the same TestBench in step 4; if the two are consistent, it means that the simulation has passed, and continue to the subsequent test steps; if the two are inconsistent, it means that the simulation has not passed, then save P, TestBench, HP, HLO, O and the corresponding TCL file, generate and store the Bug report, increase the number of tests by 1, and jump to step 2 to continue execution; if the simulation has passed, continue to the next test step;

[0018] Step 6: Check whether the test case seed P is mutable;

[0019] Specifically: Check the number of times the seed program has mutated. If the number of mutations has exceeded the set value, it cannot mutate again, the test number increases by 1, and jumps to step 2 to continue execution; if it has not exceeded, continue with subsequent tests.

[0020] Step 7: According to the coverage information C of the test case seed program P obtained in step 4, the code mutation program performs mutation operations such as code deletion, insertion, and conversion on the test case seed P to obtain a new test case P′ after mutation;

[0021] Among them, code deletion is to delete the code in the test case seed code that is not covered by the TestBench test, code insertion is to insert the code that does not affect the program output and conforms to the circuit design specification into the local test case seed code, and code conversion is to merge, stream, parallelize and perform other operations on the for loops in the test case seed code, split, map and reorganize the array, and inline and reorganize the function according to the optional functions in the synthesis process of the tool to be tested.

[0022] Step 8: The script modification program modifies the original TCL script and saves it as a new TCL script. The script execution program calls the high-level synthesis tool to be tested through the terminal command line to perform high-level synthesis on the mutated test case P′ obtained in step 7 according to the new TCL script to obtain the HDL code HP′.

[0023] Specifically, first, the script modification program obtains the original TCL file corresponding to the seed code, modifies the project name, test case, and solution name therein, so that the new TCL file corresponds to the mutated test case P′, and saves the modified TCL file as a new TCL file. Then, the script execution program calls the high-level synthesis tool to be tested through the Linux terminal command line to perform high-level synthesis on the mutated test case P′ obtained in step 7 according to the new TCL script to obtain the HDL code HP′;

[0024] Step 9: Use a high-level language compiler and a digital circuit simulator to perform co-simulation on P′ and HP′, and use the TestBench in step 4 to obtain the output HLO′ of P′ and the output O′ of HP′;

[0025] Specifically, use a high-level language compiler to compile and run program P′, use a digital circuit simulator to simulate program HP′, use the TestBench in step 4 to test P′, use the TestBench file synthesized by a high-level synthesis tool to test HP′, and obtain the output HLO′ of program P′ and the output O′ of program HP′.

[0026] Step 10: Check whether the result O in step 4 is the same as the result O′ in step 9; if the results are different, save P′, HLO′, TestBench, HP′, O′ and the corresponding TCL file, generate and store the Bug report, increase the number of mutations by 1, and jump to step 6 to continue execution; if the results are the same, directly increase the number of mutations by 1, and jump to step 6 to continue execution.

[0027] Beneficial effects of the present invention: The method of the present invention can generate effective test cases for high-level synthesis tool testing, can effectively detect defects existing in high-level synthesis tools, and can automatically generate bug reports for developers to quickly repair defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 The present invention is a schematic diagram of the workflow of an advanced integrated tool defect detection method based on equivalent module testing. DETAILED DESCRIPTION

[0029] The method of the present invention is described in detail below in conjunction with the accompanying drawings, technical solutions and embodiments.

[0030] In this embodiment, the method of the present invention is deployed on a Linux server, and the specific configuration of the server is shown in Table 1. According to the experimental requirements, the corresponding GCC compiler, Iverilog circuit simulation simulator and other tool software and the tested software Xilinx Vivado HLSv2019.2 are installed, and Xilinx Vivado HLSv2019.2 is tested. The method of the present invention consists of programs such as building a test case seed pool, code mutation, script processing (generation and modification) and execution, and collaborative simulation.

[0031] Table 1: Linux server configuration information table

[0032] Processor model Memory operating system Intel Core i9-10900 32GB Ubuntu 18.04.01

[0033] like Figure 1As shown, a high-level synthesis tool defect detection method based on equivalent modular testing of the present invention is carried out according to the following process. Construct a test case seed pool, select test case seeds, perform high-level synthesis and collaborative simulation, and record the corresponding results. Mutate the test case seeds, obtain the mutated test cases, and perform high-level synthesis and collaborative simulation on the mutated test cases, record the corresponding results, and compare the differences between the simulation results before and after to find the defects of the high-level synthesis tool. The specific steps are as follows:

[0034] Step 1: Use crawler technology to collect existing C language test case programs and build a test case seed pool;

[0035] Specifically, the test case programs are obtained from the AnghaBench library and the CHStoneHLS library, invalid programs are filtered out, and a test case seed pool is constructed. AnghaBench is a publicly available C language test case collection that mines more than 1 million C programs from open source repositories. The official website is: http: / / cuda.dcc.ufmg.br / angha / home. The CHStoneHLS test case library is a high-level synthetic benchmark program collection based on the C language, which can be obtained from GitHub. The specific website is: https: / / github.com / lesterhuang / CHStone. This method uses crawler technology to select the CSmith collection and the Liveness-Driven Random Generated Programs (ldrgen) collection from the AnghaBench library and all the test cases in the CHStoneHLS library for download, all of which are stored in the server / home / hls / seed_programs path. After filtering and processing by this method, a test case seed pool with 20,000 programs is constructed.

[0036] Step 2: Check whether the number of tests has been reached. If not, randomly select a program file from the seed pool constructed in step 1 as the test case seed P, and generate the corresponding TestBench file based on the test case seed. If it has been reached, stop the test;

[0037] Specifically: This method automatically sets the number of tests to 20,000 times based on the number of seed programs in the seed pool. If the test number limit is not reached, a program file is randomly selected from the seed pool as the test case seed P, renamed to Test_xxx_Seed.c, and saved in the / home / hls / TestCases / Test_xxx / Test_Seed path. Extract the main() function from Test_xxx_Seed.c to generate a TestBench file, named Test_xxx_TB.c, and also save it in the / home / hls / TestCases / Test_xxx / Test_Seed path, and delete the main() function in Test_xxx_Seed.c. If the test number limit has been reached, stop the test. The "xxx" in the above file name and path is filled with the number of tests.

[0038] Step 3: The script generator generates and stores a TCL script, and the script executor calls Vivado HLS through the terminal command line to perform high-level synthesis on the test case seed P according to the TCL script to obtain the HDL code HP;

[0039] Specifically: The script generator in this step generates a TCL script based on the seed case P and the TestBench file location. Specify a new project name for Vivado HLS through the TCL file. The specific generated code is: open_project-reset proj_test_xxx_seed, specify the test case seed P and TestBench file for Vivado HLS, and the specific generated code is: add_filesTest_xxx_Seed.c and add_files-tbTest_xxx_TB.c, set the top-level function, and the specific generated code is: set_topmaintest(), set the solution name, and the specific generated code is: open_solution-reset solution_test_xxx_seed, define the technology and clock rate, and the specific generated code is: set_partzynq and create_clock–period "75MHz", specify the synthesis process, and the specific generated code is: sethls_exec 1, set the optimization command, and the specific generated code is: setdirective_pipeline "VectoerAdd / myloop", run the high-level synthesis tool to be tested for synthesis, and the specific generated code is: csynth_design and exit. The above code is stored as a TCL file named "test_xxx_run_hls.tcl", and the TCL file is stored in the path / home / hls / TestCases / Test_xxx / Test_Seed. The script execution program calls VivadoHLS through the Linux terminal to execute the TCL script for high-level synthesis. The command executed by the terminal is: vivado_hls-f / home / hls / TestCases / Test_xxx / Test_Seed / test_xxx_run_hls.tcl. The HDL code HP obtained after high-level synthesis is stored in the path / home / hls / TestCases / Test_xxx / Test_Seedproj_test_xxx_seed / solution_test_xxx_seed / syn / Verilog and named "Test_xxx_Seed.v". The "xxx" in the above file name and path is filled with the number of tests.

[0040] Step 4: Use the GCC compiler and Iverilog simulator to perform co-simulation on P and HP. Under the same TestBench, obtain and save the coverage C of P, the output result HLO of P, and the output result O of HP.

[0041] Specifically: Use the GCC compiler to compile and run P. The specific execution commands are: gccTest_xxx_Seed.cTest_xxx_TB.c-oTest_xxx_Seed and . / Test_xxx_Seed, get the output result HLO, use the Iverilog simulator to simulate HP, use TestBench to test P, the specific code is: iverilogTest_xxx_Seed.vTest_xxx_TB.v, get the output result O of HP. Use the gcov tool to get the coverage of P C. Among them, GCC is an open source compiler that can be used to compile and execute high-level language codes such as C / C++ and Java. Iverilog is a lightweight, free, open source Verilog compiler that can compile and simulate Verilog code. gcov is a command-line console program that is released with the GCC compiler and works with GCC to implement statement coverage and branch coverage tests for C / C++ files. Test_xxx_TB.v represents the TestBench file converted by Vivado HLS. The high-level language compiler and digital circuit simulator used in this method can also be selected by the user. The user only needs to complete the installation of the corresponding software and modify the calling function in this method. The "xxx" in the above file name and path is filled with the test number.

[0042] Step 5: Check whether the test case seed P and its converted code HP pass the simulation; if not, save P, TestBench, HP, HLO, O and the corresponding TCL file, generate and store the bug report, increase the test count by 1, and jump to step 2 to continue. If the simulation passes, proceed to the next step;

[0043] Specifically: Compare the output result HLO of P and the output result O of HDL under the same TestBench in step 4. If the two are consistent, it means that the simulation has passed, and the subsequent test steps will continue. If the two are inconsistent, it means that the simulation has not passed, then save P, TestBench, HP, HLO, O and the corresponding TCL file, generate and store the Bug report, increase the number of tests by 1, and jump to step 2 to continue. If the simulation passes, continue to the next step; wherein, the Bug report involved in the present invention mainly includes the seed test case code P, TestBench file, HP, HLO, O and the corresponding TCL file and other contents. The specific content of the Bug report can be configured by the tester according to the needs.

[0044] Step 6: Check whether the test case seed P is mutable;

[0045] Specifically, in this embodiment, the maximum number of mutations of the seed program is set to 10. This step detects the number of mutations of the seed program. If the number of mutations exceeds 10, no further mutations are allowed. The number of tests is increased by 1, and the process jumps to step 2. If not, the subsequent test is continued.

[0046] Step 7: According to the coverage C of the test case seed program P obtained in step 4, the code mutation program performs mutation operations such as code deletion, insertion, and conversion on the test case seed P to obtain the mutated test case P′;

[0047] Specifically: according to the coverage C of the test case seed program P obtained in step 4, the code mutation program randomly selects mutation operations such as code deletion, insertion, and conversion for P to obtain a new test case P′ after mutation. The file is named "Test_xxx_Mutation_yy.c" and saved in the path / home / hls / TestCases / Test_xxx / Test_xxx_Mutation_yy. Among them, code deletion is to delete the code in the test case seed code that is not covered by TestBench, code insertion is to insert the code that does not affect the program output and conforms to the circuit design into the local test case seed code, and code conversion is to merge, stream, parallelize and perform other operations on the for loops in the test case seed code according to the optional functions in the synthesis process of the tool to be tested, split, map and reorganize the array, and inline and reorganize the function. The "xxx" in the above file name and path is filled with the number of tests, and the "yy" is filled with the number of mutations;

[0048] Step 8: The script modification program modifies the original TCL script and saves it as a new TCL script. The script execution program calls Vivado HLS through the terminal command line to perform high-level synthesis on the mutated test case P′ obtained in step 7 according to the new TCL script to obtain the HDL code HP′.

[0049] Specifically: first, the script modification program obtains the original TCL file "test_xxx_run_hls.tcl" corresponding to the seed code, changes the project name to "proj_test_xxx_mutation_yy", changes the test case to "Test_xxx_Mutation_yy.c", and changes the solution name to "solution_test_xxx_mutation_yy", so that the modified new TCL file corresponds to the mutated test case P′, and saves the modified TCL file as a new TCL file named "test_xxx_mutation_yy_run_hls.tcl", and saves it in the / home / hls / TestCases / Test_xxx / Test_xxx_Mutation_yy path. Then, the script execution program calls Vivado HLS through the Linux terminal command line to perform high-level synthesis on the mutated test case P′ obtained in step 7 according to the new TCL script. The terminal execution command is: vivado_hls-f / home / hls / TestCases / Test_xxx / Test_xxx_Mutation_yy / test_xxx_mutation_yy_run_hls.tcl to obtain the HDL code HP′. The obtained HDL code HP′ is stored in the path / home / hls / TestCases / Test_xxx / proj_test_xxx_mutation_yy / solution_test_xxx_mutation_yy / syn / Verilog and is named "Test_xxx_Mutation_yy.v". The "xxx" in the above file name and path is filled with the number of tests and the "yy" is filled with the number of mutations;

[0050] Step 9: Use the GCC compiler and Iverilog simulator to perform co-simulation on P′ and HP′ to obtain the output HLO′ of P′ and the output O′ of HP′;

[0051] Specifically: In this step, the GCC compiler is used to compile and run P'. The specific execution code is: gccTest_xxx_Mutation_yy.cTest_xxx_TB.c-oTest_xxx_Mutation_yy and . / Test_xxx_Mutation_yy, and the output result HLO' is obtained. HP' is simulated using the Iverilog simulator, and P' is tested using the TestBench in step 4. HP' is tested using the TestBensh file synthesized by the advanced synthesis tool. The specific execution code is: iverilogTest_xxx_Mutation_yy.vTest_xxx_TB.v, and the HP' output result O' is obtained. The "xxx" in the above file name and path is filled with the number of tests, and the "yy" is filled with the number of mutations;

[0052] Step 10: Check whether the result O in step 4 is the same as the result O′ in step 9; if they are different, save P′, HLO′, TestBench, HP′, O′ and the corresponding TCL file, generate and store the bug report, increase the mutation count by 1, and jump to step 6 to continue. If the results are the same, directly increase the mutation count by 1 and jump to step 6 to continue.

[0053] When the above process reaches the limited number of tests, the test work is stopped. The method of the present invention completes the test work for the advanced synthesis tool by automatically running the above process. The method of the present invention collects a large number of seed programs, and through multiple mutation operations, a large number of test cases can be obtained, which greatly increases the number of tests and improves the efficiency of defect detection of advanced synthesis tools. The method of the present invention can effectively detect the defects of advanced synthesis tools through collaborative simulation.

Claims

1. A method for defect detection of advanced synthesis tools based on equivalent modulus testing, characterized in that: The specific steps are as follows: Step 1: Collect existing C language test case programs and build a test case seed pool; Use crawler technology to obtain test case programs from the AnghaBench library and CHStoneHLS test case library, filter out invalid programs, and build a test case seed pool; Step 2: Check whether the test times are reached; If it is not reached, a program file is randomly selected from the seed pool constructed in step 1 as the test case seed P, and the corresponding TestBench file is generated according to the test case seed; if it is reached, the test is stopped; The TestBench file contains test input, design to be tested and test output; the test input provides input data for the module to be tested, the design to be tested is the module to be tested, and the test output is the output result of the module to be tested under the current test input; Step 3: The script generation program generates and stores a TCL script according to the seed code P and the location where the TestBench file is stored. The script execution program calls the high-level synthesis tool to be tested through the terminal command line to perform high-level synthesis on the test case seed P according to the TCL script to obtain the HDL code HP. Specifically, first, the script generation program generates and stores a TCL script according to the seed code P and the location where the TestBench file is stored, so as to specify a new project name, a test case seed P and a TestBench file for the high-level synthesis tool to be tested, set a top-level function, set a solution name, define a technology and a clock rate, specify a synthesis process, set an optimization command, and run the high-level synthesis tool to be tested for synthesis through the TCL script; then, the script execution program calls the high-level synthesis tool to be tested through the Linux terminal to execute the TCL script for high-level synthesis, so as to obtain the HDL code HP; Step 4: Use a high-level language compiler and a digital circuit simulator to perform co-simulation, and obtain and save the coverage C of P, the output result HLO of P, and the output result O of HP under the same TestBench; Specifically, the program P is compiled and run using a high-level language compiler, the program HP is simulated using a digital circuit simulator, the program P is tested using TestBench, the program HP is tested using a TestBench file synthesized by a high-level synthesis tool, and the coverage information C of P, the output result HLO of P, and the output result O of HP are obtained and saved; Step 5: Check whether the test case seed P and the code HP obtained after high-level synthesis pass the simulation; Specifically, compare the output result HLO of P and the output result O of HP under the same TestBench in step 4; if the two are consistent, it means that the simulation has passed, and continue to the subsequent test steps; if the two are inconsistent, it means that the simulation has not passed, then save P, TestBench, HP, HLO, O and the corresponding TCL file, generate and store the Bug report, increase the number of tests by 1, and jump to step 2 to continue execution; if the simulation has passed, continue to the next test step; Step 6: Check whether the test case seed P is mutable; Specifically: Check the number of mutations of the seed program. If the number of mutations exceeds the set value, no further mutations are allowed, the number of tests increases by 1, and the program jumps to step 2 to continue. If the number of mutations does not exceed the set value, the program continues to perform subsequent tests. Step 7: According to the coverage information C of the test case seed program P obtained in step 4, the code mutation program performs mutation operations on the test case seed P, including code deletion, insertion and conversion, and obtains a new test case P′ after mutation; Among them, code deletion is to delete the code in the test case seed code that is not covered by the TestBench test, code insertion is to insert the code that does not affect the program output and conforms to the circuit design specification locally in the test case seed code, and code conversion is to merge, stream and parallelize the for loops in the test case seed code according to the optional functions in the synthesis process of the tool to be tested, split, map and reorganize the array, and inline and reorganize the function; Step 8: The script modification program modifies the original TCL script and saves it as a new TCL script. The script execution program calls the high-level synthesis tool to be tested through the terminal command line to perform high-level synthesis on the mutated test case P′ obtained in step 7 according to the new TCL script to obtain the HDL code HP′. Specifically, first, the script modification program obtains the original TCL file corresponding to the seed code, modifies the project name, test case and solution name therein, so that the new TCL file corresponds to the mutated test case P′, and saves the modified TCL file as a new TCL file; then, the script execution program calls the high-level synthesis tool to be tested through the Linux terminal command line to perform high-level synthesis on the mutated test case P′ obtained in step 7 according to the new TCL script to obtain the HDL code HP′; Step 9: Use a high-level language compiler and a digital circuit simulator to perform co-simulation on P′ and HP′, and use the TestBench in step 4 to obtain the output HLO′ of P′ and the output O′ of HP′; Specifically, the program P′ is compiled and run using a high-level language compiler, the program HP′ is simulated using a digital circuit simulator, P′ is tested using the TestBench in step 4, HP′ is tested using the TestBench file synthesized by a high-level synthesis tool, and the output HLO′ of the program P′ and the output O′ of the program HP′ are obtained; Step 10: Check whether the result O in step 4 is the same as the result O′ in step 9; if the results are different, save P′, HLO′, TestBench, HP′, O′ and the corresponding TCL file, generate and store the Bug report, increase the number of mutations by 1, and jump to step 6 to continue execution; if the results are the same, directly increase the number of mutations by 1, and jump to step 6 to continue execution.

Citation Information

Patent Citations

  • Method and system for supporting automatic testing and recording of EDA software

    CN111428431A

  • Method for realizing SystemC verification and verification platform component architecture

    CN112241347A

  • Code defect detection method and device based on cross validation

    CN108763092A

  • Defect detection and extraction method for C language project

    CN110750456A

  • Automatic detection method for Java software project dependency conflict semantic consistency

    CN112181858A