DRC test result verification method and device, equipment and storage medium

Abstract

The invention relates to a DRC test result verification method and device, equipment and a storage medium, and the method comprises the steps: executing a test case based on a to-be-tested tool, obtaining a test report of a to-be-tested circuit, and obtaining a standard result file corresponding to the to-be-tested circuit; establishing a matching instruction based on a preset test violation, establishing a to-be-tested matching instruction by adapting to the test report, and establishing a standard matching instruction by adapting to the standard result file; capturing a corresponding test result from the test report based on the to-be-tested matching instruction, and capturing a corresponding standard result from the standard result file based on the standard matching instruction; and matching the captured test result aiming at the same test violation with a standard result, and if the matching is detected to be successful, judging that the test result of the to-be-tested tool is correct. By adopting the method, the test accuracy of the self-developed to-be-tested tool can be efficiently determined.

Description

Technical Field

[0001] The present application relates to the technical field of circuit design, and in particular, to a method, device, equipment and storage medium for verifying DRC test results. Background Art

[0002] In modern integrated circuit design, DFT (Design for test) is a key link to ensure that the design can effectively detect and repair defects during the manufacturing and verification phases. Design rule check (DRC) is an important step to verify whether the design meets the pending design rules and process requirements. Applying DRC to DFT aims to ensure that the test structures introduced in the design comply with the design rules, thereby improving the testability of the design and the reliability of testing.

[0003] Traditional DRC functional tests rely on manual test verification, which usually has the following difficulties: the DRC check rules, digital circuit structures, and digital signals are all diverse, and the number of scenarios that can trigger the check rules in an actual circuit is extremely large. Manual testing consumes a lot of time and also results in incomplete scenario coverage. Correspondingly, DRC tests can be performed based on existing test tools or self-developed test tools. For self-developed test tools, the test report content may contain a large amount of test violation information. For different self-developed tools, their test reports contain a large amount of information, and their description structures, formats, etc. may all be different. Therefore, it is crucial to quickly determine the correctness of their inspection results.

[0004] Currently, no effective solution has been proposed for how to quickly determine the accuracy of the test results of self-developed test tools in the prior art. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a method, device, equipment and storage medium for verifying DRC test results.

[0006] In a first aspect, the present application provides a method for verifying DRC test results. The method includes:

[0007] Executing a test case based on a tool to be tested to obtain a test report of a circuit to be tested, and obtaining a standard result file corresponding to the circuit to be tested;

[0008] Establishing a matching instruction based on a preset test violation, where the matching instruction includes a to-be-tested matching instruction and a standard matching instruction, establishing the to-be-tested matching instruction adapted to the test report, and establishing the standard matching instruction adapted to the standard result file;

[0009] Scrape the corresponding test results from the test report based on the test matching instructions to be tested, and scrape the corresponding standard results from the standard result file based on the standard matching instructions;

[0010] Match the test results and standard results for the same test violation that have been scraped. If a successful match is detected, it is determined that the test results of the tool to be tested are correct.

[0011] In one embodiment, establishing matching instructions based on preset test violations includes:

[0012] Obtain the keywords for each violation item according to the preset test violations;

[0013] Determine the first character expression form of the keyword in the test report and the second character expression form in the standard result file;

[0014] For each violation item, establish the test matching instructions to be tested respectively based on the keyword and the corresponding first character expression form, and establish the standard matching instructions based on the keyword and the corresponding second character expression form.

[0015] In one embodiment, before obtaining the standard result file corresponding to the circuit to be tested, it further includes:

[0016] Obtain the test scenario of the tool to be tested;

[0017] Determine the test parameters matching the tool to be tested according to the test scenario;

[0018] Select the standard result file matching the tool to be tested according to the test parameters and the corresponding relationship between the preset test parameters and the standard result file.

[0019] In one embodiment, generating test cases includes:

[0020] Obtain the preset test template;

[0021] Obtain the first storage path corresponding to the netlist file of the circuit to be tested and the second storage path corresponding to the command configuration file at the preset test level;

[0022] Determine the test data tuple according to the storage paths corresponding to the netlist file and the command configuration file, and fill it into the test template to obtain the test case corresponding to the preset test level.

[0023] In one embodiment, the netlist file includes a first-level netlist file and a second-level netlist file. The first storage path includes the storage paths of the first-level netlist file and the second-level netlist file. The second storage path includes the storage path of the command configuration file corresponding to the first-level netlist file and the storage path of the command configuration file corresponding to the second-level netlist file. Determine the test data tuples according to the storage paths corresponding to the netlist file and the command configuration file, and fill them into the test template to obtain the test cases corresponding to the preset test levels, including:

[0024] Determine the first-level test data tuples according to the storage path of the first-level netlist file and the storage path of the command configuration file corresponding to the first-level netlist file, and fill the first-level test data tuples into the test template to obtain the first-level test cases. Determine the second-level test data tuples according to the storage path of the second-level netlist file and the storage path of the command configuration file corresponding to the second-level netlist file, and fill the second-level test data tuples into the test template to obtain the second-level test cases. Among them, the first-level test cases and the second-level test cases are different test levels of the same set of test cases.

[0025] In one embodiment, the method further includes:

[0026] Determine the type of the test case;

[0027] According to the type of the test case, call the tool to be tested and execute the test case. Among them, if the type of the test case is the first use case type, the first-level test cases are executed accordingly. If the type of the test case is the second use case type, the second-level test cases are executed accordingly;

[0028] Isolate and store the test reports corresponding to the test cases returned by each test process.

[0029] In one embodiment, the method further includes:

[0030] Output the matching result of the test result and the standard result to obtain a test report. Among them, the test report includes the test case execution result, the test report, the standard result file, and the matching result information.

[0031] In a second aspect, the present application further provides a verification device for DRC test results. The device includes:

[0032] A calculation module, configured to execute test cases based on the tool to be tested to obtain a test report, and obtain the standard result file corresponding to the circuit to be tested;

[0033] A calculation module, configured to establish matching instructions based on preset test violations, where the matching instructions include a to-be-tested matching instruction and a standard matching instruction, adapt to the test report to establish the to-be-tested matching instruction, and adapt to the standard result file to establish the standard matching instruction; grab the corresponding test results from the test report based on the to-be-tested matching instruction, and grab the corresponding standard results from the standard result file based on the standard matching instruction;

[0034] A generation module, configured to match the grabbed test results and standard results for the same test violation. If a successful match is detected, it is determined that the test results of the to-be-tested tool are correct.

[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0036] Execute test cases based on the to-be-tested tool to obtain a test report of the to-be-tested circuit, and obtain a standard result file corresponding to the to-be-tested circuit;

[0037] Establish matching instructions based on preset test violations, where the matching instructions include a to-be-tested matching instruction and a standard matching instruction, adapt to the test report to establish the to-be-tested matching instruction, and adapt to the standard result file to establish the standard matching instruction;

[0038] Grab the corresponding test results from the test report based on the to-be-tested matching instruction, and grab the corresponding standard results from the standard result file based on the standard matching instruction;

[0039] Match the grabbed test results and standard results for the same test violation. If a successful match is detected, it is determined that the test results of the to-be-tested tool are correct.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0041] Execute test cases based on the to-be-tested tool to obtain a test report of the to-be-tested circuit, and obtain a standard result file corresponding to the to-be-tested circuit;

[0042] Establish matching instructions based on preset test violations, where the matching instructions include a to-be-tested matching instruction and a standard matching instruction, adapt to the test report to establish the to-be-tested matching instruction, and adapt to the standard result file to establish the standard matching instruction;

[0043] Grab the corresponding test results from the test report based on the to-be-tested matching instruction, and grab the corresponding standard results from the standard result file based on the standard matching instruction;

[0044] Match the test results and standard results captured for the same test violation. If a successful match is detected, it is determined that the test results of the tool under test are correct.

[0045] The above DRC test result verification method, device, equipment, and storage medium execute test cases based on the tool under test to obtain a test report, and obtain a standard result file corresponding to the circuit under test; establish a matching instruction based on the test violation, and this matching instruction includes a test matching instruction and a standard matching instruction. Adapt the test matching instruction to the test report and adapt the standard matching instruction to the standard result file; capture the corresponding test results from the test report based on the test matching instruction, and capture the corresponding standard results from the standard result file based on the standard matching instruction; match the test results and standard results captured for the same test violation. If a successful match is detected, it is determined that the test results of the tool under test are correct. Through different matching instructions in this application, the corresponding test results are captured from the test report corresponding to the tool under test, and the corresponding standard results are captured from the standard result file, and the test results and standard results are matched. Thus, when the tool description structure and format change, only different matching instructions need to be established respectively for quick adaptation, without modifying the test cases. Furthermore, the test accuracy of the self-developed tool under test can be quickly determined. This method is fast, convenient, highly flexible, well adapted to the rhythm of agile development, and efficiently completes the judgment of the accuracy of the self-developed tool under test. Description of the Drawings

[0046] Figure 1 It is an application environment diagram of the DRC test result verification method in an embodiment;

[0047] Figure 2 It is a schematic flowchart of the DRC test result verification method in an embodiment;

[0048] Figure 3 It is a schematic diagram of the storage structure of a netlist file and a process configuration file in an embodiment;

[0049] Figure 4 It is a schematic flowchart of the DRC test result verification method in a preferred embodiment;

[0050] Figure 5 It is a schematic diagram of the DRC test system structure framework in an embodiment;

[0051] Figure 6 It is a structural block diagram of the DRC test result verification device in an embodiment;

[0052] Figure 7 It is an internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0053] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

[0054] The DRC test result verification method provided by the embodiments of the present application can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers. First, execute the test case based on the tool to be tested to obtain a test report, and obtain the standard result file corresponding to the circuit to be tested; then establish a matching instruction based on a preset test violation, where the matching instruction includes a test matching instruction to be tested and a standard matching instruction, and grab the corresponding test result from the test report based on the test matching instruction to be tested, and grab the corresponding standard result from the standard result file based on the standard matching instruction; match the grabbed test result and standard result for the same test violation, and if it is detected that the match is successful, it is determined that the test result of the tool to be tested is correct. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0055] In one embodiment, as Figure 2 shown, a test result verification method is provided. Taking the method applied to the Figure 1 server as an example, the method includes the following steps:

[0056] Step S210: Execute a test case based on the tool to be tested to obtain a test report of the circuit to be tested, and obtain the standard result file corresponding to the circuit to be tested.

[0057] Specifically, test cases are generated, where the test cases include, but are not limited to, Gate-Level test cases, RTL-level test cases (Register Transfer Level Test Case), etc. In practical applications, the generation of test cases can be implemented in a data-driven manner. In the test cases, the pytest.mark.parametrize() method provided by the pytest framework is used to implement data input. This method can find the test data path required for the use case in a preset file (keyword + data path) according to the keyword, and then pass the read test data path into the use case template to generate specific use cases. Among them, the test data corresponds to the circuit under test, and the test data characterizes the test conditions and test results (such as excitation vectors, boundary conditions, etc.) of the circuit under test.

[0058] Then, the test cases are executed by the tool under test to obtain a test report. Among them, the tool under test is a self-developed tool written by a computer program for verification testing of the circuit under test, which can run in environments such as computers and servers. The specific form and content of this self-developed tool are not limited here. The test accuracy of the tool under test can be tested through the test result verification method in this application. Further, the above test report is the test result of the tool under test applying the test data carried in the test cases to test the circuit under test. The test report includes DRC check result information, which reflects the log information related to the violations corresponding to the test cases.

[0059] The standard result file can be a design reference file that meets specific design requirements for the circuit under test. Optionally, the standard result file can be a process manual provided by a semiconductor manufacturer, which is more standardized and standard in the industry, a design specification file formulated by a design team and company, or a design case provided and verified by a third party, or it can also be internal design rules given according to specific circuit design products or experiences. This application does not limit this here. The standard result file gives a summary, detailed information, rule description, and repair suggestions of the design rules and related test violations. For different standard result files, the content such as the design description structure and format is different. The standard result file provides a benchmark for design and verification to ensure that the design meets manufacturing and performance requirements.

[0060] Step S220, establish matching instructions based on preset test violations, where the matching instructions include the matching instruction to be tested and the standard matching instruction, and the matching instruction to be tested is established according to the test report, and the standard matching instruction is established according to the standard result file.

[0061] Specifically, the preset test violation is obtained. The test violation is a violation of the design rule found in the design rule check (DRC). Some specific design structures in the digital logic circuit that are not friendly to the test will cause the circuit testability to decrease. According to the characteristics of these structures, some design rules can be specified. In the early stage of DFT design, these rules can be used to identify the structures in the logic circuit that are not conducive to the test. Engineers can optimize the circuit structure according to the inspection results to improve the testability of the circuit. Common violations include scan chain, port, timing control, signal connection, logic coverage, etc., such as scan chain violations such as length, number of chains not meeting the requirements or signals not being connected correctly; port violations such as not being connected or configured correctly; clock control violations such as clock no response or not being synchronized with the data path, some path timing not meeting the requirements, scan chain shift speed is low, etc.; logic coverage violations such as some key nodes are not covered by the DFT structure. Further, the above matching instructions correspond to test violations, and the matching instructions are used to capture corresponding content from the test report and standard result file.

[0062] Furthermore, since the test case is executed based on the tool to be tested in the present application to obtain a test report of the circuit to be tested, the test report format of the self-developed tool and the way of expressing violation information may be different from the standard result file. In order to efficiently complete the matching of the test results, the present embodiment is suitable for establishing matching instructions to be tested in the test report and for establishing standard matching instructions in the standard result file. Specifically, in order to better complete the capture of information, it is necessary to set corresponding matching instructions according to the structure, format and other information of the corresponding test report and standard result file. The standard matching instructions and matching instructions to be tested for the same test violation differ only in the form of expression, and their effects and captured objects are the same. Therefore, when the tool to be tested or the standard result file used changes, resulting in a corresponding change in the description of the test violation, only the matching instruction needs to be adjusted, without adjusting the expected results of each test case. For example, when the test report is: Clock input 'CK'of 'DFF''dff1'is uncontrolled., the corresponding matching instruction to be tested should be: Clock input '(\w+)'of 'DFF' '(\w+)' is uncontrolled. If another tool to be tested is replaced, resulting in the corresponding test report becoming: Clock input CK of DFF dff isuncontrolled., you only need to modify the matching instruction to: Clock input (\w+)of DFF (\w+) isuncontrolled. No need to modify anything in the test case.

[0063] In some preferred embodiments, the results matched by the matching instruction include the device name, the pins of the device, and the number of devices that trigger violations.

[0064] Step S230, retrieving the corresponding test results from the test report based on the test matching instruction to be tested, and retrieving the corresponding standard results from the standard result file based on the standard matching instruction.

[0065] Specifically, in some preferred embodiments, after establishing a matching instruction based on a preset test violation, according to the test matching instruction to be tested and the standard matching instruction respectively, the test results and standard results for the test violation are retrieved from the test report and the standard result file.

[0066] In some embodiments, the matching instruction may be a regular expression, that is, in this embodiment, information on DRC check results in the two files of the test report and the standard result file can be retrieved through the regular expression.

[0067] Step S240, matching the retrieved test results and standard results for the same test violation. If a successful match is detected, it is determined that the test results of the tool to be tested are correct.

[0068] Specifically, the retrieved test results and standard results for the same test violation are matched, so as to determine whether the execution result of the work is accurate. If it is detected that the violation information represented by the test results and the standard results is the same (such as both are interface not connected, etc.), it is determined that the test results of the tool to be tested are correct. In practical applications, the method of retrieving information for comparison by establishing different matching instructions for different tools is more flexible. When the tool changes, only the matching instruction needs to be adjusted, without the need to adjust each use case or modify any content in the use case. At the same time, if additional matching items are needed, only the matching instruction needs to be modified, without the need to design additional test cases.

[0069] Through steps S210 to S240, the corresponding test results are retrieved from the test report corresponding to the tool to be tested through different matching instructions, and the corresponding standard results are retrieved from the standard result file, and the test results and the standard results are matched. Thus, when the tool description structure and format change, only different matching instructions need to be established respectively to quickly adapt, without modifying the test cases, and then the test accuracy of the self-developed tool to be tested can be quickly determined. This method is fast, convenient, highly flexible, and well adapts to the rhythm of agile development.

[0070] In some of these embodiments, establishing a matching instruction based on a preset test violation includes:

[0071] Obtain the keywords for each violation item according to the preset test violations; determine the first character expression form of the keyword in the test report and the second character expression form in the standard result file; for each violation item, respectively establish a to-be-tested matching instruction based on the keyword and the corresponding first character expression form, and establish a standard matching instruction based on the keyword and the corresponding second character expression form.

[0072] Specifically, this embodiment provides a method for establishing a matching instruction, through which the corresponding test result can be extracted from the test report and the corresponding standard result can be retrieved from the standard result file. In some preferred embodiments, the matching instruction can be a regular expression.

[0073] First, determine the keywords for each violation item. The keywords include one or more of the following information: violation objects such as device inst, scan chain, port names, violation items such as timing, connection, speed, etc., key information such as pins of inst, violation types, violation quantities, etc. The above test violations can include multiple violation items, and each violation item is a situation where the DRC violates the design rules and is a fixed violation item.

[0074] Then, determine the first character expression form of the keyword in the test report and the second character expression form in the standard result file. Since the test cases are executed using a self-developed to-be-tested tool, the expression forms of the violation information in the generated test report and the standard result file are not exactly the same. In order to better retrieve the corresponding information through the matching instruction, different matching instructions need to be established to adapt to the test report and the standard result file. Therefore, it is necessary to determine the first character expression form in the test report and the second character expression form in the standard result file. The first character expression form represents information such as the format and expression rules of the characters in the test report, and the second character expression form is the same.

[0075] Finally, for each violation item, establish a to-be-tested matching instruction based on the keyword and the first character expression form, and establish a standard matching instruction based on the keyword and the second character expression form, so as to retrieve the corresponding test result from the test report based on the to-be-tested matching instruction. Similarly, retrieve the corresponding standard result from the standard result file based on the standard matching instruction.

[0076] Exemplarily, taking the DFTR1 violation as an example for illustration, the test report related to the DFTR1 violation of the to-be-tested tool is as follows:

[0077] The clock input 'CK' of 'DFF', 'dff1' is uncontrolled. Use the'manDFTDRC-1001' command to obtain more details about the log. (DFTR1-1);

[0078] 2) There were 1 DRC rule 'DFTR1' fails. (Clock input of DFF is uncontrolled);

[0079] Among them, the first item is the description information of DFTR1, which describes the name and type of the violated object inst, the type of DRC violation that occurred, and the number of the DRC violation. The second item of information is the number of DRC test violations of this type.

[0080] The expression forms of the standard result file and the description information of the tool to be tested will be different. For example, the standard result file information is as follows:

[0081] 1) Clock input CK of DFF, dff1 was not controlled. (D1-1);

[0082] From the above description information, it can be seen that for the result information of the same DFTR1 violation, the description method of the tool to be tested is different from that of the standard result file, and different matching instructions need to be established respectively.

[0083] First, extract the keywords. After analysis, it is determined that the content to be compared includes three types of information: the name of inst, the pins of inst, and the number of violations. Then determine the first character expression form of the keywords in the test report ('CK' of 'DFF') and the second character expression form of the keywords in the standard result file (CK of DFF).

[0084] Based on the above information, the following test matching instructions to be tested are established respectively based on the keywords and the corresponding first character expression forms:

[0085] 1) Clock input '(\w+)' of 'DFF' '(\w+)' is uncontrolled. ;

[0086] 2) There were (\d) DRC rule 'SCLK1' fails. ;

[0087] Establish the standard matching instruction based on the keyword and the corresponding second character expression form as follows:

[0088] Clock input (\w+) of DFF (\w+) is not controlled.

[0089] The relevant test information of DFTR1 can be matched from the test report through the matching instruction to be tested, and the information we need can be directly obtained from the text, including the name of the pin of DFF, the name of DFF, and the number of violations. Similarly, we can obtain the same information content from the standard result file through the standard matching instruction, and then compare the two pieces of information obtained to determine whether the execution result of the tool to be tested is correct. It can be seen from the above example that the corresponding comparison information can be matched by establishing the matching instruction to be tested and the standard matching instruction respectively, and at the same time, the numbers of CK, dff1, and DFTR1 can be captured for comparison and judgment.

[0090] The method of using different matching instructions to capture information from the test reports corresponding to different tools to be tested and compare them is more flexible. When the description information in the test report of the tool to be tested changes, only the matching instruction needs to be adjusted, without adjusting each test case. For example, when the description information is modified to Clock input CK of DFF dff is uncontrolled., we only need to modify the matching instruction to Clock input (\w+) of DFF (\w+) is uncontrolled. That's all, without modifying any content in the test case. At the same time, if additional matching items are needed, only the corresponding matching instruction needs to be modified, without designing additional test cases. This method well adapts to the rhythm of agile development.

[0091] By establishing different matching instructions based on the keywords and character expression forms in the test report and the standard result file to match the DRC inspection results, and then comparing the outputs of the self-developed tool and the standard result file, the test results can be quickly adapted without modifying each test case, and the accuracy of the test results can be judged.

[0092] In some of the embodiments, before obtaining the standard result file corresponding to the circuit to be tested, it further includes:

[0093] Obtain the test scenario of the tool to be tested; determine the test parameters matching the tool to be tested according to the test scenario; select the standard result file matching the tool to be tested according to the test parameters and the preset corresponding relationship between the test parameters and the standard result file.

[0094] Specifically, determine a preset test scenario and corresponding test parameters according to the test scenario. The test scenario includes, but is not limited to, timing, area, power consumption, logical consistency, etc. Under different test scenarios, different matching parameters are set correspondingly to select a corresponding standard result file according to the corresponding relationship between the preset test parameters and the standard result file.

[0095] In summary, in this embodiment, a suitable standard result file is selected through the required test scenario and test parameters, which is convenient for subsequent comparison with the test results of the tool to be tested. By selecting the standard result file to be compared according to the test parameters matching the tool to be tested, this method of optional standard result file effectively solves the deficiency that a single standard result file cannot meet the expectations of all design scenarios.

[0096] In some of these embodiments, generating test cases includes:

[0097] Obtain a preset test template;

[0098] Obtain the first storage path corresponding to the netlist file of the circuit to be tested and the second storage path corresponding to the command configuration file at a preset test level;

[0099] Determine test data tuples according to the storage paths corresponding to the netlist file and the command configuration file, and fill them into the test template to obtain test cases corresponding to the preset test level.

[0100] Specifically, the test template is an incomplete test case without test data, and the test templates corresponding to different levels of use cases (including but not limited to Gate-level test cases and RTL-level test cases) are the same.

[0101] Then, according to the preset test levels, determine the first storage path where the corresponding netlist file is located and the second storage path where the command configuration file is located. The above test levels include, but are not limited to, Gate-level and RTL levels. Correspondingly, here, when the preset test level is the Gate-level test case, obtain the first storage path where the netlist file corresponding to the Gate-level test case is located, and the second storage path where the command configuration file corresponding to the Gate-level test case is located. Similarly, when the preset test level is the RTL-level test case, obtain the first storage path where the netlist file corresponding to the RTL-level test case is located and the second storage path where the command configuration file corresponding to the RTL-level test case is located. Among them, the corresponding test data can be obtained according to the read path. In practical applications, the first-level netlist storage path and the second-level netlist storage path are mostly stored in the existing yaml file, which is mostly used for data-driven. The above netlist file is a circuit design file written for various scenarios that can trigger DRC violations. The netlist file corresponds to the test level, that is, it includes netlists at the Gate-Level and RTL levels.

[0102] Then, determine the test data tuple according to the storage paths corresponding to the netlist file and the command configuration file, that is, the corresponding test data can be read according to the storage path corresponding to the netlist, and the test data is assembled and the test data tuple (gate / rtl, verilog-path, tcl-path) is returned. Among them, "gate / rtl" indicates whether the test data is the corresponding Gate-level netlist or RTL-level netlist, "verilog-path" indicates the path of the test data, and "tcl-path" indicates the storage path corresponding to the tcl (Tool Command Language Script File) process configuration file. Fill the above test data tuple into the test template, and the Gate-level test case and the RTL-level test case can be obtained. It should be noted that the Gate-level test case and the RTL-level test case are two complete sets of test cases, and all the test cases are included in these two sets of test cases, that is, these two sets of test cases are different test levels of the same set of test cases.

[0103] In summary, through the present application, it is possible to automatically generate Gate-level or RTL-level test cases through data-driven, realize the generation of test cases for two scenarios with one set of test templates, and effectively improve the efficiency of test case generation.

[0104] In some of these embodiments, the netlist file includes a first-level netlist file and a second-level netlist file. The first storage path includes the storage paths of the first-level netlist file and the second-level netlist file, and the second storage path includes the storage path of the command configuration file corresponding to the first-level netlist file and the storage path of the command configuration file corresponding to the second-level netlist file. Test data tuples are determined based on the storage paths corresponding to the netlist file and the command configuration file and filled into the test template to obtain test cases corresponding to a preset test level, including:

[0105] Based on the storage path of the first-level netlist file and the storage path of the command configuration file corresponding to the first-level netlist file, first-level test data tuples are determined and filled into the test template to obtain first-level test cases. Based on the storage path of the second-level netlist file and the storage path of the command configuration file corresponding to the second-level netlist file, second-level test data tuples are determined and filled into the test template to obtain second-level test cases. Here, the first-level test cases and the second-level test cases are different test levels of the same set of test cases.

[0106] Specifically, this embodiment provides a specific method for generating test cases of different levels. The above-mentioned netlist file includes a first-level netlist file and a second-level netlist file, namely, the Gate-level netlist file and the RTL-level netlist file. Similarly, the first storage path includes the storage paths of the Gate-level netlist file and the RTL-level netlist file, and the second storage path includes the storage path of the command configuration file corresponding to the Gate-level netlist file and the storage path of the command configuration file corresponding to the RTL-level netlist file. Then, based on the storage path of the first netlist file and the storage path of the corresponding command configuration file, first-level test data tuples can be determined and filled into the test template to obtain the above-mentioned first-level test cases (Gate-level). Similarly, based on the storage path of the second-level netlist file and the storage path of the corresponding command configuration file, second-level test data tuples are determined and filled into the test template to obtain the above-mentioned second-level test cases (RTL-level).

[0107] In some of these embodiments, the method further includes:

[0108] Determine the type of the test case;

[0109] Based on the type of the test case, call the tool to be tested and execute the test case. Here, if the type of the test case is the first use case type, the first-level test cases are correspondingly executed; if the type of the test case is the second use case type, the second-level test cases are correspondingly executed.

[0110] Isolate and store the test reports corresponding to the test cases returned by each test process.

[0111] Specifically, this embodiment provides a method for executing test cases. First, it is necessary to clarify the storage path of the first-level netlist, the storage path of the process configuration file tcl corresponding to the first-level netlist, the storage path of the second-level netlist, and the storage path of the process configuration file tcl corresponding to the second-level netlist. The execution of test cases is achieved through the functions provided by the existing pytest framework. The tool to be tested is deployed on the test environment through a compiled installation package. The automated test system is connected to the test environment, and the DRC check tool is called through a Python script to execute test data. The tool to be tested is called through Python, and based on the interaction commands in the process configuration tcl file, the tool to be tested executes the test cases to perform DRC checks. The interaction with the test tool is implemented by selecting the subprocess module of Python. The run method of this module is used to generate a child process and interact with it, supporting the execution of external commands and programs and obtaining the output. Therefore, this module is very suitable for building a test system in our test scenario.

[0112] Then, the test file includes a test case type variable, that is, whether the test case to be executed is a first-level test case or a second-level test case, and the tool to be tested is called to execute the test case. Among them, if it is detected that the test case to be executed is of the first case type, the first-level test case, that is, the Gate-level test case, is executed accordingly. Similarly, if the test case to be executed is of the second case type, the second-level test case, that is, the RTL test case, is executed accordingly. Since the command flows in the process configuration files required for different levels of test cases are different, the test cases and the process configuration files need to be used in a matching manner. Moreover, in this embodiment, the test reports corresponding to the test cases returned by each test process are isolated and stored to ensure that the data of each test case does not interfere with each other, so that multiple test cases can be executed concurrently using multi-threading. In actual applications, the pytest-xdist plugin can be used to implement multi-threading. pytest-xdist is a plugin for distributed testing that can reduce the test time by running tests in parallel. In the test, the number of parallel jobs can be specified through the -n option. For example, pytest -n 16 means 16 parallel jobs. By concurrent execution, the execution time of the test cases is shortened to less than one-tenth of that of single-threading, greatly improving the test efficiency.

[0113] In one of the embodiments, the first-level netlist and the second-level netlist are stored in a preset first directory, and the process configuration file is stored in a preset second directory, and the first directory is not equal to the second directory.

[0114] Specifically, the first-level netlist and the second-level netlist are stored in a preset first directory, which can be an existing Verilog directory, and the process configuration file is stored in a preset second directory, which can be an existing dofile directory. The netlist file is a circuit design file written to trigger various DRC violation scenarios, and the file contains two levels of netlists, preferably Verilog netlists at the Gate-Level and RTL levels. The above process configuration file is a process file for the execution commands of the DRC tool.

[0115] By separating the storage of the netlist file and the process configuration file, the test data is decoupled from the test cases. The test cases and test data can be updated independently, reducing code modifications. At the same time, the same test data can be reused in multiple test cases, reducing code duplication and improving the maintainability, scalability, and flexibility of the test data. In summary, the storage structure of the netlist file and the process configuration file is as Figure 3 shown.

[0116] In one embodiment, both the test result and the standard result include the instance name of the violation, the interface name, and the number of violations.

[0117] Specifically, the test report and the standard result file include the relevant information of each test violation. The required violation name, interface name, and number can be directly obtained through the matching instruction. Furthermore, the test result obtained from the test report can be compared with the standard result obtained from the standard result file to determine whether the test result of the tool to be tested is accurate.

[0118] In one embodiment, the method further includes:

[0119] Outputting the matching result of the test result and the standard result to obtain a test report; wherein, the test report includes the execution result of the test case, the test report, the standard result file, and the matching result information.

[0120] Specifically, in this embodiment, the existing pytest-html plugin can be used to implement the function of generating the test report. This plugin can output the test report of pytest as an HTML format report that is easy to read, and the generated report is easy to view, enabling a quick understanding of the overall situation of the test. First, use the pip install pytest-html command to install the plugin, and configure pytest-html=report.html during runtime to generate a file named report.html. The generated HTML format test report contains the execution results of the test cases, log information, test statistics, etc., as well as the detailed output of each test. The reason for the failure of the test case can be better analyzed through the information and context of the case results in the file.

[0121] The present application also provides a preferred embodiment of a DRC test result verification method. Figure 4 For a preferred embodiment, it is a schematic flowchart of a test result verification scheme.

[0122] Step S410, build a test system framework. Specifically, in practical applications, the test system can be built based on the pytest framework (it can also be built based on other frameworks, such as unittest of Python, or other frameworks). The system structure is as Figure 5 shown. Among them, the common directory stores the general methods of the system, the config directory is used to store the configuration files of the system, the global directory is used to store the global variables used by the system, the liberty directory is used to store the library files required for the tool to run, the log directory is used to store the operation logs of the automation system, the logic directory is used to store some methods for the underlying logic processing of test cases, the report directory is used to store the execution logs of the DRC tool, that is, the test report, the resource directory is used to store the test source files, the testcase directory is used to store the test cases, the testdata directory is used to store the test data, that is, the netlist and the process configuration tcl file, and the utils directory is used to store utility functions and tools. By storing different methods in specific directories in this way, the overall standardization of the system is maintained.

[0123] Step S420, according to the type of the test case, call the tool to be tested and execute the test case. Among them, if the type of the test case is the first case type, then execute the first-level test case accordingly. If the type of the test case is the second case type, then execute the second-level test case accordingly.

[0124] Step S430, select a standard result file. Specifically, the standard result file can be selected through test parameters, and this parameter is set by the tester according to the test scenario of the tool to be tested.

[0125] Step S440, read test data. Specifically, the Gate-level test netlist path, tcl path, and the RTL test netlist path, tcl path can be obtained from the yaml file to obtain the test data.

[0126] Step S450, execute the test case. Specifically, call the tool to be tested, and based on the interaction commands in the process configuration file, the tool to be tested executes the above test case. Among them, if it is detected that the case type marked by a certain case is Gate-level, then execute the Gate-level case accordingly and skip the rtl-level case. Otherwise, the rtl-level case will be executed.

[0127] Step S460, test report output, specifically including that a test report can be generated through the pytest-html plugin to output an HTML format report that is easy to read. Further, the test report includes, but is not limited to, the execution results of test cases, log information, test statistics, etc., as well as the detailed output of each test case.

[0128] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the indications of the arrows, these steps do not necessarily need to be executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily need to be executed at the same moment, but can be executed at different moments. The execution order of these steps or stages does not necessarily need to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0129] Based on the same inventive concept, the embodiments of the present application further provide a DRC test result verification device for implementing the DRC test result verification method involved above. The implementation solution provided by this device to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more of the following embodiments of the test result verification device can refer to the limitations on the test result verification method in the above text, and will not be repeated here.

[0130] In one embodiment, as Figure 6 shown, a DRC test result verification device is provided, including: a calculation module 61 and a generation module 62, where:

[0131] The calculation module 61 is configured to execute test cases based on the tool to be tested to obtain a test report of the circuit to be tested, and obtain a standard result file corresponding to the circuit to be tested;

[0132] The calculation module 61 is configured to establish matching instructions based on preset test violations. The matching instructions include a to-be-tested matching instruction and a standard matching instruction. The to-be-tested matching instruction is adapted to be established for the test report, and the standard matching instruction is adapted to be established for the standard result file; grab the corresponding test results from the test report based on the to-be-tested matching instruction, and grab the corresponding standard results from the standard result file based on the standard matching instruction;

[0133] A generation module 62 is configured to match the test results and standard results captured for the same test violation. If a successful match is detected, it is determined that the test results of the tool under test are correct.

[0134] Each module in the above DRC test result verification device can be implemented in whole or in part by software, hardware, and their combinations. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0135] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 7 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the test result verification method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a test result verification method.

[0136] Those skilled in the art can understand that Figure 7 the structure shown in

[0137] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0138] Execute the test case based on the tool under test to obtain a test report of the circuit under test, and obtain the standard result file corresponding to the circuit under test;

[0139] Establish a matching instruction based on a preset test violation. The matching instruction includes a test matching instruction and a standard matching instruction. Adapt the test matching instruction to the test report and the standard matching instruction to the standard result file;

[0140] Capture the corresponding test results from the test report based on the test matching instruction, and capture the corresponding standard results from the standard result file based on the standard matching instruction;

[0141] Match the captured test results and standard results for the same test violation. If a successful match is detected, it is determined that the test results of the tool to be tested are correct.

[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties.

[0143] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0144] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0145] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A DRC test result verification method, characterized in that, The method includes: Executing test cases based on the tool to be tested, obtaining a test report of the circuit to be tested, and acquiring the standard result file corresponding to the circuit to be tested; Establishing matching instructions based on preset test violations, where the matching instructions include the to-be-tested matching instructions and the standard matching instructions, establishing the to-be-tested matching instructions adapted to the test report, and establishing the standard matching instructions adapted to the standard result file; Fetching the corresponding test results from the test report based on the to-be-tested matching instructions, and fetching the corresponding standard results from the standard result file based on the standard matching instructions; Matching the fetched test results and the standard results for the same test violation, and if successful matching is detected, determining that the test results of the tool to be tested are correct.

2. The method according to claim 1, characterized in that The establishing matching instructions based on preset test violations includes: Obtaining keywords for each violation item according to the preset test violations; Determining the first character expression form of the keywords in the test report and the second character expression form of the keywords in the standard result file; For each violation item, respectively establishing the to-be-tested matching instructions based on the keywords and the corresponding first character expression form, and establishing the standard matching instructions based on the keywords and the corresponding second character expression form.

3. The method according to claim 1, wherein Before acquiring the standard result file corresponding to the circuit to be tested, it further includes: Obtaining the test scenario of the tool to be tested; Determining the test parameters matching the tool to be tested according to the test scenario; Selecting the standard result file matching the tool to be tested according to the test parameters and the corresponding relationship between the preset test parameters and the standard result file.

4. The method according to claim 1, wherein Generating the test cases includes: Obtaining a preset test template; Obtaining the first storage path corresponding to the netlist file of the circuit to be tested and the second storage path corresponding to the command configuration file at the preset test level; Determining test data tuples according to the storage paths corresponding to the netlist file and the command configuration file, and filling them into the test template to obtain the test cases corresponding to the preset test level.

5. The method according to claim 4, wherein The netlist file includes a first-level netlist file and a second-level netlist file. The first storage path includes the storage path of the first-level netlist file and the storage path of the second-level netlist file. The second storage path includes the storage path of the command configuration file corresponding to the first-level netlist file and the storage path of the command configuration file corresponding to the second-level netlist file. The determining test data tuples according to the storage paths corresponding to the netlist file and the command configuration file, and filling them into the test template to obtain the test cases corresponding to the preset test level includes: Determine a first-level test data tuple according to the storage path of the first-level netlist file and the storage path of the command configuration file corresponding to the first-level netlist file, and fill the first-level test data tuple into the test template to obtain a first-level test case; determine a second-level test data tuple according to the storage path of the second-level netlist file and the storage path of the command configuration file corresponding to the second-level netlist file, and fill the second-level test data tuple into the test template to obtain a second-level test case, where the first-level test case and the second-level test case are different test levels of the same set of test cases.

6. The method according to claim 4, wherein The method further includes: Determine the type of the test case; According to the type of the test case, call the tool to be tested and execute the test case. Wherein, if the type of the test case is the first case type, the first-level test case is executed accordingly; if the type of the test case is the second case type, the second-level test case is executed accordingly; Isolatedly store the test reports corresponding to the test cases returned by each test process.

7. The method according to claim 1, characterized in that The method further includes: Output the matching result of the test result and the standard result to obtain a test report; wherein the test report includes the test case execution result, the test report, the standard result file and the matching result information.

8. A verification device for DRC test results, characterized in that The device includes: A calculation module, configured to execute a test case based on the tool to be tested to obtain a test report of the circuit to be tested, and obtain a standard result file corresponding to the circuit to be tested; The calculation module is further configured to establish a matching instruction based on a preset test violation, where the matching instruction includes a to-be-tested matching instruction and a standard matching instruction, establish the to-be-tested matching instruction adapted to the test report, and establish the standard matching instruction adapted to the standard result file; grab the corresponding test result from the test report based on the to-be-tested matching instruction, and grab the corresponding standard result from the standard result file based on the standard matching instruction; A generation module, configured to match the grabbed test result and the standard result for the same test violation, and if it is detected that the match is successful, determine that the test result of the tool to be tested is correct.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images