Natural language-oriented logic synthesis tool test method and test framework

Abstract

The invention belongs to the technical field of comprehensive tool testing, and particularly relates to a natural language-oriented logic comprehensive tool testing method and a natural language-oriented logic comprehensive tool testing framework. The method comprises the steps that after keyword groups are obtained, a random selection module automatically selects keywords and inputs the keywords into a test requirement module; the test demand module generates and inputs statements recognized by the large language model to obtain an RTL design meeting the test demand; the RTL design passing the grammar check enters a self-defined logic comprehensive process, the design which does not pass the grammar check enters a data statistics module, and cases and error reasons are recorded; the logic comprehensive process is successfully executed to enter the equivalence verification module and then enter the data statistics module, and whether an exit condition is met or not is checked. The problems that all command options, boundary conditions and some complex design scenes of the comprehensive tool are difficult to cover, the test efficiency is low, and the coverage rate is not high are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of synthesis tool testing, and more specifically, relates to a natural language-oriented logic synthesis tool testing method and testing framework. Background Art

[0002] Logic synthesis is the process of converting a high-level description of a digital design into an optimized gate-level netlist based on a standard cell library and specific design constraints. Logic synthesis tools mainly include three processes: translation, optimization, and mapping.

[0003] Chinese patent document CN113434390A discloses a mutation-based fuzz testing method for FPGA logic synthesis tools. The method first constructs a suitable test case set for the logic synthesizer. Then, while complying with Verilog HDL syntax rules, mutation operators are set and mutation operations are performed on the test case seed program to obtain more effective test cases. The FPGA logic synthesis tool under test is then executed to synthesize the test case program and convert it into a synthesized netlist that matches the hardware resources. Finally, the FPGA tool is determined to have defects by verifying the equivalence of the netlist with the pre-synthesis Verilog design.

[0004] Existing testing methods mainly rely on manually written test cases or rule-based automated tools to generate random cases. They are highly targeted, but it is difficult to cover all command options, boundary conditions and some complex design scenarios of the synthesis tool. There are problems of low testing efficiency and low coverage. Summary of the Invention

[0005] This invention aims to overcome at least one of the above-mentioned shortcomings of the prior art by providing a natural language-based logic synthesis tool testing method and framework. This approach addresses the problems of low efficiency, insufficient coverage, and time-consuming testing of existing logic synthesis tool testing methods. This approach improves test coverage, enhances tool accuracy and stability, reduces verification costs, and accelerates error location.

[0006] The detailed technical solutions of the present invention are as follows: A natural language-oriented logic synthesis tool testing method, the method comprising: S1. The user generates a keyword group and a custom TCL file; the keyword group randomly selects keywords to generate test requirements, and further, the test requirements are composed of natural language; S2. Obtain the generated test requirements, call the large language model to generate and output the RTL design file; S3. Perform a syntax check on the RTL design file. If the syntax check passes, proceed to the next step, and the RTL design file is used for subsequent equivalence verification. If the syntax check module fails, the case and the error cause are recorded and the process proceeds to S6; S4. Execute the customized logic synthesis process, i.e., call the customized TCL file. If the execution is successful, the synthesized netlist file is output and the process goes to S5. If the execution is unsuccessful, the case and the error cause are recorded and the process goes to S6. S5: Enter the equivalence verification process to verify whether the netlist file output by the logic synthesis tool is consistent with the RTL design file; if the consistency check fails, record the case and error cause and enter S6; If the consistency check passes, record this case and proceed to S6; S6. Check whether the exit conditions are met: If the exit condition is met, the test case result table and error case table are generated and output; If the exit condition is not met, continue to select keywords from the keyword group and repeat S2 to S6 until the exit condition is met.

[0007] Furthermore, the equivalence verification is specifically as follows: The RTL design file before synthesis is used as the golden file, and the netlist file output after synthesis is used as the revise file. The equivalent verification tool is used to perform logic consistency check.

[0008] Furthermore, the logic synthesis process includes translation, optimization and mapping.

[0009] On the other hand, the present invention also includes a natural language-oriented logic synthesis tool testing framework for implementing a natural language-oriented logic synthesis tool testing method, specifically including: a logic synthesis tool internal module and a logic synthesis tool external module; The modules in the logic synthesis tool include a random keyword selection module, a test requirement generation module, a large language model, a grammar checking module, and a logic synthesis module; The logic synthesis tool external module includes an equivalence verification module, a data statistics module, an exit condition judgment module and a report generation module; The user generates a keyword group and a custom TCL file. After obtaining the keyword group, the random keyword selection module will automatically select the keyword and input the keyword into the test requirement generation module; The test requirement generation module generates test requirements based on keywords and inputs a large language model to generate RTL design files that meet the test requirements; Perform syntax check on the RTL design file. If the syntax check passes, perform logic synthesis. If the syntax check fails, enter the data statistics module to record the case and error cause. After the logic synthesis process reads the RTL design files that pass the syntax check module, it executes the user-generated custom TCL files. RTL design files that fail the custom synthesis process enter the data statistics module to record cases and error causes. If the logic synthesis process is successfully executed, it will enter the equivalence verification module, and then enter the data statistics module to record the successful execution cases; After the data statistics module is completed, it enters the exit condition judgment module, that is, to determine whether the number of executed cases meets the specified number: If not, the random keyword selection module continues to select keywords from the keyword group and repeats the above process until the exit condition is met; If it is satisfied, it will enter the report generation module and output the result table and error table for researchers to analyze.

[0010] In another aspect of the present invention, a computer-readable storage medium is provided, which stores executable instructions. When the instructions are executed, the machine executes the natural language-oriented logic synthesis tool testing method as described above.

[0011] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention provides a natural language-oriented logic synthesis tool testing method and testing framework. By using a large language model to generate diverse RTL design files, the need for manual test case writing is eliminated, thereby improving test coverage and verification efficiency. At the same time, a variety of test cases can be generated according to test requirements, including unreasonable designs and potential boundary conditions, thereby improving the stability and accuracy of the logic synthesis tool.

[0012] (2) The present invention provides a natural language-oriented logic synthesis tool testing method and testing framework, which generates a result table and an error table of the test case by extracting data information from the test results, thereby accelerating the location of the error cause and shortening the inspection time. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flow chart of the natural language-oriented logic synthesis tool testing method described in the present invention.

[0014] Figure 2 This is a common logic synthesis test flow chart in Example 1 of the present invention.

[0015] Figure 3 This is a schematic diagram of a natural language-oriented logic synthesis tool testing framework in Example 2 of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0018] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0019] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0020] Example 1 Ginseng Figure 1 This embodiment provides a natural language-oriented logic synthesis tool testing method, including: S1. The user generates a keyword group and a custom TCL file. Keywords are randomly selected from the keyword group and test requirements written in natural language are generated based on the selected keyword group. Furthermore, the number of times keywords are selected from the keyword group depends on an exit condition. If the exit condition is not met, keywords will continue to be selected and permuted and combined until the exit condition is met.

[0021] For example, if the keyword group is "adder; multiplier", the first keyword obtained in sequence is adder, and the generated test requirement can be "Please use Verilog language to generate a design containing keyword 1".

[0022] S2. After obtaining the generated test requirements, call the large language model to generate and output the RTL design file. Preferably, it should be noted that the present invention allows the large language model to generate unreasonable RTL design, which is part of the accuracy test of the logic synthesis tool.

[0023] Preferably, the large language model can be Gemini 2.5 Pro (Exp), DeepSeek V3, QWQ-2.5Max (Qwen), etc.

[0024] S3. Perform a syntax check on the RTL design file. If the syntax check passes, proceed to the next step, and the RTL design file is used for subsequent equivalence verification. If the syntax check module fails, the case and the error cause are recorded and the process proceeds to S6; S4. Execute a customized logic synthesis process. The synthesis process calls a TCL file, which is a user-defined TCL file. Different test requirements require different command processes to be executed. This method can avoid setting redundant commands and speed up execution. The logic synthesis process includes translation, optimization and mapping.

[0025] If the execution is successful, the synthesized netlist file is output and the process goes to S5; If the execution is unsuccessful, for example, the logic synthesis tool crashes during the execution of the case, the case and the cause of the error are recorded and the process proceeds to S6.

[0026] Preferably, the logic synthesis tool used is such as Synopsys Design Compiler or Cadence Genus.

[0027] S5: Enter the equivalence verification process to verify whether the netlist file output by the logic synthesis tool is consistent with the RTL design file; if the consistency check fails, record the case and error cause and enter S6; If the consistency check passes, record this case and proceed to S6; Furthermore, the equivalence verification is specifically as follows: The RTL design file before synthesis is used as the golden file, and the netlist file output after synthesis is used as the revise file. The equivalent verification tool is used to perform logic consistency check.

[0028] S6. Check whether the exit conditions are met: If the exit conditions are met, the result table and error case table are output for case analysis and error location; If the exit condition is not met, continue to select keywords from the keyword group and repeat S2 to S6 until the exit condition is met.

[0029] Existing tool testing processes such as Figure 2 As shown, the user generates a TCL file and an RTL design, which are input into a logic synthesis process together. The logic synthesis process includes translation, optimization, and mapping, and then performs equivalence verification. The RTL design is generated based on rules through an automated tool or written manually.

[0030] Existing testing methods mainly rely on manually written test cases or rule-based automated tools to generate random cases. They are highly targeted, but it is difficult to cover all command options, boundary conditions and some complex design scenarios of synthesis tools. There are problems of low testing efficiency and low coverage.

[0031] Example 2 This embodiment provides a natural language-oriented logic synthesis tool testing framework, such as Figure 3 As shown, the framework is used to implement a natural language-oriented logic synthesis tool testing method, specifically including: a logic synthesis tool internal module and a logic synthesis tool external module; The modules in the logic synthesis tool include a random keyword selection module, a test requirement generation module, a large language model, a grammar checking module, and a logic synthesis module; The logic synthesis tool external module includes an equivalence verification module, a data statistics module, an exit condition judgment module and a report generation module; The user generates a keyword group and a custom TCL file. After obtaining the keyword group, the random keyword selection module will automatically select the keyword and input the keyword into the test requirement generation module; The test requirement generation module generates test requirements based on keywords and inputs a large language model to generate RTL design files that meet the test requirements; Perform syntax check on the RTL design file. If the syntax check passes, perform logic synthesis. If the syntax check fails, enter the data statistics module to record the case and error cause. After the logic synthesis process reads the RTL design files that pass the syntax check module, it executes the user-generated custom TCL files. RTL design files that fail the custom synthesis process enter the data statistics module to record cases and error causes. If the logic synthesis process is successfully executed, it will enter the equivalence verification module, and then enter the data statistics module to record the successful execution cases; After the data statistics module is completed, it enters the exit condition judgment module, that is, to determine whether the number of executed cases meets the specified number: If not, the random keyword selection module continues to select keywords from the keyword group and repeats the above process until the exit condition is met; If it is satisfied, it will enter the report generation module and output the result table and error table for researchers to analyze.

[0032] This embodiment relates to a natural language-oriented logic synthesis tool testing framework. The invention encompasses the use of a large language model to automatically generate diverse RTL design files for testing logic synthesis tools, improving test coverage, rapidly locating test errors, and simultaneously testing the stability and accuracy of the logic synthesis tool. This invention can be applied to the field of integrated circuit design, particularly in the verification testing of logic synthesis tools, improving test efficiency, reducing development costs, and shortening product time to market.

[0033] Example 3 This embodiment further provides a computer-readable storage medium storing executable instructions, which, when executed, enable the machine to execute the natural language-oriented logic synthesis tool testing method as described above.

[0034] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.

[0035] In this case, the program code itself read from the computer-readable medium can realize the function of any one of the above embodiments, and thus the computer-readable code and the computer-readable storage medium storing the computer-readable code constitute part of this specification.

[0036] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-Rs, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs, DVD-RWs), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or a cloud via a communication network.

[0037] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0038] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A natural language-oriented logic synthesis tool testing method, characterized in that: The method comprises: S1. The user generates a keyword group and a custom TCL file; the keyword group randomly selects keywords to generate test requirements; S2. Obtain the generated test requirements, call the large language model to generate and output the RTL design file; S3, performing a syntax check on the RTL design file. If the syntax check passes, proceed to S4, where the RTL design file that passes the syntax check is used for equivalence verification; If the syntax check module fails, the case and the error cause are recorded and the process proceeds to S6; S4. Execute the customized logic synthesis process, i.e., call the customized TCL file. If the execution is successful, the synthesized netlist file is output and the process goes to S5. If the execution is unsuccessful, the case and the error cause are recorded and the process goes to S6. S5. Enter the equivalence verification process to verify whether the netlist file output by the logic synthesis tool is consistent with the RTL design file. The RTL design file before synthesis is used as the golden file, and the netlist file output after synthesis is used as the revise file. Use the equivalence verification tool to perform logic consistency check; If the consistency check fails, record the case and the cause of the error and proceed to S6; If the consistency check passes, record this case and proceed to S6; S6. Check whether the exit conditions are met: If the exit condition is met, the test case result table and error case table are generated and output; If the exit condition is not met, continue to select keywords from the keyword group and repeat S2 to S6 until the exit condition is met.

2. A natural language oriented logic synthesis tool testing method according to claim 1, characterized in that: The test requirements are composed of natural language.

3. A natural language oriented logic synthesis tool testing method according to claim 2, characterized in that: The logic synthesis process includes translation, optimization and mapping.

4. A natural language-oriented logic synthesis tool testing framework, characterized in that: The framework specifically includes: a logic synthesis tool internal module and a logic synthesis tool external module; The modules in the logic synthesis tool include a random keyword selection module, a test requirement generation module, a large language model, a grammar checking module, and a logic synthesis module; The logic synthesis tool external module includes an equivalence verification module, a data statistics module, an exit condition judgment module and a report generation module; The user generates a keyword group and a custom TCL file. After obtaining the keyword group, the random keyword selection module will automatically select the keyword and input the keyword into the test requirement generation module; The test requirement generation module generates test requirements based on keywords and inputs a large language model to generate RTL design files that meet the test requirements; Perform syntax check on the RTL design file. If the syntax check passes, execute logic synthesis; if the syntax check fails, enter the data statistics module to record the case and error cause; After the logic synthesis process reads the RTL design files that pass the syntax check module, it executes the user-generated custom TCL files. RTL design files that fail the custom synthesis process enter the data statistics module to record cases and error causes. If the logic synthesis process is successfully executed, it will enter the equivalence verification module, and then enter the data statistics module to record the successful execution cases; After the data statistics module is completed, it enters the exit condition judgment module, that is, to determine whether the number of executed cases meets the specified number: If not, the random keyword selection module continues to select keywords from the keyword group and repeats the above process until the exit condition is met; If satisfied, enter the report generation module and output the result table and error table.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images