Verilog generation verification method and device for multi-modal circuit design specification

Abstract

The invention provides a Verilog generation verification method and device for multi-modal circuit design specifications, and relates to the technical field of artificial intelligence. The method comprises the steps of obtaining a circuit code data set, generating a first group of circuit pictures, performing natural language description by using a large language model, and constructing a multi-modal circuit data set; generating a second group of circuit pictures according to the Python code, and constructing a multi-modal universal data set; performing fine tuning training on the multi-modal large language model; customizing cue words for different tasks, and constructing a plurality of task agents according to the cue words and the trained multi-modal large language model; and designing a core interaction agent, and completing circuit design, circuit verification and circuit knowledge question and answer tasks according to the core interaction agent and the plurality of task agents. The circuit design verification accuracy can be improved, and the design verification time is shortened.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and integrated circuit technology, and in particular to a Verilog generation and verification method and apparatus for multimodal circuit design specifications. Background Technology

[0002] Large language models have made significant progress in code generation for programming languages ​​such as C and Python. However, extending them to the field of electronic design automation, especially in automatically generating and validating Verilog hardware description language from natural language specifications, faces numerous challenges. The main difficulty lies in the fact that natural language specifications typically only provide functional requirements descriptions, lacking rigorous formal definitions and failing to accurately express hardware characteristics such as timing, parallelism, and modular design in Verilog. This often results in generated Verilog code that does not match the expected hardware behavior. In recent years, the development of multimodal and multi-agent technologies has provided new opportunities to address this problem.

[0003] Multimodal large language models, by processing multiple types of input data simultaneously, are able to understand and generate information in richer and more complex contexts. In terms of code generation, they can combine graph structures of code (such as abstract syntax trees, control flow graphs, dependency graphs, etc.) to represent complex relationships between program elements, thereby improving the ability to understand, reason about, and generate code.

[0004] Multi-agent technology overcomes the limitations of a single agent in handling complex reasoning and multi-step decision-making by having multiple autonomous agents (usually instantiated from large language models) collaborate and divide tasks among themselves. In code generation tasks, each agent undertakes responsibilities such as analysis, design, implementation, and testing, working together through collaboration and information sharing to solve highly complex code generation problems. This division of roles effectively reduces the cognitive burden on a single model when dealing with complex logic, and by introducing testing agents and other post-verification methods, it improves the correctness and stability of the final code results.

[0005] Existing technologies primarily focus on generating circuit code from text-based modal circuit design specifications, commonly employing techniques such as training models using natural language and Verilog code pairs. However, these technologies lack a unified modeling and understanding capability for multimodal design specifications, struggling to handle common multimodal information found in design specifications such as circuit block diagrams, timing diagrams, state transition diagrams, and flowcharts, resulting in low functional correctness of the generated code. Furthermore, their automated verification capabilities are limited, with low test coverage, failing to effectively guide circuit optimization. In summary, existing Verilog code generation methods still have shortcomings in multimodal understanding, functional correctness verification, and reasoning capabilities. Summary of the Invention

[0006] To address the issue of existing circuit design specifications containing multiple modalities, and the challenge of designing a multimodal, multi-agent circuit design verification system to overcome the poor Verilog code generation capabilities of large language models and the inability to simultaneously perform design and verification, this invention provides a Verilog generation and verification method and apparatus for multimodal circuit design specifications. The technical solution is as follows:

[0007] On the one hand, a Verilog generation and verification method for multimodal circuit design specifications is provided. This method is implemented by a Verilog generation and verification device and includes:

[0008] S1. Obtain the circuit code dataset, generate the first set of circuit images based on the circuit code dataset, use a large language model to perform natural language description on the circuit code dataset, obtain the natural language description results of the circuit code, and then construct a multimodal circuit dataset; wherein, the first set of circuit images includes circuit structure diagram, state transition diagram, timing diagram and flowchart.

[0009] S2. Generate a second set of circuit images based on the Python code, use a large language model to perform natural language description on the Python code, obtain the natural language description results of the Python code, and then construct a multimodal general dataset.

[0010] S3. Fine-tune the multimodal large language model based on the multimodal circuit dataset and the multimodal general dataset to obtain the trained multimodal large language model.

[0011] S4. Customize prompt words for different tasks, and construct multiple task agents based on the prompt words and the trained multimodal large language model.

[0012] S5. Design the core interactive agent, and complete circuit design, circuit verification, and circuit knowledge question-and-answer tasks based on the core interactive agent and multiple task agents.

[0013] Optionally, in S1, a first set of circuit images is generated based on the circuit code dataset, and a large language model is used to perform natural language description on the circuit code dataset to obtain the natural language description results of the circuit codes, thereby constructing a multimodal circuit dataset, including:

[0014] For each circuit code in the circuit code dataset, a corresponding circuit image is generated using a script to obtain the first set of circuit images; a large language model is used to perform natural language description on the circuit code dataset to obtain the natural language description results of the circuit codes; a multimodal circuit dataset is constructed based on the first set of circuit images and the corresponding natural language description results of the circuit codes.

[0015] Optionally, in S2, a second set of circuit images is generated based on the Python code, and a large language model is used to perform natural language description of the Python code to obtain the natural language description results of the Python code. This leads to the construction of a multimodal general dataset, including:

[0016] A second set of circuit images is generated based on the Python code. The Python code is then described in natural language using a large language model to obtain the natural language description results of the Python code. A multimodal general dataset is constructed based on the second set of circuit images and the corresponding natural language description results of the Python code. The second set of circuit images includes circuit structure diagrams, state transition diagrams, timing diagrams, and flowcharts.

[0017] Optionally, S2 generates a second set of circuit diagrams based on Python code, including:

[0018] The second set of circuit images is constructed by using Python code to call relevant libraries for generating circuit structure diagrams, state transition diagrams, timing diagrams, and flowcharts, and randomly generating multiple circuit images.

[0019] Optionally, S3 also includes:

[0020] Design a multimodal design specification understanding evaluation benchmark to evaluate the trained multimodal large language model.

[0021] Optionally, S4 provides customized prompts for different tasks, and constructs multiple task agents based on these prompts and a trained multimodal large language model, including:

[0022] Customized prompts are provided for circuit design, circuit verification, and circuit knowledge question-and-answer tasks. For any task, multiple task agents are constructed based on the prompts and a trained multimodal large language model.

[0023] Optionally, in S5, the core interactive agent and multiple task agents complete circuit design, circuit verification, and circuit knowledge question-answering tasks, including:

[0024] The core interactive agent reads user input and determines the user's task, assigns the user's task to multiple task agents for execution, integrates the outputs of multiple task agents, and returns the integrated result to the user; among them, the user's tasks include circuit design, circuit verification, and circuit knowledge Q&A.

[0025] On the other hand, a Verilog generation and verification apparatus for multimodal circuit design specifications is provided. This apparatus is applied to a Verilog generation and verification method for multimodal circuit design specifications. The apparatus includes:

[0026] The multimodal circuit dataset construction module is used to acquire a circuit code dataset, generate the first set of circuit images based on the circuit code dataset, use a large language model to perform natural language description on the circuit code dataset, obtain the natural language description results of the circuit code, and then construct the multimodal circuit dataset; wherein, the first set of circuit images includes circuit structure diagram, state transition diagram, timing diagram and flowchart.

[0027] The multimodal general dataset building module is used to generate a second set of circuit images based on Python code, use a large language model to perform natural language description on the Python code, obtain the natural language description results of the Python code, and then build a multimodal general dataset.

[0028] The training module is used to fine-tune the multimodal large language model based on the multimodal circuit dataset and the multimodal general dataset to obtain the trained multimodal large language model.

[0029] The module is used to customize prompt words for different tasks and build multiple task agents based on the prompt words and the trained multimodal large language model.

[0030] The output module is used to design the core interactive agent, and to complete circuit design, circuit verification, and circuit knowledge question-and-answer tasks based on the core interactive agent and multiple task agents.

[0031] Optionally, the multimodal circuit dataset building module is further used for:

[0032] For each circuit code in the circuit code dataset, a corresponding circuit image is generated using a script to obtain the first set of circuit images; a large language model is used to perform natural language description on the circuit code dataset to obtain the natural language description results of the circuit codes; a multimodal circuit dataset is constructed based on the first set of circuit images and the corresponding natural language description results of the circuit codes.

[0033] Optionally, the multimodal general dataset building module is further used for:

[0034] A second set of circuit images is generated based on the Python code. The Python code is then described in natural language using a large language model to obtain the natural language description results of the Python code. A multimodal general dataset is constructed based on the second set of circuit images and the corresponding natural language description results of the Python code. The second set of circuit images includes circuit structure diagrams, state transition diagrams, timing diagrams, and flowcharts.

[0035] Optionally, the multimodal general dataset building module is further used for:

[0036] The second set of circuit images is constructed by using Python code to call relevant libraries for generating circuit structure diagrams, state transition diagrams, timing diagrams, and flowcharts, and randomly generating multiple circuit images.

[0037] Optionally, the training module is further used for:

[0038] Design a multimodal design specification understanding evaluation benchmark to evaluate the trained multimodal large language model.

[0039] Optionally, building modules are further used for:

[0040] Customized prompts are provided for circuit design, circuit verification, and circuit knowledge question-and-answer tasks. For any task, multiple task agents are constructed based on the prompts and a trained multimodal large language model.

[0041] Optionally, the output module is further used for:

[0042] The core interactive agent reads user input and determines the user's task, assigns the user's task to multiple task agents for execution, integrates the outputs of multiple task agents, and returns the integrated result to the user; among them, the user's tasks include circuit design, circuit verification, and circuit knowledge Q&A.

[0043] On the other hand, a Verilog generation and verification device is provided, the Verilog generation and verification device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the methods described above for Verilog generation and verification of multimodal circuit design specifications is implemented.

[0044] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the methods described above in the Verilog generation and verification methods for multimodal circuit design specifications.

[0045] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0046] This invention proposes a Verilog code generation and verification method based on a multimodal, multi-agent large language model. It fine-tunes the multimodal large language model by introducing large-scale open-source datasets and image datasets; designs prompt words to improve the multimodal large model's capabilities in circuit generation, circuit verification, and circuit knowledge question answering tasks, forming multiple agents; and designs a collaborative framework between the core interactive agent and the task agent to integrate task allocation and responses. This invention can generate and verify circuits for design specifications containing multiple modalities, solving the problems of low accuracy in circuit generation and low efficiency in verification of existing methods. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This invention provides a Verilog generation and verification method for multimodal circuit design specifications. Figure 1 ;

[0049] Figure 2 This invention provides a Verilog generation and verification method for multimodal circuit design specifications. Figure 2 ;

[0050] Figure 3 This is a block diagram of a Verilog generation and verification device for multimodal circuit design specifications provided in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the structure of a Verilog generation and verification device provided in an embodiment of the present invention. Detailed Implementation

[0052] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0053] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0054] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0055] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0056] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0057] This invention provides a Verilog generation and verification method for multimodal circuit design specifications. This method can be implemented using a Verilog generation and verification device, which can be a terminal or a server. Figure 1 , Figure 2 The flowchart shown is a Verilog generation and verification method for multimodal circuit design specifications. The processing flow of this method may include the following steps:

[0058] S1. Obtain the circuit code dataset, generate the first set of circuit images based on the circuit code dataset, use a large language model to perform natural language description on the circuit code dataset, obtain the natural language description results of the circuit code, and then construct a multimodal circuit dataset; wherein, the first set of circuit images includes circuit structure diagram, state transition diagram, timing diagram and flowchart.

[0059] Specifically, for each circuit code in the circuit code dataset, a script is used to generate a corresponding circuit image to obtain the first set of circuit images; a large language model is used to perform natural language description on the circuit code dataset to obtain the natural language description results of the circuit codes; and a multimodal circuit dataset is constructed based on the first set of circuit images and the corresponding natural language description results of the circuit codes.

[0060] In one feasible implementation, a large-scale circuit dataset is collected from an open-source website, and a script is used to generate circuit structure diagrams, state transition diagrams, etc., corresponding to the circuit code. A large language model is then used to describe the aforementioned dataset in natural language, ultimately forming a multimodal circuit dataset.

[0061] S2. Generate a second set of circuit images based on the Python code, use a large language model to perform natural language description on the Python code, obtain the natural language description results of the Python code, and then construct a multimodal general dataset.

[0062] Specifically, Python code is used to call relevant libraries for generating circuit structure diagrams, state transition diagrams, timing diagrams, and flowcharts. Multiple circuit images are randomly generated to construct a second set of circuit images. A large language model is used to perform natural language description on the Python code to obtain the natural language description results of the Python code. A multimodal general dataset is constructed based on the second set of circuit images and the corresponding natural language description results of the Python code. The second set of circuit images includes circuit structure diagrams, state transition diagrams, timing diagrams, and flowcharts.

[0063] In one feasible implementation, scripts are used to generate state transition diagrams, timing diagrams, flowcharts, circuit structure diagrams, etc., and large language models are used for natural language description and data cleaning, ultimately forming a multimodal general dataset.

[0064] Specifically, Python is used to call libraries for generating state transition diagrams, timing diagrams, flowcharts, and circuit structure diagrams, and multiple images are randomly generated. The Python code that generates these images is then converted into natural language descriptions using a general large language model, thereby obtaining multiple pairs of images and their natural language descriptions. These pairs can be used to train the model and improve its ability to interpret image content.

[0065] S3. Fine-tune the multimodal large language model based on the multimodal circuit dataset and the multimodal general dataset to obtain the trained multimodal large language model.

[0066] Optionally, S3 also includes:

[0067] The design of a multimodal design specification evaluation benchmark includes circuit design specifications, circuit-related images, reference Verilog code, and a Verilog test platform. Syntactic correctness and functional correctness are used to evaluate the trained multimodal large language model.

[0068] S4. Customize prompt words for different tasks, and construct multiple task agents based on the prompt words and the trained multimodal large language model.

[0069] In one feasible implementation, customized prompts are used to construct agents for various tasks such as circuit design, circuit verification, and circuit knowledge Q&A, with each task corresponding to an agent, thereby improving the model's ability to perform specific tasks.

[0070] S5. Design the core interactive agent, and complete circuit design, circuit verification, and circuit knowledge question-and-answer tasks based on the core interactive agent and multiple task agents.

[0071] Specifically, the core interactive agent reads user input and determines the user's task, assigns the user's task to multiple task agents for execution, integrates the outputs of the multiple task agents, and returns the integrated result to the user; among them, the user's tasks include circuit design, circuit verification, and circuit knowledge Q&A.

[0072] In one feasible implementation, a collaborative architecture between the core interactive agent and the task agent is designed to integrate task allocation and responses.

[0073] Specifically, the core interactive agent is responsible for interacting with the user, reading the user input and determining the user's needs (circuit design, circuit verification, or circuit knowledge Q&A), then assigning the user input to a specific agent for task execution, waiting for the task agent to respond, integrating the output of the task agent, and returning the final result to the user.

[0074] This invention proposes a Verilog code generation and verification method based on a multimodal, multi-agent large language model. It fine-tunes the multimodal large language model by introducing large-scale open-source datasets and image datasets; designs prompt words to improve the multimodal large model's capabilities in circuit generation, circuit verification, and circuit knowledge question answering tasks, forming multiple agents; and designs a collaborative framework between the core interactive agent and the task agent to integrate task allocation and responses. This invention can generate and verify circuits for design specifications containing multiple modalities, solving the problems of low circuit generation accuracy and low verification efficiency in existing methods.

[0075] Figure 3 This is a block diagram illustrating a Verilog generation and verification apparatus for a multimodal circuit design specification, according to an exemplary embodiment. The apparatus is used for a Verilog generation and verification method for a multimodal circuit design specification. (Refer to...) Figure 3 The device includes a multimodal circuit dataset construction module 310, a multimodal general dataset construction module 320, a training module 330, a construction module 340, and an output module 350. Wherein:

[0076] The multimodal circuit dataset construction module 310 is used to acquire a circuit code dataset, generate a first set of circuit images based on the circuit code dataset, use a large language model to perform natural language description on the circuit code dataset, obtain the natural language description results of the circuit code, and then construct a multimodal circuit dataset; wherein, the first set of circuit images includes circuit structure diagrams, state transition diagrams, timing diagrams and flowcharts.

[0077] The multimodal general dataset construction module 320 is used to generate a second set of circuit images based on Python code, use a large language model to perform natural language description on the Python code, obtain the natural language description results of the Python code, and then construct a multimodal general dataset.

[0078] Training module 330 is used to fine-tune the multimodal large language model based on the multimodal circuit dataset and the multimodal general dataset to obtain the trained multimodal large language model.

[0079] Module 340 is used to customize prompt words for different tasks and build multiple task agents based on the prompt words and the trained multimodal large language model.

[0080] The output module 350 is used to design the core interactive agent, and to complete circuit design, circuit verification and circuit knowledge question answering tasks based on the core interactive agent and multiple task agents.

[0081] This invention proposes a Verilog code generation and verification method based on a multimodal, multi-agent large language model. It fine-tunes the multimodal large language model by introducing large-scale open-source datasets and image datasets; designs prompt words to improve the multimodal large model's capabilities in circuit generation, circuit verification, and circuit knowledge question answering tasks, forming multiple agents; and designs a collaborative framework between the core interactive agent and the task agent to integrate task allocation and responses. This invention can generate and verify circuits for design specifications containing multiple modalities, solving the problems of low circuit generation accuracy and low verification efficiency in existing methods.

[0082] Figure 4 This is a schematic diagram of the structure of a Verilog generation and verification device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the Verilog generation and verification apparatus may include the above-mentioned Figure 3 The illustrated Verilog generation and verification apparatus is for multimodal circuit design specifications. Optionally, the Verilog generation and verification apparatus 410 may include a first processor 2001.

[0083] Optionally, the Verilog generation and verification apparatus 410 may also include a memory 2002 and a transceiver 2003.

[0084] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0085] The following is combined Figure 4 A detailed description of each component of the Verilog generation and verification apparatus 410 is provided below:

[0086] The first processor 2001, the control center of the Verilog generation and verification device 410, can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0087] Optionally, the first processor 2001 can perform various functions of the Verilog generation and verification device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0088] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0089] In a specific implementation, as one example, the Verilog generation and verification device 410 may also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0090] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0091] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be used in conjunction with the interface circuitry of the Verilog generation verification device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0092] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0093] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0094] Optionally, transceiver 2003 can be integrated with the first processor 2001 or exist independently, and the interface circuit of verification device 410 can be generated via Verilog. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0095] It should be noted that, Figure 4 The structure of the Verilog generation and verification device 410 shown in the diagram does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0096] Furthermore, the technical effect of the Verilog generation and verification device 410 can be referred to the technical effect of the Verilog generation and verification method for multimodal circuit design specifications described in the above method embodiments, and will not be repeated here.

[0097] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0098] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0099] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0100] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0101] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0102] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0105] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0106] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0108] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A Verilog generation and verification method for multimodal circuit design specifications, characterized in that, The method includes: S1. Obtain the circuit code dataset, generate the first set of circuit images based on the circuit code dataset, use a large language model to perform natural language description on the circuit code dataset, obtain the natural language description results of the circuit code, and then construct a multimodal circuit dataset; wherein, the first set of circuit images includes circuit structure diagram, state transition diagram, timing diagram and flowchart; S2. Generate a second set of circuit images based on the Python code, use a large language model to perform natural language description on the Python code, obtain the natural language description results of the Python code, and then construct a multimodal general dataset; S3. Fine-tune the multimodal large language model based on the multimodal circuit dataset and the multimodal general dataset to obtain the trained multimodal large language model; S4. Customize prompt words for different tasks, and construct multiple task agents based on the prompt words and the trained multimodal large language model; S5. Design a core interactive agent, and complete circuit design, circuit verification, and circuit knowledge question-and-answer tasks based on the core interactive agent and the multiple task agents.

2. The Verilog generation and verification method for multimodal circuit design specifications according to claim 1, characterized in that, In step S1, a first set of circuit images is generated based on the circuit code dataset. A large language model is then used to perform natural language description on the circuit code dataset to obtain the natural language description results of the circuit codes. This process then constructs a multimodal circuit dataset, including: For each circuit code in the circuit code dataset, a corresponding circuit image is generated using a script to obtain the first set of circuit images; a large language model is used to perform natural language description on the circuit code dataset to obtain the natural language description results of the circuit codes; a multimodal circuit dataset is constructed based on the first set of circuit images and the corresponding natural language description results of the circuit codes.

3. The Verilog generation and verification method for multimodal circuit design specifications according to claim 1, characterized in that, In step S2, a second set of circuit images is generated based on Python code. A large language model is used to perform natural language description on the Python code, resulting in a natural language description of the Python code. This leads to the construction of a multimodal general dataset, including: A second set of circuit images is generated based on the Python code. The Python code is then described using a large language model to obtain the natural language description results of the Python code. A multimodal general dataset is constructed based on the second set of circuit images and the corresponding natural language description results of the Python code. The second set of circuit images includes circuit structure diagrams, state transition diagrams, timing diagrams, and flowcharts.

4. The Verilog generation and verification method for multimodal circuit design specifications according to claim 1, characterized in that, The generation of the second set of circuit images based on Python code in S2 includes: The second set of circuit images is constructed by using Python code to call relevant libraries for generating circuit structure diagrams, state transition diagrams, timing diagrams, and flowcharts, and randomly generating multiple circuit images.

5. The Verilog generation and verification method for multimodal circuit design specifications according to claim 1, characterized in that, S3 further includes: Design a multimodal design specification understanding evaluation benchmark to evaluate the trained multimodal large language model.

6. The Verilog generation and verification method for multimodal circuit design specifications according to claim 1, characterized in that, The S4 section describes customized prompts for different tasks, and the construction of multiple task agents based on these prompts and a trained multimodal large language model, including: Customized prompts are provided for circuit design, circuit verification, and circuit knowledge question-and-answer tasks. For any task, multiple task agents are constructed based on the prompts and a trained multimodal large language model.

7. The Verilog generation and verification method for multimodal circuit design specifications according to claim 1, characterized in that, The step S5, which involves completing circuit design, circuit verification, and circuit knowledge question-and-answer tasks based on the core interactive agent and the multiple task agents, includes: The core interactive agent reads user input and determines the user's task, assigns the user's task to multiple task agents for execution, integrates the outputs of the multiple task agents, and returns the integrated result to the user; among them, the user's tasks include circuit design, circuit verification, and circuit knowledge Q&A.

8. A Verilog generation and verification apparatus for multimodal circuit design specifications, wherein the Verilog generation and verification apparatus for multimodal circuit design specifications is used to implement the Verilog generation and verification method for multimodal circuit design specifications as described in any one of claims 1-7, characterized in that, The device includes: A multimodal circuit dataset construction module is used to acquire a circuit code dataset, generate a first set of circuit images based on the circuit code dataset, use a large language model to perform natural language description on the circuit code dataset, obtain the natural language description results of the circuit code, and then construct a multimodal circuit dataset; wherein, the first set of circuit images includes circuit structure diagrams, state transition diagrams, timing diagrams and flowcharts; The multimodal general dataset construction module is used to generate a second set of circuit images based on Python code, use a large language model to perform natural language description on the Python code, obtain the natural language description results of the Python code, and then construct a multimodal general dataset. The training module is used to fine-tune the multimodal large language model based on the multimodal circuit dataset and the multimodal general dataset to obtain a trained multimodal large language model. The module is used to customize prompt words for different tasks and construct multiple task agents based on the prompt words and the trained multimodal large language model. The output module is used to design the core interactive agent and complete circuit design, circuit verification, and circuit knowledge question-and-answer tasks based on the core interactive agent and the multiple task agents.

9. A Verilog generation and verification device, characterized in that, The Verilog generation and verification device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images