Circuit diagram solving method and device based on multi-modal model and electronic equipment

Abstract

The invention provides a circuit diagram solving method and device based on a multi-modal model and electronic equipment, and the method comprises the steps: inputting a to-be-solved circuit diagram into a pre-trained netlist recognition model, and outputting a netlist text; inputting the netlist text into a circuit simulator for circuit simulation to obtain a simulation result; obtaining a final answer based on the simulation result and the question text of the user; wherein the netlist identification model is a multi-modal model and is obtained by training based on a sample circuit diagram and a netlist label text, and due to the fact that the problem that an existing multi-modal large model cannot accurately understand circuit topology is effectively solved through the pre-trained netlist identification model; the problem that the mathematical reasoning calculation capability of an existing multi-mode large model is insufficient is solved through the circuit simulator, and therefore the method can analyze the circuit diagram more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of large models, and in particular to a circuit diagram solving method, device and electronic equipment based on a multimodal model. Background Art

[0002] In the field of artificial intelligence, the development of multimodal large models provides new possibilities for the automatic solution of various scientific problems. In particular, in the field of education, the construction of AI teaching assistant systems, especially AI teaching assistants for circuit-related courses, is of great significance for improving teaching efficiency and quality.

[0003] Existing technologies have obvious limitations when processing circuit images and related issues, mainly manifested in: 1) Although current multimodal large models, such as GPT4V and Claude3, have made significant progress in natural language processing, they perform poorly in analyzing and understanding circuit images. These models have difficulty accurately extracting key structural information such as components and connection relationships from circuit images.

[0004] 2) Existing large multimodal models have limited capabilities in automatically solving circuit problems, especially in processing text equations and performing numerical calculations, and it is difficult to directly provide accurate numerical answers.

[0005] With the development of electronics and computer science, the demand for circuit design and analysis is growing. The automatic understanding and analysis of circuit images, as well as the automatic solution of circuit problems, are of great value in improving circuit design efficiency, optimizing teaching resources, and promoting student learning. Therefore, developing an AI system that can effectively process circuit images and related problems has become an important research direction in this field. Summary of the invention

[0006] The present invention provides a circuit diagram solving method, device and electronic device based on a multimodal model, so as to solve the defects of the multimodal model in the prior art in the analysis and understanding of circuit images and the automatic solution of circuit problems.

[0007] The present invention provides a circuit diagram solving method based on a multimodal model, the method comprising: Input the circuit diagram to be solved into the pre-trained netlist recognition model and output the netlist text; Inputting the netlist text into a circuit simulator to perform circuit simulation and obtain simulation results; Based on the simulation results and the user's question text, a final answer is obtained; The netlist recognition model is a multimodal model, which is trained based on sample circuit diagrams and netlist label texts.

[0008] According to a circuit diagram solving method based on a multimodal model provided by the present invention, the circuit diagram to be solved is input into a pre-trained netlist recognition model, and a netlist text is output, which specifically includes: Input the user's question text and the generated netlist text into the preset large language model to obtain a specific question description, wherein the question description is used to guide the netlist recognition model to generate a netlist text that meets the requirements, ensuring that the component parameters and connection relationships in the netlist text are consistent with the requirements in the question text; The problem description and the circuit diagram to be solved are input into a pre-trained netlist recognition model, and the netlist text is output.

[0009] According to a circuit diagram solving method based on a multimodal model provided by the present invention, after outputting the netlist text, the method also includes: inputting the netlist text into a large language model for verification; inputting the netlist text into a circuit simulator for circuit simulation to obtain a simulation result, including: inputting the verified netlist text into a circuit simulator for circuit simulation to obtain a simulation result.

[0010] According to a circuit diagram solving method based on a multimodal model provided by the present invention, the simulation result includes circuit state quantities and measurement quantities; Based on the simulation results and the user's question text, a final answer is obtained, including: If the user's question text is to obtain any circuit state quantity and / or measurement quantity, the final answer is obtained directly according to the simulation results and the user's question text; If the user's question text is to solve the secondary parameters based on the simulation results, the user's question text, netlist text and simulation results are input into the large language model for further reasoning to obtain a final answer including the secondary parameters.

[0011] According to a circuit diagram solving method based on a multimodal model provided by the present invention, the training method of the netlist recognition model includes: inputting the sample circuit diagram into the netlist recognition model to obtain the netlist initial text; calculating the loss value of the netlist initial text and the netlist label text, and adjusting the parameters of the netlist recognition model according to the loss value until the loss value is less than a threshold.

[0012] According to a circuit diagram solving method based on a multimodal model provided by the present invention, before training the netlist recognition model, the method also includes: generating a grid including multiple grid points according to a random number seed, setting whether a component is connected and the attribute information of the component at each grid point; converting the grid into a description language for describing the circuit diagram, and compiling and generating a sample circuit diagram based on the description language; converting the grid into a netlist structure that describes the circuit topology structure, and converting the netlist structure into a netlist label text.

[0013] The present invention also provides a circuit diagram solving device based on a multimodal model, the device comprising: A netlist text output module is used to input the circuit diagram to be solved into a pre-trained netlist recognition model and output the netlist text; A simulation result generating module, used for inputting the netlist text into a circuit simulator to perform circuit simulation and obtain simulation results; A final answer output module, used to obtain a final answer based on the simulation result and the user's question text; The netlist recognition model is a multimodal model, which is trained based on sample circuit diagrams and netlist label texts.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the circuit diagram solving method based on the multimodal model as described in any one of the above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the circuit diagram solving method based on a multimodal model as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the circuit diagram solving method based on the multimodal model as described in any one of the above is implemented.

[0017] The circuit diagram solving method and device based on a multimodal model provided by the present invention input the circuit diagram to be solved into a pre-trained netlist recognition model to output a netlist text; input the netlist text into a circuit simulator to perform circuit simulation to obtain a simulation result; and obtain a final answer based on the simulation result and the user's question text; because the present invention effectively solves the problem that the existing multimodal large model cannot accurately understand the circuit topology through the pre-trained netlist recognition model, and alleviates the problem of insufficient mathematical reasoning and computing power of the existing multimodal large model through the circuit simulator, the method of the present invention makes the analysis of the circuit diagram more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is one of the flow charts of the circuit diagram solving method based on the multimodal model provided by the present invention.

[0020] Figure 2 This is the second flow chart of the circuit diagram solving method based on the multimodal model provided by the present invention.

[0021] Figure 3 It is a schematic diagram of the construction of the training data provided by the present invention.

[0022] Figure 4 It is a schematic diagram of the construction flow of the training data provided by the present invention.

[0023] Figure 5 It is a schematic diagram of a circuit diagram solving framework based on a multimodal model provided by the present invention.

[0024] Figure 6 This is the third flow chart of the circuit diagram solving method based on the multimodal model provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of a circuit diagram solving device based on a multimodal model provided by the present invention.

[0026] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In order to overcome the technical defects of the prior art that large multimodal models perform poorly in the analysis and understanding of circuit images and the automatic solution of circuit problems, the embodiments of the present invention introduce a pre-trained multimodal model to improve the accuracy of understanding the topological relationship in the circuit diagram, and alleviate the defect of the multimodal model's insufficient mathematical reasoning ability through a simulator, thereby improving the circuit diagram solving capability.

[0029] Combine the following Figure 1-Figure 7 A circuit diagram solving method and device based on a multimodal model provided by an embodiment of the present invention are described.

[0030] See also Figure 1 The embodiment of the present invention provides a circuit diagram solving method based on a multimodal model, which is used in a multimodal circuit analysis framework (ANPS framework) based on a netlist recognition model and a simulator. The method comprises: 101. Input the circuit diagram to be solved into the pre-trained netlist recognition model and output the netlist text.

[0031] Before inputting the circuit diagram to be solved, image preprocessing can be performed on the circuit diagram to be solved as needed, such as resizing, contrast enhancement, noise removal, etc., to improve the accuracy of the netlist recognition model.

[0032] In this embodiment, the netlist text may be a SPICE netlist text. The reason why the SPICE netlist text is selected to describe the netlist text is that, on the one hand, the netlist can describe the topological structure and component structure of the circuit, which can help the language model directly obtain the basic structural information of the circuit; on the other hand, if the value of the component is given, then the external simulator can be directly called to obtain the values ​​of all state quantities and measurement quantities of the circuit.

[0033] SPICE netlist text is a text description language used for electronic circuit simulation. It is mainly used to describe the topology of the circuit and the properties of the components. It is the core component of the SPICE (Simulation Program with Integrated Circuit Emphasis) simulation software. The following are some key features of SPICE netlist text: Describing circuit topology: A netlist defines the components in a circuit (such as resistors, capacitors, inductors, diodes, transistors, etc.) and how they are connected in text form.

[0034] Component properties: Each component in the netlist has a corresponding property description, including component type, parameter value (such as resistance value, capacitance value, inductance value, etc.).

[0035] Node definition: Each node (connection point) in the circuit has a unique identifier in the netlist, which is used to represent the connection relationship between components.

[0036] Simulation parameters: The netlist can also contain simulation parameters, such as simulation time, step size, temperature, etc. These parameters affect the accuracy and results of the simulation.

[0037] Measurement instructions: The netlist can contain measurement instructions to record and output parameters such as voltage and current of specific nodes during simulation.

[0038] Model definition: In addition to basic components, the netlist can also define more complex circuit models, such as semiconductor device models.

[0039] Take the following SPICE netlist text example: Circuit Description: V1 in 0 DC 5V

[0040] R1 in out 1k

[0041] C1 out 0 1uF

[0042] Simulation parameters: .TRAN 0 10ms

[0043] Measurement instructions: .PRINT TRAN V(in) V(out)

[0044] .END

[0045] In this example: V1 is a DC power source, which defines the name of the voltage source, the positive and negative nodes, and the voltage value.

[0046] R1 is a resistor that defines the name of the resistor, the nodes it connects to, and the resistance value.

[0047] C1 is a capacitor, which defines the name of the capacitor, the nodes it connects to, and the capacitance value.

[0048] The .TRAN directive defines the time range for a transient simulation.

[0049] The .PRINT directive defines the voltage values ​​that need to be recorded during the simulation.

[0050] The netlist recognition model is a multimodal model, which is trained based on sample circuit diagrams and netlist label text. The netlist recognition model needs to process image information (circuit diagram) and convert it into text information (netlist text), which involves the conversion from visual information to language information, which is in line with the characteristics of a multimodal model.

[0051] The training method of the netlist recognition model includes: inputting the sample circuit diagram into the netlist recognition model to obtain the netlist initial text; calculating the loss value of the netlist initial text and the netlist label text, and adjusting the parameters of the netlist recognition model according to the loss value until the loss value is less than a threshold.

[0052] Through step 101, the netlist recognition model can learn how to extract the topology and component properties of the circuit from the visual representation of the circuit diagram to be solved, and generate accurate SPICE netlist text to provide support for circuit simulation and analysis.

[0053] 102. Input the netlist text into a circuit simulator to perform circuit simulation and obtain simulation results.

[0054] The circuit simulator can be LTspice or HSPICE, which supports multiple circuit simulation types, including DC, AC, and transient analysis.

[0055] In a specific example, a netlist text is entered into a simulator, such as: V1 in 0 DC 5V

[0056] R1 in out 1k

[0057] C1 out 0 1uF

[0058] The simulator parses the netlist text, identifies the component types (such as V1 for DC power supply, R1 for resistor, C1 for capacitor), and their connection relationships and parameters. Then the simulator controls the simulation process, calculates the voltage value of each node at each time step, calculates the current flowing through each branch (component), and finally obtains the simulation results.

[0059] In this embodiment, the selection and configuration of the simulator should be based on the specific requirements and characteristics of the circuit. The setting of simulation parameters should ensure that the dynamic behavior of the circuit can be accurately captured. The parsing of the netlist text and the control of the simulation process are key steps to achieve accurate simulation. The calculation of state quantities and the analysis of simulation results lay the foundation for providing an accurate final answer.

[0060] 103. Obtain a final answer based on the simulation result and the user's question text.

[0061] Specifically, the simulation results include circuit state quantities and measurement quantities.

[0062] State quantities usually refer to the state of a circuit at a certain moment. They change over time and reflect the transient or steady-state behavior of the circuit. In analog circuit simulation, common state quantities include: node voltages, branch currents, capacitor voltages, inductor currents, measurements, etc.

[0063] Measurement quantities are usually used to evaluate circuit performance or solve specific circuit problems. Common measurement quantities include: voltage, current, power, frequency, impedance, gain, and phase difference.

[0064] Ideally, if the question is just to find a measurement quantity marked in the figure, then as long as the simulation result is obtained, the answer to the question can be obtained. Correspondingly, if the user's question text is to obtain any circuit state quantity and / or measurement quantity, the final answer can be obtained directly based on the simulation result and the user's question text.

[0065] If the user's question text is to solve secondary parameters based on the simulation results, such as power, size relationship, etc., or the netlist text cannot be directly simulated, the large language model is required to extract the simulation results and further reason to obtain the final answer. Specifically, the user's question text, netlist text and simulation results are input into the large language model for further reasoning to obtain the final answer including the secondary parameters.

[0066] The circuit diagram solving method based on a multimodal model provided in an embodiment of the present invention inputs the circuit diagram to be solved into a pre-trained netlist recognition model to output a netlist text; inputs the netlist text into a circuit simulator to perform circuit simulation to obtain a simulation result; and obtains a final answer based on the simulation result and the user's question text; because the present invention effectively solves the problem that the existing multimodal large model cannot accurately understand the circuit topology through the pre-trained netlist recognition model, and alleviates the problem of insufficient mathematical reasoning and computing power of the existing multimodal large model through the circuit simulator, the method of the present invention makes the analysis of the circuit diagram more accurate.

[0067] Optionally, in the process of generating the netlist text in this embodiment, a preset large language model is further applied. Figure 2 , step 101 specifically includes: 201. Input the user's question text and the generated netlist text into a preset large language model to obtain a specific question description.

[0068] The problem description is used to guide the netlist recognition model to generate a netlist text that meets the requirements, ensuring that the component parameters and connection relationships in the netlist text are consistent with the requirements in the problem text.

[0069] First, the large language model needs to understand the user's question, including the specific question on the circuit diagram and the type of answer required. This may involve parsing key terms, circuit components, measurement points, and required calculations in natural language. Then, key information is extracted from the user's question, such as the type of circuit component, connection method, measurement point, and required circuit parameters (such as voltage, current, etc.).

[0070] Based on the extracted key information, the large language model generates a clear problem description, which will guide the netlist recognition model (VLM) to generate a netlist text that meets the requirements.

[0071] 202. Input the problem description and the circuit diagram to be solved into a pre-trained netlist recognition model, and output the netlist text.

[0072] The circuit diagram is converted into a SPICE netlist text through the large language model LLM assisted by the netlist recognition model VLM, ensuring that the component parameters and connection relationships in the netlist are consistent with the requirements in the user's question.

[0073] Furthermore, the large language model can also verify the generated netlist text. After outputting the netlist text, the method further includes: inputting the netlist text into the large language model for verification; and then inputting the verified netlist text into a circuit simulator for circuit simulation to obtain a simulation result.

[0074] If a problem is found, the large language model LLM can guide the netlist recognition model VLM to make corrections.

[0075] Compared with the existing methods, the main contribution of the embodiment of the present invention is to train a new multimodal model (netlist recognition model) for circuit understanding and introduce a circuit simulator. In order to train the netlist recognition model, this embodiment needs to construct a data pair of <sample circuit diagram, netlist label text>. Since such data pairs are extremely lacking in the current field, this embodiment designs a set of automatic data construction processes, and uses the generated data to train the visual language model to obtain the netlist recognition model.

[0076] Specifically, see Figure 3 , Figure 3The diagram of the construction of the training data of this embodiment is shown. Among them, Seed represents a random number seed; Grid represents a grid matrix, which depicts the position of the grid points, whether each edge between the grid points exists, whether there are components, and the component type, value, label and other information; Netlist represents a netlist, which shows the actual connection mode of each component in the circuit diagram; LaTex is a circuit diagram drawing language, through the grid matrix, LaTex code can be directly and automatically generated, and the circuit image can be compiled by the LaTex compiler; SPICE is a circuit simulation language, which can depict the value and connection relationship of a circuit element, and can add measurement code, and then be directly executed by the simulator to obtain the circuit simulation result; VLM represents a visual language model (Visual Language Model), which is the basic model used in the training of this embodiment.

[0077] See also Figure 4 The training data construction process of the embodiment of the present invention includes: 401. Generate a grid including a plurality of grid points according to a random number seed, and set whether to connect an element and attribute information of the element at each of the grid points.

[0078] In this embodiment, a grid of size 𝑚× 𝑛 (representing 𝑚 rows and 𝑛 columns) and random distance is generated according to a random number seed, and then at each point on the grid, it is set to select whether to connect a component, the type, value, and label of the connected component, whether there is voltage measurement, current measurement, and other attribute information.

[0079] Measurement quantities refer to the voltage and current symbols marked on components or circuits in the figure, which usually means that the problem requires information about these circuit quantities, so these annotations are called measurement annotations because during simulation, the method of this embodiment needs to measure these circuit quantities.

[0080] 402. Convert the grid into a description language for describing a circuit diagram, and compile and generate a sample circuit diagram based on the description language.

[0081] In this embodiment, the grid is converted into a LaTex description language that can describe a circuit diagram. Given the components and the starting position and the ending position of the circuit diagram, the LaTex description language can realize the drawing of the circuit diagram through a specific circuit drawing macro package.

[0082] Then, the LaTex description language is compiled through the LaTex compiler to generate a pixel-level circuit diagram and converted into a sample circuit diagram in image format.

[0083] 403. Convert the grid into a netlist structure that describes a circuit topology, and convert the netlist structure into a netlist label text.

[0084] In this embodiment, the grid is converted into a netlist structure (Netlist) that describes the circuit topology. The netlist structure is a directed graph structure, where each node represents an equipotential point and each edge represents a component.

[0085] Then, the netlist structure is converted into the netlist label text of the simulation language SPICE. On the one hand, the SPICE language directly describes the netlist structure of the circuit; on the other hand, when the numerical value is given, the SPICE language can be directly used for simulation.

[0086] After the above steps 401 to 403, this embodiment constructs a data pair of <circuit diagram, SPICE description>. By repeating this process, an infinite number of random data pairs can be constructed and provided to VLM training. Finally, this embodiment uses the open source CogVLM-17B of Zhipu-AI as the base VLM for training.

[0087] In order to evaluate the capability of the netlist recognition model of this embodiment, this embodiment first evaluates the netlist generation accuracy of the netlist recognition model. This embodiment introduces a new indicator for measuring the consistency of the generated netlist and the label netlist - simulation consistency accuracy. That is, whether the simulation execution result of the generated netlist is consistent with the execution result of the label netlist. If they are consistent, in terms of probability, the generated netlist is equivalent to the label netlist with a high probability.

[0088] For the automatically constructed circuit diagram data, the accuracy of netlist conversion is shown in Table 1 (1k images): Table 1

[0089] It can be seen that the netlist recognition model achieves a conversion accuracy of more than 85% for both types of circuits. This embodiment also collects 20 circuit diagrams from real circuit data, and evaluates the migration capability and application capability of the netlist recognition model in real scenarios as shown in Table 2 below.

[0090] Table 2

[0091] It can be seen from Table 2 that the netlist recognition model also achieves more than 50% accuracy on the real circuit diagram.

[0092] Finally, this example selected 20 exercises with circuit diagrams in real scenarios and compared the framework with some existing methods.

[0093] The comparison method is as follows: GPT4V directly generates answers: Given a circuit image and a circuit text question, let GPT4V directly infer the answer. This method is abbreviated as GPT4V in this embodiment.

[0094] GPT4V + multimodal thought chain: This method is divided into two steps. The first step is to give a circuit diagram and a circuit text question. In this embodiment, GPT4V generates a description of the circuit by itself (thought chain

[21] ). The second step is to let GPT4V generate a solution based on the circuit diagram, text question, and self-generated text circuit description. The circuit text description can be in natural language. In this embodiment, this method is abbreviated as GPT4V+CoT-NL (Chain-of-Thought, Natural Languege); it can also be a netlist language. In this embodiment, this method is abbreviated as GPT4V+CoT-Net (Chain-of-Thought, Netlist).

[0095] GPT4 + Netlist Perception Model: This method combines the netlist recognition model trained in this embodiment, and passes the circuit netlist generated by the netlist recognition model to GPT4 together with the text question and the circuit image, so that GPT4 can generate the final answer. This method is equivalent to a comparison method of the ANPS framework of this embodiment minus the simulator-assisted reasoning part. This embodiment abbreviates this method as GPT4 + NPM (Netlist Perception Model).

[0096] GPT4 + Multimodal Circuit Analysis Framework (ANPS) Based on Netlist Recognition Model and Simulator Assistance: This is the method of this embodiment. The main idea is to use the netlist recognition model and LLM to generate a netlist that can be directly simulated, and then directly infer the final result through the netlist language, simulation results and text questions. This method is recorded as GPT4+ANPS (Ours), and Ours indicates that this is a new method proposed in this embodiment.

[0097] Table 3

[0098] The final evaluation results are shown in Table 3. It can be seen that in the 20 real circuit exercises selected in this embodiment, the calculation accuracy of the framework of this embodiment is significantly better than the existing multi-modal large model and the traditional multi-modal reasoning algorithm. This is mainly because the netlist recognition model of this embodiment effectively solves the problem that the existing multi-modal large model cannot accurately understand the circuit topology, and the simulator of this embodiment alleviates the problem of insufficient mathematical reasoning and calculation ability of the large model, thereby making the ANPS framework more accurate in analyzing the circuit.

[0099] To facilitate understanding of the solution framework of this embodiment and the circuit diagram solution method implemented based on the solution framework, see Figure 5 and Figure 6 .

[0100] The process of solving multimodal circuit problems is divided into two stages. The first stage is netlist generation (Netlist Generation), and the second stage is simulation-aided reasoning (Simulation Aided Reasoning).

[0101] The main modules include: 1. Netlist Perception Model (VLM in the figure), trained using a visual language model, is used to convert a circuit diagram into a SPICE netlist text.

[0102] 2. Circuit simulator (Simulator, in this work, NgSPICE which supports SPICE language simulation is used): used to perform circuit simulation and obtain simulation results.

[0103] 3. Large Language Model (LLM, GPT4 is used here): used to parse problem requirements, improve simulation language, extract simulation results, and calculate and reason about the final answer.

[0104] See also Figure 6 , the method of this embodiment includes: 601. Input the user's question text and the generated netlist text into a preset large language model to obtain a specific question description.

[0105] The problem description is used to guide the netlist recognition model to generate a netlist text that meets the requirements, ensuring that the component parameters and connection relationships in the netlist text are consistent with the requirements in the problem text.

[0106] 602. Input the problem description and the circuit diagram to be solved into a pre-trained netlist recognition model, and output the netlist text.

[0107] 603. Input the netlist text into a circuit simulator to perform circuit simulation and obtain simulation results.

[0108] 604. If the user's question text is to obtain any circuit state quantity and / or measurement quantity, a final answer is obtained directly according to the simulation result and the user's question text.

[0109] 605. If the user's question text is to solve the secondary parameters based on the simulation results, the user's question text, netlist text and simulation results are input into the large language model for further reasoning to obtain a final answer including the secondary parameters.

[0110] Through steps 601-605, this embodiment achieves an effective solution to the problem that the existing multimodal large model cannot accurately understand the circuit topology through a pre-trained netlist recognition model, and alleviates the problem of insufficient mathematical reasoning and computing capabilities of the existing multimodal large model through a circuit simulator, thereby making the method of the present invention more accurate in analyzing circuit diagrams.

[0111] The circuit diagram solving device based on the multimodal model provided by the present invention is described below. The circuit diagram solving device based on the multimodal model described below and the circuit diagram solving method based on the multimodal model described above can be referred to each other.

[0112] The embodiment of the present invention provides a circuit diagram solving device based on a multi-modal model, see Figure 7 ,include: The netlist text output module 701 is used to input the circuit diagram to be solved into the pre-trained netlist recognition model and output the netlist text; A simulation result generating module 702 is used to input the netlist text into a circuit simulator to perform circuit simulation and obtain simulation results; A final answer output module 703 is used to obtain a final answer based on the simulation result and the user's question text; The netlist recognition model is a multimodal model, which is trained based on sample circuit diagrams and netlist label texts.

[0113] Optionally, the netlist text output module 701 is specifically used for: Input the user's question text and the generated netlist text into the preset large language model to obtain a specific question description, wherein the question description is used to guide the netlist recognition model to generate a netlist text that meets the requirements, ensuring that the component parameters and connection relationships in the netlist text are consistent with the requirements in the question text; The problem description and the circuit diagram to be solved are input into a pre-trained netlist recognition model, and the netlist text is output.

[0114] Optionally, the apparatus further comprises: a verification module, configured to input the netlist text into a large language model for verification after outputting the netlist text; The simulation result generating module 702 is specifically used to: input the verified netlist text into the circuit simulator to perform circuit simulation and obtain the simulation result.

[0115] Optionally, the simulation result includes circuit state quantities and measurement quantities; the final answer output module 703 is specifically used to: If the user's question text is to obtain any circuit state quantity and / or measurement quantity, then the final answer is obtained directly according to the simulation results and the user's question text; If the user's question text is to solve the secondary parameters based on the simulation results, the user's question text, netlist text and simulation results are input into the large language model for further reasoning to obtain a final answer including the secondary parameters.

[0116] Optionally, the device of this embodiment further includes a training module, which is used to: Inputting the sample circuit diagram into a netlist recognition model to obtain an initial text of the netlist; The loss value of the netlist initial text and the netlist label text is calculated, and the parameters of the netlist recognition model are adjusted according to the loss value until the loss value is less than a threshold value.

[0117] Optionally, the device also includes: a training data construction module, which is used to generate a grid including multiple grid points according to a random number seed before training the netlist recognition model, and set whether a component is connected and the attribute information of the component at each grid point; convert the grid into a description language that describes the circuit diagram, and generate a sample circuit diagram based on the description language compilation; convert the grid into a netlist structure that describes the circuit topology structure, and convert the netlist structure into a netlist label text.

[0118] The circuit diagram solving device based on a multimodal model provided in an embodiment of the present invention inputs the circuit diagram to be solved into a pre-trained netlist recognition model to output a netlist text; inputs the netlist text into a circuit simulator to perform circuit simulation to obtain a simulation result; and obtains a final answer based on the simulation result and the user's question text; because the present invention effectively solves the problem that the existing multimodal large model cannot accurately understand the circuit topology through the pre-trained netlist recognition model, and alleviates the problem of insufficient mathematical reasoning and computing power of the existing multimodal large model through the circuit simulator, the method of the present invention makes the analysis of the circuit diagram more accurate.

[0119] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communications interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute a circuit diagram solving method based on a multimodal model, the method comprising: inputting the circuit diagram to be solved into a pre-trained netlist recognition model and outputting a netlist text; inputting the netlist text into a circuit simulator for circuit simulation to obtain a simulation result; obtaining a final answer based on the simulation result and the user's question text; wherein the netlist recognition model is a multimodal model, which is trained based on a sample circuit diagram and a netlist label text.

[0120] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0121] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the circuit diagram solving method based on a multimodal model provided by the above-mentioned methods, and the method includes: inputting the circuit diagram to be solved into a pre-trained netlist recognition model and outputting a netlist text; inputting the netlist text into a circuit simulator for circuit simulation to obtain a simulation result; obtaining a final answer based on the simulation result and the user's question text; wherein the netlist recognition model is a multimodal model, which is trained based on a sample circuit diagram and a netlist label text.

[0122] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the circuit diagram solving method based on a multimodal model provided by the above-mentioned methods, the method comprising: inputting the circuit diagram to be solved into a pre-trained netlist recognition model, and outputting a netlist text; inputting the netlist text into a circuit simulator for circuit simulation to obtain a simulation result; obtaining a final answer based on the simulation result and the user's question text; wherein the netlist recognition model is a multimodal model, which is trained based on a sample circuit diagram and a netlist label text.

[0123] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0124] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A circuit diagram solving method based on a multimodal model, characterized in that: The method comprises: Input the circuit diagram to be solved into the pre-trained netlist recognition model and output the netlist text; Inputting the netlist text into a circuit simulator to perform circuit simulation and obtain simulation results; Based on the simulation results and the user's question text, a final answer is obtained; The netlist recognition model is a multimodal model, which is trained based on sample circuit diagrams and netlist label texts.

2. The circuit diagram solving method based on a multimodal model according to claim 1, characterized in that: Input the circuit diagram to be solved into the pre-trained netlist recognition model and output the netlist text, including: Input the user's question text and the generated netlist text into the preset large language model to obtain a specific question description, wherein the question description is used to guide the netlist recognition model to generate a netlist text that meets the requirements, ensuring that the component parameters and connection relationships in the netlist text are consistent with the requirements in the question text; The problem description and the circuit diagram to be solved are input into a pre-trained netlist recognition model, and the netlist text is output.

3. The circuit diagram solving method based on a multimodal model according to claim 1 or 2, characterized in that: After outputting the netlist text, the method further includes: Inputting the netlist text into a large language model for verification; The netlist text is input into a circuit simulator to perform circuit simulation and obtain a simulation result, including: the verified netlist text is input into a circuit simulator to perform circuit simulation and obtain a simulation result.

4. The circuit diagram solving method based on a multimodal model according to claim 1, characterized in that: The simulation results include circuit state quantities and measurement quantities; Based on the simulation results and the user's question text, a final answer is obtained, including: If the user's question text is to obtain any circuit state quantity and / or measurement quantity, then the final answer is obtained directly according to the simulation results and the user's question text; If the user's question text is to solve the secondary parameters based on the simulation results, the user's question text, netlist text and simulation results are input into the large language model for further reasoning to obtain a final answer including the secondary parameters.

5. The circuit diagram solving method based on a multimodal model according to claim 1, characterized in that: The training method of the netlist recognition model includes: Inputting the sample circuit diagram into a netlist recognition model to obtain an initial text of the netlist; The loss value of the netlist initial text and the netlist label text is calculated, and the parameters of the netlist recognition model are adjusted according to the loss value until the loss value is less than a threshold value.

6. The circuit diagram solving method based on a multimodal model according to claim 1 or 5, characterized in that: Before training the netlist recognition model, the method further includes: Generate a grid including a plurality of grid points according to a random number seed, and set whether to connect an element and attribute information of the element at each of the grid points; Converting the grid into a description language for describing a circuit diagram, and compiling and generating a sample circuit diagram based on the description language; The grid is converted into a netlist structure describing a circuit topology, and the netlist structure is converted into a netlist label text.

7. A circuit diagram solving device based on a multimodal model, characterized in that: The device comprises: A netlist text output module is used to input the circuit diagram to be solved into a pre-trained netlist recognition model and output the netlist text; A simulation result generating module, used for inputting the netlist text into a circuit simulator to perform circuit simulation and obtain simulation results; A final answer output module, used to obtain a final answer based on the simulation result and the user's question text; The netlist recognition model is a multimodal model, which is trained based on sample circuit diagrams and netlist label texts.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the circuit diagram solving method based on a multimodal model as claimed in any one of claims 1 to 6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the circuit diagram solving method based on the multimodal model as described in any one of claims 1 to 6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the circuit diagram solving method based on a multimodal model as claimed in any one of claims 1 to 6 is implemented.

Images