Method for efficiently and accurately extracting parameters by using MLLM and RAG

Abstract

The invention relates to the field of AI, and provides a method for efficiently and accurately extracting parameters by using MLLM and RAG. The method comprises the steps of user demand input, initial configuration, process design, extraction script generation, operation extraction, model evaluation, output and the like, and aims at solving the problems of efficiency and accuracy in a traditional parameter extraction method. According to the main scheme, automatic parameter extraction is achieved through the MLLM and RAG technologies, decision consistency is improved through a multi-agent collaborative decision and self-alignment method, a model specific RAG is constructed to enhance domain knowledge of the MLLM, and complex extraction tools and visual information are processed through the visual analysis capacity of the MLLM. The method is suitable for the field of semiconductor device model parameter extraction, the parameter extraction efficiency and accuracy can be improved, the labor cost is reduced, and technical support is provided for research and application in related fields.

Description

Technical Field

[0001] The present invention relates to the field of AI and provides a method for efficient and accurate parameter extraction using MLLM and RAG. Background Art

[0002] With the rapid development of applications such as artificial intelligence and 5G communications, the demand for circuit performance is growing. To meet these demands, it is necessary to explore new semiconductor materials and devices and perform efficient circuit simulation and design. Device models are the basic bridge between semiconductor manufacturing and circuit simulation and design. They can simulate the electrical behavior of devices and provide important reference information for circuit design.

[0003] In order to achieve a high-precision device model, a large number of parameters need to be introduced to characterize the complex physical effects inside the device. However, the extraction process of these parameters faces many challenges:

[0004] Large number of parameters and complex optimization process: Device models usually contain hundreds or even thousands of parameters, which leads to large amount of calculation and low efficiency in the optimization process.

[0005] Reliance on manual intervention and difficult to automate: Traditional parameter extraction methods require experts to manually set weight functions, optimization areas, etc., which is difficult to automate and limits further development.

[0006] Lack of domain knowledge and difficulty in understanding the model: Existing machine learning models such as deep learning networks lack knowledge of the device modeling domain, making it difficult to understand the physical meaning of the model and the relationship between parameters, resulting in poor generalization and interpretability.

[0007] Limited interaction methods and difficulty in processing visual information: Existing model parameter extraction tools usually present results in the form of graphics or images, requiring experts to constantly interact with visual graphics and charts, which places high demands on the visual understanding ability of machine learning models.

[0008] In order to overcome the above challenges, researchers have explored a variety of methods, including intelligent optimization methods (such as genetic algorithms and Bayesian optimization), deep learning methods, etc. However, these methods still have limitations and are difficult to meet the needs of practical industrial applications.

[0009] In recent years, large language models (LLMs) have achieved success in tasks in various fields, such as natural language processing, information retrieval, and data analysis. LLM-based agents have demonstrated the ability to interact skillfully and efficiently with external environments through tools, which provides good inspiration for automatically generating semiconductor device models.

[0010] The powerful capabilities of LLM give it great potential in automatic device modeling. By combining the decision-making ability, knowledge acquisition ability, and visual understanding ability of LLM, an efficient and automated device model parameter extraction method can be developed to overcome the limitations of existing methods and meet the needs of practical industrial applications.

[0011] Disadvantages of existing technology

[0012] Although LLM has the potential for automatic device modeling, some challenges remain:

[0013] LLM lacks decision consistency: The parameter extraction process requires iterative adjustments across multiple steps, and LLM may lack decision consistency, resulting in unstable optimization results.

[0014] LLMs lack domain knowledge: LLMs lack knowledge of the device modeling domain, which hinders their ability to learn or discover rules that accelerate parameter extraction.

[0015] Limited visual understanding ability of LLM: The extraction criteria are usually presented in the form of graphs or images using complex extraction tools, which places high demands on the visual understanding ability of LLM. Summary of the invention

[0016] The technical problem involved in this invention is how to improve the efficiency and accuracy of parameter extraction in the field of semiconductor device model parameter extraction. Traditional parameter extraction methods have problems with efficiency and accuracy, including a large number of parameters, a complex optimization process, reliance on manual intervention and difficulty in automation, lack of domain knowledge and difficulty in understanding the model, and limited interaction methods and difficulty in processing visual information.

[0017] This invention proposes a method for efficient and accurate parameter extraction using MLLM and RAG, aiming to solve the problems in traditional methods. Automated parameter extraction is achieved through steps such as user demand input, initial configuration, process design, extraction script generation, run extraction, model evaluation and output. The application of MLLM and RAG technology can significantly improve efficiency and accuracy, and is suitable for the field of semiconductor device model parameter extraction, providing strong technical support for research and application in related fields.

[0018] A method for efficient and accurate parameter extraction using MLLM and RAG (retrieval-augmented generation), comprising the following steps:

[0019] Step 1: User input requirements: The user inputs task requirements, including the device model and parameters to be extracted;

[0020] Step 1.1: The user inputs the device model and parameters to be extracted.

[0021] Step 1.2: Retrieve relevant information from the knowledge base through RAG (retrieval enhanced generation) based on user input to understand the physical characteristics and parameter meanings of the device model, including model definition, parameter physical meaning, and extraction process, and use MLLMJudge to evaluate the user input to ensure its rationality and feasibility;

[0022] Step 2, initial configuration: retrieve device model parameters from the knowledge base, set initial values ​​and boundary values, check and adjust improper parameter settings;

[0023] Step 2.1, set the initial value and boundary value according to the relevant information retrieved from the knowledge base;

[0024] Step 2.2: Use the MLLM Judge module to evaluate the initial configuration and adjust improper parameter settings based on the evaluation results, such as parameter values ​​that exceed the boundary range or parameters that contradict each other;

[0025] Step 3, process design: Design the extraction process according to the configuration file, determine the relevant parameters that need to be extracted, and provide feedback and suggestions when the process is incomplete;

[0026] Step 3.1: Use the model-specific RAG to retrieve the extraction process information from the knowledge base, and design the extraction process according to user requirements and initial configuration, including extraction steps, parameter extraction order, etc.

[0027] Step 3.2: MLLM Judge can evaluate the completeness of the process and provide hints for missing steps. If the process is incomplete, the MLLM Judge module will provide feedback and suggestions, such as suggesting missing parameters or recommending appropriate extraction steps.

[0028] Step 4: Extract script generation:

[0029] Generate an extraction script based on the initial configuration and extraction process. The extraction script contains global parameter settings, specific parameter settings, and execution steps.

[0030] MLLM Judge evaluates the extraction script to ensure the correctness and feasibility of the script, such as checking for syntax errors or parameter setting errors in the script.

[0031] Step 5. Run extraction: Execute the extraction script to ensure that all steps are executed, complete the parameter extraction process, and obtain the model parameters.

[0032] Use the toolchain integration feature to interact with external simulators and databases, execute the extraction script, and perform parameter extraction according to the settings in the script. If an error occurs during the extraction process, the framework provides debugging options, such as displaying error information or providing modification suggestions.

[0033] Step 6, model evaluation: Use the extracted parameters to evaluate the model and check whether they meet the accuracy requirements. If they do, record the logs and store them in the database. If they do not, go to step 2 and continue execution after adjusting the inappropriate parameters according to the logs.

[0034] Step 6.1: Use the extracted parameters to evaluate the model and calculate indicators such as weighted RMSE.

[0035] Step 6.2: MLLM Judge can evaluate the model fitting effect and compare it with the results of human experts.

[0036] Step 6.3: If the model meets the accuracy requirements, the log is recorded and stored in the database.

[0037] Step 6.4: If the model does not meet the accuracy requirements, adjust the inappropriate parameters according to the log and return to step 2 to continue.

[0038] Step 7. Output: Output the extracted model parameters and display parameter tables and charts.

[0039] Furthermore, the parameter extraction process in step 5 is implemented as follows:

[0040] The parameter extraction process is automatically completed using the multimodal large language model (MLLM) and RAG (retrieval-augmented generation) technology, which includes the following steps:

[0041] Step a1, simulation operation: Use the initial parameter configuration to run the simulator and generate characteristic curve images of the device model. These images usually include the device's voltage-current (IV) curve, capacitance-voltage (CV) curve, etc., which are used to characterize the device's electrical characteristics.

[0042] Step a2, MLLM Judge visual evaluation: MLLM Judge uses its visual analysis capabilities to analyze the characteristic curve image, identify key areas in the image, and evaluate the model's fit. For example, MLLM Judge can identify the cutoff region, linear region, and saturation region of a MOSFET, and give a score based on how the curve fits in these regions.

[0043] Here are step a2 and its sub-steps:

[0044] Step a2: MLLM Judge visual evaluation and multi-agent collaborative decision-making:

[0045] Step a2.1, multi-agent system initialization:

[0046] Initialize multiple agents (q i ∈Q), each agent will be responsible for analyzing different parts of the feature curve image and giving a score.

[0047] Sub-step a2.2: Establishment of horizontal communication structure:

[0048] Establish a horizontal communication structure to ensure that each agent can actively share and refine its decisions i , promoting mutual understanding and collaboration among intelligent agents.

[0049] Sub-step a2.3: Visual analysis and establishment of scoring criteria:

[0050] Each agent uses its visual analysis capabilities to analyze the feature curve image, identify key areas in the image, and evaluate the model's fit.

[0051] A scoring table is established with six scoring categories (0-5) to provide scoring criteria and examples for MLLM.

[0052] Sub-step a2.4, implementation of self-alignment method:

[0053] At t=0, the agent scores only through the rating table, and stores the chart and the rating as a data pair in the example database.

[0054] At t>0, the agent retrieves historical examples from the database, compares them with the current graph, scores them in conjunction with the scoring table, and then stores the results again.

[0055] Sub-step a2.5: Collective opinion integration:

[0056] The integration function f(*) (here the mean function is selected) is used to merge the collective opinions of the agents into the group decision of the current round, expressed as S = f(s i ), i∈S.

[0057] Sub-step a2.6, decision result output:

[0058] Output the group decision result, which is the evaluation score of the characteristic curve fitting effect.

[0059] Step a3, model-specific RAG retrieval: Model-specific RAG searches the knowledge base based on the device model information input by the user to obtain relevant device model parameter information, including parameter definition, physical meaning, common range, model guide and engineering examples, etc. This information will be used to guide MLLM to generate scoring prompts.

[0060] Step a4, prompt generation:

[0061] The model-specific RAG integrates the retrieved information into prompts to generate prompts that guide the MLLM to generate scores. The prompts contain the following:

[0062] Parameter information: The prompt lists the information of all parameters in the device model entered by the user, including parameter definition, physical meaning, common range, etc.

[0063] Model Guide: The tips will reference the device model construction guide, such as model equations, parameter extraction steps, etc., to help MLLM understand the relationship between parameters and the extraction process.

[0064] Engineering Examples: Relevant engineering examples will be cited in the prompts, such as parameter settings and extraction steps for different device models, to provide references for MLLM.

[0065] Contextual information: The prompt will contain the device model information entered by the user, such as device type, operating conditions, etc., to provide contextual information for MLLM.

[0066] Step a5, MLLM generates scores:

[0067] MLLM uses the rich context of the prompt to generate a score to evaluate the fit of the characteristic curve. The score result of MLLM will be used as part of the optimizer's objective function to guide the optimizer to optimize in the direction that meets MLLM's preferences.

[0068] Step a6: Calculate the loss value L of the MLLM score agent :

[0069]

[0070] s i 、f s They are the scores given by MLLM and the corresponding full marks;

[0071] Step a7: Calculate the total objective function L using the following formula:

[0072] L=wL obj +(1-w)L agent

[0073] Among them, L obj is a loss function based on the error between the model simulation characteristic curve and the actual characteristic curve, and w is a weight factor used to balance L obj and L agent the relative importance of .

[0074] Step a8: Check whether the objective function meets the threshold:

[0075] Compare the total objective function L with the preset threshold. If L is greater than the threshold, it means that the model parameters need to be further optimized; if L is less than or equal to the threshold, it is considered that the model parameters are good enough and the results are output directly.

[0076] Step a9: Parameter optimization:

[0077] Use the Bayesian optimization algorithm to optimize the device model parameters according to the weighted objective function to minimize the error between the model simulation characteristic curve and the actual characteristic curve, run the simulator, generate a characteristic curve image of the device model, and jump to step a2 to continue execution.

[0078] In the above scheme, the workflow of model-specific RAG is as follows:

[0079] Step b1, create a vector knowledge base: collect technical documents and logs, divide the documents into small blocks, embed each text block, generate the corresponding vector representation, and store the generated vectors in the vector database for subsequent retrieval operations.

[0080] Step b2, create a knowledge graph: further divide the document into smaller fragments, extract key information from each fragment to form nodes, iteratively optimize the extracted nodes to ensure the accuracy and completeness of the knowledge, and connect the optimized nodes into a knowledge graph for the storage and retrieval of structured knowledge.

[0081] Step b3, Retrieval Enhancement Generation (RAG):

[0082] Step b3.1, query: input the query question and retrieve the documents and embedding vectors related to the query in the vector knowledge base;

[0083] Step b3.2, retrieval: When there is a query, retrieve the documents and embedding vectors related to the query in the vector knowledge base, convert them into high-dimensional vectors through embedding based on the word embedding model, and use the cosine similarity method to quickly retrieve the most relevant text blocks from the vector knowledge base;

[0084] Step b3.3, Enhancement: Combine the information obtained from the knowledge graph (KG) and the vector knowledge base to form an enhanced query request;

[0085] Step b3.4, Generate: Combine the enhanced query request and retrieval information to generate prompts to guide LLM to generate scores.

[0086] In the above scheme, the knowledge base of the model-specific RAG contains information related to a specific device model, including:

[0087] Model definition: definition of equations and parameters of device model;

[0088] Physical meaning of parameters: the physical meaning and function of each parameter;

[0089] Extraction process: parameter extraction steps and process;

[0090] Model Guide: Guide for building device models, including model equations and parameter extraction steps;

[0091] Project Examples: Related project examples, including parameter settings and extraction steps for different device models.

[0092] Because the present invention adopts the above technical means, it has the following beneficial effects:

[0093] 1. Improve parameter extraction efficiency and accuracy:

[0094] Automated parameter extraction is achieved through MLLM and RAG, including steps such as user demand analysis, initial configuration, process design, extraction script generation, running extraction, model evaluation and output, etc., which solves the problem that traditional parameter extraction methods rely on manual intervention, are inefficient and prone to errors, significantly improves parameter extraction efficiency and accuracy, reduces manual intervention, and reduces labor costs. Compared with traditional methods, this method has a higher degree of automation, higher efficiency, higher accuracy, and is easier to use.

[0095] 2. Solve the problem of MLLM decision inconsistency:

[0096] The multi-agent collaborative decision-making mechanism is adopted. Multiple agents analyze different parts of the characteristic curve image, score and make decisions, and finally integrate collective opinions to form a group decision. This solves the problem that MLLM may lack decision consistency in the parameter extraction process, resulting in unstable optimization results, improves the accuracy and reliability of decision-making, and avoids the bias and limitations of a single agent. Compared with single-agent decision-making, this method is more robust and the decision results are more reliable.

[0097] 3. Solve the problem of lack of domain knowledge in MLLM:

[0098] A model-specific RAG is constructed to retrieve relevant device model parameter information from the knowledge base, including parameter definition, physical meaning, common range, model guide, and engineering examples, and integrate it into prompts to provide rich contextual information for MLLM. This solves the problem that MLLM lacks knowledge in the field of device modeling and has difficulty understanding the physical meaning of the model and the relationship between parameters. It helps MLLM understand the relationship between parameters and the extraction process, and improves the accuracy and efficiency of parameter extraction. Compared with MLLM that lacks domain knowledge, this method is more able to understand the complexity of device modeling and thus extract parameters more effectively.

[0099] 4. Solve the problem of limited visual understanding ability of MLLM:

[0100] By using the visual analysis capabilities of MLLM Judge to analyze the characteristic curve image, identify the key areas in the image, and evaluate the model's fitting effect, the problem of MLLM's limited visual understanding ability and difficulty in handling complex extraction tools and visual information is solved, and MLLM's ability to understand the characteristic curve is improved, so that parameter extraction can be performed more effectively. Compared with MLLM, which lacks visual understanding capabilities, this method can analyze characteristic curves more accurately, thereby performing parameter extraction more effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] Figure 1 It is a schematic block diagram of the simplified process of the present invention;

[0102] Figure 2 Implement a flow chart for the parameter extraction process.

[0103] Figure 3 A schematic diagram of an implementation method of the process of the present invention;

[0104] Figure 4 A schematic diagram of an implementation method of the parameter extraction process of the present invention;

[0105] Figure 5 All model parameter extraction tasks and their corresponding types are listed in Table 1. Different difficulties are distinguished by background colors (blue is easy, yellow is medium, and red is difficult);

[0106] Figure 6 It is shown that the MLLM scores are highly correlated with human scores in mismatch assessment;

[0107] Figure 7 Compare the scores for MLLM;

[0108] Figure 8 This figure shows the weighted root mean square error (RMSE) of different methods in two different regions (MOS cut-off region and HEMT linear region) as the number of simulations changes. The figure contains four methods: Random, BO, MLLM-BO and Mixed-BO;

[0109] Fig. 9 An ablation study on MLLM Judge is presented. The table lists the performance of different methods on multiple indicators, including weighted root mean square error (wRMSE) and running time (Time);

[0110] Fig.10 Demonstration diagram of model-specific RAG structure and its effects;

[0111] Fig.11 Show the model fitting results;

[0112] Fig.12 This is a table showing the benchmark test results. DETAILED DESCRIPTION

[0113] The following is a detailed description of the embodiments of the present invention. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, modifications or equivalent substitutions made to the present invention should all be included in the scope of the claims of the present invention.

[0114] In addition, in order to better illustrate the present invention, numerous specific details are given in the following specific embodiments. It will be understood by those skilled in the art that the present invention can also be implemented without these specific details.

[0115] The present invention provides a method for efficient and accurate parameter extraction using MLLM and RAG (retrieval enhanced generation), comprising the following steps:

[0116] Step 1: User input requirements: The user inputs the task requirements, including the device model and parameters to be extracted;

[0117] Step 1.1: The user inputs the device model and parameters to be extracted.

[0118] Step 1.2: Retrieve relevant information from the knowledge base through RAG (retrieval enhanced generation) based on user input to understand the physical characteristics and parameter meanings of the device model, including model definition, parameter physical meaning, and extraction process, and use MLLMJudge to evaluate the user input to ensure its rationality and feasibility;

[0119] Step 2, initial configuration: retrieve device model parameters from the knowledge base, set initial values ​​and boundary values, check and adjust improper parameter settings;

[0120] Step 2.1, set the initial value and boundary value according to the relevant information retrieved from the knowledge base;

[0121] Step 2.2: Use the MLLM Judge module to evaluate the initial configuration and adjust improper parameter settings based on the evaluation results, such as parameter values ​​that exceed the boundary range or parameters that contradict each other;

[0122] Step 3, process design: Design the extraction process according to the configuration file, determine the relevant parameters that need to be extracted, and provide feedback and suggestions when the process is incomplete;

[0123] Step 3.1: Use the model-specific RAG to retrieve the extraction process information from the knowledge base, and design the extraction process according to user requirements and initial configuration, including extraction steps, parameter extraction order, etc.

[0124] Step 3.2: LLM Judge can evaluate the completeness of the process and provide hints for missing steps. If the process is incomplete, the MLLM Judge module will provide feedback and suggestions, such as suggesting missing parameters or recommending appropriate extraction steps.

[0125] Step 4: Extract script generation:

[0126] Generate an extraction script based on the initial configuration and extraction process. The extraction script contains global parameter settings, specific parameter settings, and execution steps.

[0127] MLLM Judge evaluates the extraction script to ensure the correctness and feasibility of the script, such as checking for syntax errors or parameter setting errors in the script.

[0128] Step 5. Run extraction: Execute the extraction script to ensure that all steps are executed, complete the parameter extraction process, and obtain the model parameters.

[0129] Use the toolchain integration feature to interact with external simulators and databases, execute the extraction script, and perform parameter extraction according to the settings in the script. If an error occurs during the extraction process, the framework provides debugging options, such as displaying error information or providing modification suggestions.

[0130] Step 6, model evaluation: Use the extracted parameters to evaluate the model and check whether they meet the accuracy requirements. If they do, record the logs and store them in the database. If they do not, go to step 2 and continue execution after adjusting the inappropriate parameters according to the logs.

[0131] Step 6.1: Use the extracted parameters to evaluate the model and calculate indicators such as weighted RMSE.

[0132] Step 6.2: MLLM Judge can evaluate the model fitting effect and compare it with the results of human experts.

[0133] Step 6.3: If the model meets the accuracy requirements, the log is recorded and stored in the database.

[0134] Step 6.4: If the model does not meet the accuracy requirements, adjust the inappropriate parameters according to the log and return to step 2 to continue.

[0135] Step 7. Output: Output the extracted model parameters and display parameter tables and charts.

[0136] Furthermore, the parameter extraction process in step 5 is implemented as follows:

[0137] The parameter extraction process is automatically completed using the multimodal large language model (MLLM) and RAG (retrieval-augmented generation) technology, which includes the following steps:

[0138] Step a1, simulation operation: Use the initial parameter configuration to run the simulator and generate characteristic curve images of the device model. These images usually include the device's voltage-current (IV) curve, capacitance-voltage (CV) curve, etc., which are used to characterize the device's electrical characteristics.

[0139] Step a2, MLLM Judge visual evaluation: MLLM Judge uses its visual analysis capabilities to analyze the characteristic curve image, identify key areas in the image, and evaluate the model's fit. For example, MLLM Judge can identify the cutoff region, linear region, and saturation region of a MOSFET, and give a score based on how the curve fits in these regions.

[0140] Here are the sub-steps of step a2:

[0141] Step a2: MLLM Judge visual evaluation and multi-agent collaborative decision-making:

[0142] Step a2.1, multi-agent system initialization:

[0143] Initialize multiple agents (q i ∈Q), each agent will be responsible for analyzing different parts of the feature curve image and giving a score.

[0144] Sub-step a2.2: Establishment of horizontal communication structure:

[0145] Establish a horizontal communication structure to ensure that each agent can actively share and refine its decisions i , promoting mutual understanding and collaboration among intelligent agents.

[0146] Sub-step a2.3: Visual analysis and establishment of scoring criteria:

[0147] Each agent uses its visual analysis capabilities to analyze the feature curve image, identify key areas in the image, and evaluate the model's fit.

[0148] A scoring table is established with six scoring categories (0-5) to provide scoring criteria and examples for MLLM.

[0149] Sub-step a2.4, implementation of self-alignment method:

[0150] At t=0, the agent scores only through the rating table, and stores the chart and the rating as a data pair in the example database.

[0151] At t>0, the agent retrieves historical examples from the database, compares them with the current graph, scores them in conjunction with the scoring table, and then stores the results again.

[0152] Sub-step a2.5: Collective opinion integration:

[0153] The integration function f(*) (here the mean function is selected) is used to merge the collective opinions of the agents into the group decision of the current round, expressed as S = f(s i ), i∈S.

[0154] Sub-step a2.6, decision result output:

[0155] Output the group decision result, which is the evaluation score of the characteristic curve fitting effect.

[0156] Through the above sub-steps, step a2 not only includes the visual evaluation of MLLM, but also integrates multi-agent collaborative decision-making and self-alignment methods to improve the accuracy and reliability of scoring. Such a design helps to avoid the bias and limitations of a single agent, and at the same time continuously improves the accuracy of scoring through iterative learning of historical data.

[0157] Step a3, model-specific RAG retrieval: Model-specific RAG searches the knowledge base based on the device model information input by the user to obtain relevant device model parameter information, including parameter definition, physical meaning, common range, model guide and engineering examples, etc. This information will be used to guide MLLM to generate scoring prompts.

[0158] Step a4, prompt generation:

[0159] The model-specific RAG integrates the retrieved information into prompts to generate prompts that guide the MLLM to generate scores. The prompts contain the following:

[0160] Parameter information: The prompt lists the information of all parameters in the device model entered by the user, including parameter definition, physical meaning, common range, etc.

[0161] Model Guide: The tips will reference the device model construction guide, such as model equations, parameter extraction steps, etc., to help MLLM understand the relationship between parameters and the extraction process.

[0162] Engineering Examples: Relevant engineering examples will be cited in the prompts, such as parameter settings and extraction steps for different device models, to provide references for MLLM.

[0163] Contextual information: The prompt will contain the device model information entered by the user, such as device type, operating conditions, etc., to provide contextual information for MLLM.

[0164] Step a5, MLLM generates scores:

[0165] MLLM uses the rich context of the prompt to generate a score to evaluate the fit of the characteristic curve. The score result of MLLM will be used as part of the optimizer's objective function to guide the optimizer to optimize in the direction that meets MLLM's preferences.

[0166] Step a6: Calculate the loss value L of the MLLM score agent :

[0167]

[0168] s i 、f s They are the scores given by MLLM and the corresponding full marks;

[0169] Step a7: Calculate the total objective function L using the following formula:

[0170] L=wL obj +(1-w)L agent

[0171] Among them, L obi is a loss function based on the error between the model simulation characteristic curve and the actual characteristic curve, and w is a weight factor used to balance L obj and L agent the relative importance of .

[0172] Step a8: Check whether the objective function meets the threshold:

[0173] Compare the total objective function L with the preset threshold. If L is greater than the threshold, it means that the model parameters need to be further optimized; if L is less than or equal to the threshold, it is considered that the model parameters are good enough and the results are output directly.

[0174] Step a9: Parameter optimization:

[0175] Use the Bayesian optimization algorithm to optimize the device model parameters according to the MLLM scoring results to minimize the error between the model simulation characteristic curve and the actual characteristic curve, run the simulator, generate a characteristic curve image of the device model, and jump to step a2 to continue execution.

[0176] In the above scheme, the workflow of model-specific RAG is as follows:

[0177] Step b1, create a vector knowledge base: collect technical documents and logs, divide the documents into small blocks, embed each text block, generate the corresponding vector representation, and store the generated vectors in the vector database for subsequent retrieval operations.

[0178] Step b2, create a knowledge graph: further divide the document into smaller fragments, extract key information from each fragment to form nodes, iteratively optimize the extracted nodes to ensure the accuracy and completeness of the knowledge, and connect the optimized nodes into a knowledge graph for the storage and retrieval of structured knowledge.

[0179] Step b3, Retrieval Enhancement Generation (RAG):

[0180] Step b3.1, query: input the query question and retrieve the documents and embedding vectors related to the query in the vector knowledge base;

[0181] Step b3.2, Retrieval: When there is a query, retrieve the documents and embedding vectors related to the query in the vector knowledge base, convert them into high-dimensional vectors through embedding based on text embedding-3 small model 3, and use the cosine similarity method to quickly retrieve the most relevant text blocks from the vector knowledge base;

[0182] Step b3.3, Enhancement: Combine the information obtained from the knowledge graph (KG) and the vector knowledge base to form an enhanced query request;

[0183] Step b3.4, Generate: Combine the enhanced query request and retrieval information to generate prompts to guide LLM to generate scores.

[0184] In the above scheme, the knowledge base of the model-specific RAG contains information related to a specific device model, including:

[0185] Model definition: definition of equations and parameters of device model;

[0186] Physical meaning of parameters: the physical meaning and function of each parameter;

[0187] Extraction process: parameter extraction steps and process;

[0188] Model Guide: Guide for building device models, including model equations and parameter extraction steps;

[0189] Project Examples: Related project examples, including parameter settings and extraction steps for different device models.

[0190] Example 1

[0191] The knowledge base contains rich device model knowledge and related field information, providing necessary support for parameter extraction. Among them, the model-specific RAG is responsible for retrieving relevant device model parameter information, including parameter definition, physical meaning, common range, etc., based on user input and information in the knowledge base. The following is a detailed process of how to use the model-specific RAG to complete this task:

[0192] 1. User input:

[0193] The user first inputs the device model that needs to be extracted, such as BSIMs or ASM-HEMT. After receiving the user input, it is passed to the model specific RAG.

[0194] 2. Knowledge base search:

[0195] Model-specific RAGs use the device model information entered by the user to search the knowledge base. The knowledge base contains various information related to the device model, including:

[0196] Parameter definition: basic information of each parameter such as name, unit, symbol, etc.

[0197] Physical meaning: The physical effect represented by each parameter in the device model, such as current, voltage, mobility, etc.

[0198] Common range: The common value range of each parameter, such as minimum value, maximum value, typical value, etc.

[0199] Model Guide: Guidelines for building device models, such as model equations, parameter extraction steps, etc.

[0200] Engineering Examples: Examples of common parameter configuration and extraction processes, such as parameter setting and extraction steps for different device models.

[0201] Simulation results: Simulation data and characteristic curves of the device model to verify the parameter extraction results.

[0202] 3. Information Integration:

[0203] Model-specific RAG integrates the retrieved information to build a complete device model knowledge graph (KG). KG connects various information into an organic whole through the relationship between entities, such as the dependency between parameters.

[0204] 4. Generate prompts:

[0205] The model-specific RAG uses the integrated information to generate prompts to guide the LLM to generate scores. The prompts include the following:

[0206] Parameter information: The prompt lists the information of all parameters in the device model entered by the user, including parameter definition, physical meaning, common range, etc.

[0207] Model Guide: The tips will reference the device model construction guide, such as model equations, parameter extraction steps, etc., to help LLM understand the relationship between parameters and the extraction process.

[0208] Project Examples: Relevant project examples will be cited in the prompts, such as parameter settings and extraction steps for different device models, to provide reference for LLM.

[0209] Contextual information: The prompt will contain the device model information entered by the user, such as device type, operating conditions, etc., to provide contextual information for LLM.

[0210] Example:

[0211] Assuming that the user input requires the extraction of a BSIMs model, the model-specific RAG retrieves relevant information about the BSIMs model from the knowledge base, including parameter definitions, physical meanings, common ranges, model guidelines, and engineering examples. The model-specific RAG then integrates this information into prompts to generate prompts that guide the LLM to generate scores, such as:

[0212] "The BSIMs model is used to simulate MOSFET devices and contains multiple parameters, such as Vth (threshold voltage), lambda (channel length modulation parameter), etc. Vth represents the turn-on voltage of the device, and lambda represents the intensity of the channel length modulation effect. The parameter extraction steps of the BSIMs model include setting the initial value of the parameter, running the simulation, optimizing the parameters, etc. Please evaluate the fitting effect of the BSIMs model based on the following engineering example and give a score: X (specific score)"

[0213] Through model-specific RAGs, relevant device model parameter information can be efficiently retrieved from the knowledge base and integrated into prompts to provide LLM with rich contextual information, thereby guiding LLM to generate accurate scores and ultimately achieve an efficient parameter extraction process.

[0214] Experimental data

[0215] The discussion is divided into two parts. The first part verifies the feasibility of MLLM Judge through experiments and evaluates its impact on behavior. The second part applies the proposed solution to the DC parameter extraction of the BSIMs model of MOSFET and the ASM-HEMT model of GaN HEMT.

[0216] Metrics: The main metric in the experiment is the weighted root mean square error RMSE (wRMSE), where w is defined as:

[0217]

[0218] “Region” is manually set according to the local optimization region. In addition, we use “Pass@k” (k=1, 5) to measure the ability of LLMs. It is defined as The proportion of extractions that achieve 90% accuracy in independent trials, the higher the value, the better. We conduct n trials (n ≥ k) and calculate where c is the number of successful trials. For visual scene experiments, we set n = 10. For text scene experiments, we set n = 30. ‘Number of solved tasks’ refers to the number of different tasks for which the device extraction was successfully achieved at least once in n trials.

[0219] Benchmark: We develop a comprehensive benchmark for model parameter extraction tasks, as detailed in Table 1. The data is obtained from some open source PDKs and through TCAD simulation. As the device size decreases, more parameters are required to simulate the secondary effects in the model, resulting in an increase in difficulty. We classify them into three categories based on the number of parameters: easy, medium, and hard.

[0220] MLLM Judge Evaluation

[0221] Evaluating the performance of the MLLM Judge component on parameter extraction performance. In order to evaluate the reliability of MLLM as an automatic evaluator, it is necessary to determine whether its evaluation is reasonable. A simple way is to compare the results of MLLM with those of professional engineers. If the results are highly correlated, it indicates validity; otherwise, it is invalid. Specifically, 15 different IV and CV regions (such as the cutoff region, linear region, and saturation region of MOSFET) were selected, and n = 20 different fitting scenarios were created by changing the parameters, and a total of 300 images were obtained. Then, three professional engineers and MLLM were asked to score them.

[0222] This process is repeated k = 50 times to obtain n data points for each evaluation type, denoted as A = a 1 , ..., a k and H = h 1 , ..., h k Among them, a i represents the average automatic score on the ,th random subset sample, and h i represents the average rating in the same subset. Figure 6 The results are plotted in . Clearly, the evaluation scores of MLLM are highly correlated with those of human experts. It is noteworthy that MLLM does not give too low a score when faced with poorly fitted images, and it rarely gives a perfect score. It is not difficult to guess that its training and alignment process leads to its conservative nature. Because MLLM gives high scores in the form of y=ax+b, b>0, we will remove from the score to distinguish between different grading ranges (similar to standardizing student grades).

[0223] In addition, we evaluate the performance of several recent open-source and closed-source multimodal large-scale models on the above images. Figure 7As shown. Green indicates that the model score is exactly the same as the engineer's result (out of 10 points), yellow indicates that the difference is less than 1 point, and red indicates that the difference is less than 3 points. This application improves the ability to understand domain images by adding multiple components to GPT-4o, and achieves good results in all tasks. Compared with the open source model, the closed source model achieves good results.

[0224] Finally, the effect of MLLM Judge is evaluated by optimizing 15 different types of IV and CV regions for MOSFET and HEMT, with 10 iterations for each region. Figure 8 It can be observed that MLLM-BO( In Equation 7), the performance is better than that of native BO. In addition, Mixed-BO ( ) Due to the more refined loss function, the performance is better than MLLM-BO, especially after multiple iterations. To demonstrate the auxiliary ability of MLLM Judge in local optimization, we conducted ablation experiments and the results are shown in Table 2. There are three baselines: globally optimized GP-BO, locally optimized GP-BO, and BO with human replacement of MLLM Judge. The results show that MLLM and humans have similar effects on the final results, and BO assisted by MLLM does reach or even exceed the optimization area of ​​manually set GP-BO (Local). This means that MLLM Judge is very beneficial for advancing tool automation.

[0225] Application in BSIMs and ASM-HEMT

[0226] The proposed method is applied to complete parameter extraction of BSIMs and ASM-HEMTs. First, we evaluate the impact of model-specific RAGs by simple question-answering. Fig.11 (a) shows the knowledge graph (KG) of ASM-HEMT, where complex connections link model parameters to relevant physical effects and extraction conditions. As can be seen from the zoomed-in area, the parameters (orange nodes) are connected to each other through a specific instance to form a group. When this instance is mentioned in parameter extraction, the group becomes our optimization object. At the same time, other lines indicate the correct extraction area, such as the low current area. Through RAG, the dependencies and impacts between different parameters can be effectively identified and analyzed, providing a clear path for LLM to accurately extract and optimize these parameters. Fig.11(b) shows the ablation implementation of the model-specific RAG. It can be observed that without RAG, LLM may produce a logically coherent narrative, but actually fails to reveal any task-related information. In contrast, with the help of RAG, LLM provides a clear process, including which parameters are extracted at each step and where, which facilitates decision making in the next stage. Next, we will use the proposed method to extract parameters for the BSIMs and ASM-HEMT models. The method successfully extracted the model parameters of BSIMs and ASM-HEMT, and the characteristic curves of the simulation output are shown in Figure 2. Fig.11 As shown, a good fitting effect is demonstrated. However, in the ASM-HEMT model, the simulation curves do not match the actual trends in several areas due to complex effects such as self-heating and traps. In addition, in order to explore the impact of the model on the effectiveness of parameter extraction, we selected the most advanced LLMs and compared them on the benchmarks in Table 1. To ensure a fair comparison and highlight the impact of our proposed strategy, all LLMs except GPT-4o are enhanced with MLLM Judge, model-specific RAG, and integrated toolchain. ModelGen can be regarded as an enhanced version of GPT-4o. As Fig.12 The results show that the latest open source models Llama3.1-70B and Mistral-Large2 show slightly stronger capabilities than GPT-4o-mini in model parameter extraction. ModelGen benefits from the enhancement of various strategies and achieves better performance than the original GPT-4o in more challenging tasks.

Claims

1. A method for efficient and accurate parameter extraction using MLLM and RAG, characterized in that: The following steps are involved: Step 1: User input requirements: The user inputs the task requirements, including the device model and parameters to be extracted; Step 2, initial configuration: retrieve device model parameters from the knowledge base, set initial values ​​and boundary values, check and adjust improper parameter settings; Step 3, process design: Design the extraction process according to the configuration file, determine the relevant parameters that need to be extracted, and provide feedback and suggestions when the process is incomplete; Step 4: Generate an extraction script: Generate an extraction script based on the initial configuration and extraction process. The extraction script contains global parameter settings and specific parameter settings, as well as execution steps. Step 5. Run extraction: Execute the extraction script to ensure that all steps are executed, complete the parameter extraction process, and obtain the model parameters. Use the toolchain integration function to interact with external simulators and databases, execute the extraction script, and perform parameter extraction according to the settings in the script. If an error occurs during the extraction process, provide debugging options, display error information or provide modification suggestions; Step 6, model evaluation: Use the extracted parameters to evaluate the model and check whether they meet the accuracy requirements. If they do, record the logs and store them in the database. If they do not, go to step 2 and continue execution after adjusting the inappropriate parameters according to the logs. Step 7. Output: Output the extracted model parameters and display parameter tables and charts.

2. A method for efficient and accurate parameter extraction using MLLM and RAG according to claim 1, characterized in that: Step 1 includes the following steps: Step 1.1: The user inputs the device model and parameters to be extracted. Step 1.2: Retrieve relevant information from the knowledge base through RAG based on user input to understand the physical characteristics and parameter meanings of the device model, including model definition, parameter physical meaning, and extraction process. Use MLLM Judge to evaluate the user input to ensure its rationality and feasibility, where RAG stands for retrieval enhanced generation.

3. A method for efficient and accurate parameter extraction using MLLM and RAG according to claim 1, characterized in that: Step 2 includes the following steps: Step 2.1, set the initial value and boundary value according to the relevant information retrieved from the knowledge base; Step 2.2: Use the MLLM Judge module to evaluate the initial configuration and adjust inappropriate parameter settings based on the evaluation results.

4. A method for efficient and accurate parameter extraction using MLLM and RAG according to claim 1, characterized in that: Step 3 includes the following steps: Step 3.1: Use the model-specific RAG to retrieve the extraction process information from the knowledge base, and design the extraction process according to user requirements and initial configuration, including extraction steps, parameter extraction order, etc. Step 3.2: LLM Judge can evaluate the completeness of the process and provide hints for missing steps. If the process is incomplete, the MLLM Judge module will provide feedback and suggestions, such as suggesting missing parameters or recommending appropriate extraction steps.

5. A method for efficient and accurate parameter extraction using MLLM and RAG according to claim 1, characterized in that: In step 4, the MLLM Judge module evaluates the extraction script to ensure the correctness and feasibility of the script.

6. A method for efficient and accurate parameter extraction using MLLM and RAG according to claim 1, characterized in that: Step 6 includes the following steps: Step 6.1: Use the extracted parameters to evaluate the model and calculate indicators such as weighted RMSE. Step 6.2: Evaluate the model fitting effect of the MLLM Judge module and compare it with the results of human experts; Step 6.3: If the model meets the accuracy requirements, the log is recorded and stored in the database; Step 6.4: If the model does not meet the accuracy requirements, adjust the inappropriate parameters according to the log and return to step 2 to continue.

7. A method for efficient and accurate parameter extraction using MLLM and RAG according to claim 1, characterized in that: The parameter extraction process in step 5 is implemented as follows: Using the multimodal large language model and RAG technology, the parameter extraction process is automatically completed, which includes the following steps: Step a1, simulation operation: using the initial parameter configuration, running the simulator, and generating a characteristic curve image of the device model; Step a2, MLLM Judge visual evaluation: MLLM Judge uses its visual analysis capabilities to analyze the characteristic curve image, identify the key areas in the image, and evaluate the fitting effect of the model; Step a3, model-specific RAG search: The model-specific RAG searches the knowledge base based on the device model information input by the user to obtain relevant device model parameter information, including parameter definition, physical meaning, common range, model guide and engineering examples; Step a4, prompt generation: The model-specific RAG integrates the retrieved information into prompts, generating prompts that guide the MLLM to generate scores; Step a5, MLLM generates scores: MLLM uses the rich context of the prompt to generate a score to evaluate the fit of the characteristic curve. The score result of MLLM will be used as part of the optimizer's objective function to guide the optimizer to optimize in the direction that meets MLLM's preferences. Step a6: Calculate the loss value L of the MLLM score agent : s i 、f s They are the scores given by MLLM and the corresponding full marks; Step a7: Calculate the total objective function L using the following formula: L=wL obj +(1-w)L agent Among them, L obj is a loss function based on the error between the model simulation characteristic curve and the actual characteristic curve, and w is a weight factor used to balance L obj and L agent the relative importance of . Step a8: Check whether the objective function meets the threshold: Compare the total objective function L with the preset threshold. If L is greater than the threshold, it means that the model parameters need to be further optimized; if L is less than or equal to the threshold, it is considered that the model parameters are good enough and the results are output directly. Step a9: Parameter optimization: Use the Bayesian optimization algorithm to optimize the device model parameters according to the MLLM scoring results to minimize the error between the model simulation characteristic curve and the actual characteristic curve, run the simulator, generate a characteristic curve image of the device model, and jump to step a2 to continue execution.

8. A method for efficient and accurate parameter extraction using MLLM and RAG according to claim 7, characterized in that: In step a4, the prompt includes the following: Parameter information: The prompt will list the information of all parameters in the device model entered by the user, including parameter definition, physical meaning, and common range; Model Guide: The prompt will reference the device model construction guide, including model equations and parameter extraction steps, to help MLLM understand the relationship between parameters and the extraction process; Engineering examples: Relevant engineering examples will be cited in the prompts, including parameter settings and extraction steps of different device models, to provide reference for MLLM; Contextual information: The prompt will contain the device model information entered by the user, such as device type, operating conditions, etc., to provide contextual information for MLLM.

9. A method for efficient and accurate parameter extraction using MLLM and RAG according to claim 1, characterized in that: The workflow for a model-specific RAG is as follows: Create a vector knowledge base: collect technical documents and logs, split the documents into small blocks, embed each text block, generate the corresponding vector representation, and store the generated vectors in a vector database for subsequent retrieval operations. Create a knowledge graph: further divide the document into smaller fragments, extract key information from each fragment to form nodes, iteratively optimize the extracted nodes to ensure the accuracy and completeness of the knowledge, and connect the optimized nodes into a knowledge graph for the storage and retrieval of structured knowledge. Retrieval Enhancement Generation (RAG): Query: Input a query question and retrieve documents and embedding vectors related to the query in the vector knowledge base; Retrieval: When there is a query, documents and embedding vectors related to the query are retrieved from the vector knowledge base, which are converted into high-dimensional vectors through embedding based on the Text Embedding-3 Small Model 3. The cosine similarity method is used to quickly retrieve the most relevant text blocks from the vector knowledge base; Enhancement: Combine the information obtained from the knowledge graph (KG) and the vector knowledge base to form an enhanced query request; Generate: Combine the enhanced query request and retrieved information to generate hints that guide LLM to generate scores.

10. A method for efficient and accurate parameter extraction using MLLM and RAG according to claim 9, characterized in that: The knowledge base of model-specific RAGs contains information related to a specific device model, including: Model definition: definition of equations and parameters of device model; Physical meaning of parameters: the physical meaning and function of each parameter; Extraction process: parameter extraction steps and process; Model Guide: Guide for building device models, including model equations and parameter extraction steps; Project Examples: Related project examples, including parameter settings and extraction steps for different device models.

Images