Computing device for physical simulator and computing system for physical simulation

Abstract

The present application relates to a computer system using a computing model, and discloses a computing device for a physical simulator and a computing system for a physical simulation. The computing device for a physical simulator comprises: a computing resource; an agent for calling a large language model to: receive initial simulation parameters and iteratively perform a simulation parameter generation process, comprising performing the following process for each time step: receiving current simulation parameters and current simulation results from the physical simulator; generating next simulation parameters based on the current simulation parameters and historical interaction information; providing the next simulation parameters to the physical simulator; outputting all simulation parameters and corresponding simulation results; the agent calculates a performance score based on the simulation results, and determines a reward function based on the performance score to train the large language model. The computing device according to the present application overcomes the problem that the existing agent is difficult to efficiently optimize continuous parameters under a limited simulation budget, and improves the ability of the large language model to optimize simulation parameters.

Description

Technical Field

[0001] This invention generally relates to computer systems utilizing computational models, and more specifically to computational devices for physical simulators and computational systems for physical simulation. Background Technology

[0002] As the capabilities of large language models in reasoning and decision-making improve, intelligent agents based on large language models are increasingly being used in tasks such as scientific discovery, engineering design, and industrial optimization.

[0003] Physical simulation is the process of building mathematical models based on simulation parameters to reproduce the mechanical, thermal, electromagnetic, and other behaviors of objects or systems in a virtual environment, thereby simulating and analyzing physical phenomena in the real world. In scenarios that rely on realistic physical simulation, various methods can be used to generate simulation parameters using intelligent agents, provide the generated simulation parameters to the physical simulator, and execute the simulation based on these parameters to generate simulation results.

[0004] One approach is based on a static agent using cue engineering. This method guides the model to generate parameters or simulation scripts by manually designing cue words. However, in this approach, the model itself lacks learning capabilities and cannot improve its strategy from interaction history. This makes it prone to generating a large amount of invalid computation in scenarios with high simulation call costs and high failure rates, making it difficult to obtain stable and feasible optimization results within a limited simulation budget.

[0005] Another approach is retrieval-enhanced or multi-agent methods. These methods leverage external knowledge bases or collaboration among multiple agents to mitigate the knowledge gap problem, but they cannot address the high failure rate and continuous parameter issues under limited simulation budgets.

[0006] Another approach is direct reinforcement learning. While effective in lightweight and stable environments, this method suffers from extremely low sample efficiency and difficulty in convergence in expensive and failure-prone simulation environments due to the high sensitivity of the simulation process to parameters.

[0007] Therefore, given the extremely high computational cost of each simulation call in physical simulation, existing methods suffer from drawbacks such as significant waste in simulation calls, a high failure rate, and difficulty in efficiently optimizing multi-objective metrics within a limited simulation budget.

[0008] There is a need in this field for improvements in physical simulation parameter generation techniques at at least one of the aforementioned levels. Summary of the Invention

[0009] This invention is provided to provide a technique for further improving the generation of physical simulation parameters using reinforcement learning.

[0010] One aspect of the present invention provides a computing device for a physics simulator, comprising: a computing resource configured for invocation by a large language model; an agent coupled to the physics simulator, the physics simulator being coupled to the computing device, the physics simulator being configured to receive simulation parameters and, based on the simulation parameters, perform a simulation associated with a physical process to generate a simulation result, the agent being configured to invoke the large language model to perform the following steps: S1. receiving initial simulation parameters associated with the physical process input by a user; S2. iteratively executing a simulation parameter generation process based on the initial simulation parameters, including performing the following process for each time step: S21. receiving current simulation parameters and a current simulation result associated with the current time step from the physics simulator, wherein the current simulation result is generated by the physics simulator based on the current simulation parameters, wherein the initial simulation parameters are the current simulation parameters associated with a first time step; and S22. Based on the current simulation parameters and historical interaction information, generate the next simulation parameters associated with the next time step, wherein the historical interaction information includes the current simulation parameters, the current simulation result, previous simulation parameters associated with all previous time steps, and corresponding previous simulation results; S23. Provide the next simulation parameters to the physical simulator; and S3. Output all simulation parameters and corresponding simulation results obtained by executing the simulation parameter generation process; wherein the agent trains the large language model using reinforcement learning based on the following steps: S4. Calculate a performance score based on multiple performance indicators and simulation convergence in the simulation results, wherein at least two of the multiple performance indicators are mutually constrained, and the performance score reflects the degree of deviation of each performance indicator from the corresponding target value; and S5. Determine a reward function based on the performance score, wherein the parameters of the large language model are adjusted based on the reward function.

[0011] In the computing device described above, in response to determining that the simulation convergence indicator has converged, S4 includes: S41. calculating the deviation of each of the plurality of performance indicators from its corresponding target value; S42. mapping the deviation to a satisfaction score, wherein the satisfaction score decreases as the deviation increases; and S43. calculating the performance score as a geometric aggregation of the satisfaction score.

[0012] In the computing device described above, in response to determining that the simulation convergence indicator has not converged, S4 includes: S44. Calculating the performance score as zero.

[0013] The computing device described above iteratively executes S2 for a predetermined time step.

[0014] In the computing device described above, the agent further trains the large language model using reinforcement learning based on the following steps: S6. Calculating a final-state performance improvement reward based on the difference between the final performance score corresponding to the final simulation result at the predetermined time step and the initial performance score corresponding to the initial simulation result at the first time step; S7. Calculating a process-optimal performance reward based on the difference between the highest performance score at all time steps and the initial performance score; and S8. Determining a round reward based on the final-state performance improvement reward and the process-optimal performance reward, wherein the parameters of the large language model are adjusted based on the round reward.

[0015] As described above, in the computing device, S2 further includes: S24. Calculating the distance between the next simulation parameter associated with the next time step and the historical simulation parameters in the historical parameter set, wherein the historical parameter set includes historical simulation parameters and corresponding historical simulation results; S25. When the historical simulation results corresponding to the nearest predetermined number of historical simulation parameters in the historical parameter set indicate non-convergence, regenerating the next simulation parameter associated with the next time step based on the current simulation parameters and the historical interaction information.

[0016] In the computing device described above, the agent is configured to invoke the large language model to: in a first process, iteratively execute a simulation parameter generation process up to a predetermined time step; and in a second process, further iteratively execute the simulation parameter generation process up to the predetermined time step, wherein the current simulation parameter associated with the first time step of the second process is the simulation parameter with the highest performance score in the first process.

[0017] The computing device described above, wherein the physical simulator is used to perform semiconductor device simulation, the simulation parameters including at least one of the following: gate length, source / drain extension length, source / drain length, monolithic nanosheet width, monolithic nanosheet thickness, nanosheet vertical spacing, gate oxide thickness, source / drain doping concentration, source / drain extension doping concentration, and substrate doping concentration.

[0018] Another aspect of the present invention provides a computing system for physical simulation, comprising: a computing device as described in any of the preceding claims; and the physical simulator coupled to the computing device.

[0019] Another aspect of the present invention provides a computing system for physical simulation, comprising: a computing resource; a physical simulator coupled to the computing resource, the physical simulator being configured to: receive simulation parameters; and perform a simulation associated with a physical process based on the simulation parameters to generate simulation results; an agent configured to invoke a large language model coupled to the computing resource to perform the following steps: S1. receiving initial simulation parameters associated with the physical process input by a user; S2. iteratively executing a simulation parameter generation process based on the initial simulation parameters, including performing the following process for each time step: S21. receiving current simulation parameters and a current simulation result associated with the current time step from the physical simulator, wherein the current simulation result is generated by the physical simulator based on the current simulation parameters, wherein the initial simulation parameters are current simulation parameters associated with a first time step; and S22. generating a next simulation parameter associated with the next time step based on the current simulation parameters and historical interaction information, the historical interaction information including the current simulation parameters, the current simulation result, previous simulation parameters associated with all previous time steps, and corresponding previous simulation results; S23. The next simulation parameters are provided to the physical simulator; and S3. All simulation parameters and corresponding simulation results obtained by executing the simulation parameter generation process are output; wherein the agent trains the large language model using reinforcement learning based on the following steps: S4. Calculate a performance score based on multiple performance indicators and simulation convergence in the simulation results, wherein at least two of the multiple performance indicators are mutually constrained, and the performance score reflects the degree of deviation of each performance indicator from the corresponding target value; and S5. Determine a reward function based on the performance score, wherein the parameters of the large language model are adjusted based on the reward function.

[0020] In the computational system described above, in response to determining that the simulation convergence indicator has converged, step S4 includes: S41. calculating the deviation of each of the plurality of performance indicators from its corresponding target value; S42. mapping the deviation to a satisfaction score, wherein the satisfaction score decreases as the deviation increases; and S43. calculating the performance score as a geometric aggregation of the satisfaction scores; in response to determining that the simulation convergence indicator has not converged, step S4 includes: S44. calculating the performance score as zero.

[0021] The computational system described above iteratively executes S2 up to a predetermined time step, wherein the agent further trains the large language model using reinforcement learning based on the following steps: S6. Calculate a final-state performance improvement reward based on the difference between the final performance score corresponding to the final simulation result at the predetermined time step and the initial performance score corresponding to the initial simulation result at the first time step; S7. Calculate a process-optimal performance reward based on the difference between the highest performance score at all time steps and the initial performance score; and S8. Determine a round reward based on the final-state performance improvement reward and the process-optimal performance reward, wherein the parameters of the large language model are adjusted based on the round reward.

[0022] In the computing system described above, S2 further includes: S24. Calculating the distance between the next simulation parameter associated with the next time step and the historical simulation parameters in the historical parameter set, wherein the historical parameter set includes historical simulation parameters and corresponding historical simulation results; S25. When the historical simulation results corresponding to the nearest predetermined number of historical simulation parameters in the historical parameter set indicate non-convergence, regenerating the next simulation parameter associated with the next time step based on the current simulation parameters and the historical interaction information.

[0023] The computing device and computing system of the present invention overcome the problem that existing intelligent agents have difficulty in efficiently optimizing the multi-objective performance of continuous parameters under limited simulation budget. By having the intelligent agent call a large language model to generate simulation parameters based on historical interaction information, and using multiple performance indicators fed back from the simulation results to perform reinforcement learning training on the large language model, the ability of the large language model to optimize simulation parameters under limited simulation budget is improved. Attached Figure Description

[0024] Various embodiments of the present invention are described in conjunction with the accompanying drawings.

[0025] Figure 1 This is a block diagram of a computing device for a physics simulator according to some embodiments of the present invention.

[0026] Figure 2 This is a schematic diagram illustrating the process of generating simulation parameters by an intelligent agent according to some embodiments of the present invention.

[0027] Figure 3 This is a schematic diagram of the training and inference process of a large language model for a computing device used in a physics simulator according to some embodiments of the present invention.

[0028] Figure 4 This is a block diagram of a first computing system for physical simulation according to some embodiments of the present invention.

[0029] Figure 5This is a block diagram of a second computing system for physical simulation according to some embodiments of the present invention. Detailed Implementation

[0030] In this application, the term "agent" refers to an agent capable of perceiving the environment and taking actions to perform specific goals. An agent primarily refers to software code. An agent can be executed by the system's computing resources. An agent can invoke corresponding models and tools through an Application Programming Interface (API) to interact with various forms of input or implement corresponding functions.

[0031] In this application, ordinal numbers such as "first," "second," and "third" are used to distinguish different instances of objects with the same name. The ordinal numbers "first," "second," and "third" do not indicate a relative order of the indicated objects in time, space, sequence, or other aspects.

[0032] According to one aspect of the present invention, a computing device for a physics simulator is provided.

[0033] Figure 1 This is a block diagram of a computing device 100 for a physics simulator according to some embodiments of the present invention.

[0034] The computing device 100 can be a local or remote computer, server, etc.

[0035] In some embodiments, computing device 100 may include computing resources 110, a large language model 120, and an agent 130. In some embodiments, the large language model 120 may also be external to computing device 100 (e.g., remotely to computing device 100).

[0036] In some embodiments, computing resources 110 may include a central processing unit (CPU), a graphics processing unit (GPU), and various other processing units or cores (e.g., arithmetic logic units, integer units, floating-point units, tensor units, ray tracing cores, etc.). Computing resources 110 may be configured for use by a large language model 120.

[0037] The physics simulator 140 may be coupled to the computing device 100. For example, the physics simulator 140 may be coupled to the intelligent agent 130 of the computing device 100. The physics simulator 140 may be invoked by the intelligent agent 130 as an external tool. The physics simulator 140 may be configured to receive simulation parameters and perform simulations associated with physical processes based on the simulation parameters to generate simulation results.

[0038] In some embodiments, the physical simulator 140 may include simulation software for performing a simulation process based on simulation parameters.

[0039] The intelligent agent 130 can be configured to invoke the large language model 120 to perform corresponding functions. The following will combine... Figure 2 Describe the specific functions of agent 130.

[0040] Figure 2 This is a schematic diagram illustrating the process of generating simulation parameters by an intelligent agent according to some embodiments of the present invention.

[0041] In some embodiments, the agent 130 may receive initial simulation parameters associated with a physical process input by a user. These initial simulation parameters can serve as a starting point for generating a simulation scheme. For example, the initial simulation parameters may be manually input by the user through a user interface on a computing device, selected by the user from historical simulation parameters, or obtained by the user modifying historical simulation parameters.

[0042] In some embodiments, agent 130 may iteratively execute a simulation parameter generation process based on initial simulation parameters. The simulation parameter generation process includes the following procedures executed for each time step.

[0043] First, agent 130 can receive current simulation parameters and current simulation results associated with the current time step from physical simulator 140. The current simulation results are generated by physical simulator 140 based on the current simulation parameters. The initial simulation parameters are the current simulation parameters associated with the first time step.

[0044] Then, agent 130 can generate the next simulation parameters associated with the next time step based on the current simulation parameters and historical interaction information. The historical interaction information may include the current simulation parameters, the current simulation results, the previous simulation parameters associated with all previous time steps, and the corresponding previous simulation results.

[0045] Finally, agent 130 can provide the next simulation parameters to physics simulator 140.

[0046] exist Figure 2 In the embodiment shown, agent 130 can receive current simulation parameters associated with the current time step t from physics simulator 140. And the current simulation results. The current simulation results are generated by the physics simulator 140 based on the current simulation parameters. The initial simulation parameters are generated as follows: When the time step is the first time step, the initial simulation parameters input by the user are the current simulation parameters associated with the first time step.

[0047] Then, the large language model 120 is modeled as a policy function to generate the next simulation parameter optimization scheme in a continuous parameter space. For the current time step t, the agent 130 calls the large language model 120 based on the current simulation parameters. Historical interaction information Generate the next simulation parameters associated with the next time step t+1. Historical interaction information Including current simulation parameters The current simulation results, the previous simulation parameters associated with all previous time steps 0 to t-1, and the corresponding previous simulation results. The input-output relationship of the large language model 120 can be expressed as: (1) in, Represents the parameters of a large language model The defined strategy function.

[0048] Finally, Figure 2 In the middle, the physics simulator 140 can be based on the next simulation parameters The next simulation result is generated and provided to agent 130 to continue the iterative simulation parameter generation process.

[0049] Some embodiments of the present invention generate the next simulation scheme based on historical interaction information, enabling the agent to continuously learn and optimize decision-making strategies from historical simulation feedback.

[0050] As an example of iteratively executing the simulation parameter generation process, for a first time step, agent 130 receives initial simulation parameters input by the user and initial simulation results generated by physical simulator 140 based on the initial simulation parameters. Based on the initial simulation parameters and historical interaction information (including the initial simulation parameters and initial simulation results), agent 130 generates second simulation parameters and then provides these second simulation parameters to physical simulator 140. Physical simulator 140 performs simulation based on the second simulation parameters to generate the second simulation results.

[0051] For the second time step, agent 130 receives second simulation parameters and a second simulation result generated by physical simulator 140 based on the second simulation parameters. Based on the second simulation parameters and historical interaction information (including initial simulation parameters and initial simulation results, second simulation parameters, and second simulation results), agent 130 generates third simulation parameters and then provides the third simulation parameters to physical simulator 140. Physical simulator 140 performs simulation based on the third simulation parameters to generate the third simulation result.

[0052] For the third time step, agent 130 receives third simulation parameters and a third simulation result generated by physical simulator 140 based on the third simulation parameters. Based on the third simulation parameters and historical interaction information (including initial simulation parameters and initial simulation result, second simulation parameters and second simulation result, and third simulation parameters and third simulation result), agent 130 generates fourth simulation parameters and then provides the fourth simulation parameters to physical simulator 140. Physical simulator 140 performs simulation based on the fourth simulation parameters to generate the fourth simulation result.

[0053] Although the above only shows the simulation parameter generation process for three time steps, the specific details of iteratively executing the simulation parameter generation process are not limited to this, and the iterative process can be executed until the simulation budget endpoint of the physical simulator.

[0054] In some embodiments, the agent 130 can output all simulation parameters and corresponding simulation results obtained from the simulation parameter generation process. For example, the agent 130 can provide the user with the simulation parameters and corresponding simulation results for all time steps, allowing the user to select a better simulation scheme for subsequent design based on different simulation results.

[0055] By generating simulation parameters with an intelligent agent, providing these parameters to a physical simulator for simulation, and then providing the simulation results back to the intelligent agent as the basis for generating the next set of simulation parameters, the intelligent agent can generate the next optimization scheme within a continuous parameter space.

[0056] In some embodiments, agent 130 may use reinforcement learning to train large language model 120 based on the following steps.

[0057] First, agent 130 can calculate a performance score based on multiple performance metrics from the simulation results and the simulation convergence. At least two of the multiple performance metrics are mutually constrained. The performance score reflects the degree of deviation of each performance metric from its corresponding target value.

[0058] Then, agent 130 can determine a reward function based on the performance score. The parameters of the large language model 120 can be adjusted based on the reward function.

[0059] Physical simulation tasks typically involve multiple mutually constrained performance metrics. An improvement in one metric may be accompanied by a decline in another, making it difficult to evaluate simulation results using a single metric. By mapping multidimensional performance metrics to a single scalar reward signal, the quality of different simulation results can be measured, promoting improvements in the simulation parameter generation process towards achieving comprehensive performance enhancements.

[0060] Some embodiments of the present invention address the problem that existing agents struggle to form effective simulation strategies in physical simulation environments prone to numerical non-convergence under limited simulation budgets. By enabling the agent to generate the next simulation parameters based on the current simulation parameters and historical interaction information, and by calculating performance scores based on multiple performance indicators to perform reinforcement learning training on a large language model, the agent can learn parameter optimization strategies with very few simulation attempts, thereby improving the simulation success rate.

[0061] Because the simulation process of a physics simulator consumes a large amount of computational resources and is prone to numerical non-convergence, simulation failures and low-performance simulation results lead to a large amount of invalid computation. Some embodiments of this invention improve the simulation success rate by using historical interaction information as the basis for generating simulation parameters and using performance scoring to evaluate the performance indicators of the simulation results. This improves the overall performance of the simulation results with the same or even fewer physics simulator calls, reduces invalid computational resource consumption, and saves hardware resources.

[0062] Some embodiments of this invention transform large language models from parameter generation tools that rely on static prompts into strategy models capable of continuous learning and optimization driven by feedback from real physical simulation results. This enables them to proactively learn the causal relationship between parameter changes and performance metrics based on simulation feedback. Furthermore, by uniformly mapping multiple performance metrics to a single scalar reward signal, the problems of difficulty in simultaneously addressing multiple objectives and the tendency for extreme optimization of a single metric are effectively solved.

[0063] In some embodiments, in response to determining that a simulation convergence indicator has converged, calculating a performance score may include calculating the deviation of each of a plurality of performance metrics from its corresponding target value. The deviation is then mapped to a satisfaction score, wherein the satisfaction score decreases as the deviation increases. Finally, the performance score is calculated as a geometric aggregation of the satisfaction scores.

[0064] In some embodiments, in response to determining that the simulation convergence indicator has not converged, calculating the performance score may include calculating the performance score as zero.

[0065] For example, when the physical simulator returns a current simulation result indicating convergence, let the set of performance metrics in the current simulation result be... , where K represents the number of performance metrics in the current simulation results. For each performance metric Pre-set the corresponding target value When a performance indicator deviates from the target value, the deviation is calculated. Deviation It can be represented as: (2) Then, the deviation of each performance metric is mapped to a satisfaction score. For example, it can take the form of a monotonically decreasing function: (3) Each performance metric has a corresponding mapping function. and This is an adjustable parameter used to control the penalty intensity for different performance indicators.

[0066] Finally, a geometric aggregation method is used to calculate the performance score, ensuring that degradation of any performance metric significantly impacts the overall evaluation of the simulation results. Performance Score It can be represented as: (4) If the current simulation results indicate convergence, the performance score will be calculated as a satisfaction score. The geometric aggregation result. If the current simulation result indicates non-convergence, that is, the result returned by the physical simulator is simulation failure, then the performance score is calculated as zero.

[0067] Therefore, by using the above method to calculate the performance score, if any performance index deviates significantly from the target value, or if the simulation process fails to converge numerically, the performance score will be significantly reduced, thus forming a strong constraint on infeasible parameter solutions during the reinforcement learning process of the large language model.

[0068] Although specific formulas have been given above to describe how to calculate the deviation between performance indicators and target values, the mapping relationship between deviation and satisfaction score, and how to calculate performance score, the above formulas are only an example of calculating performance score, and the embodiments of the present invention are not limited thereto.

[0069] Some embodiments of the present invention solve the problem of the difficulty in uniformly optimizing multiple indicators of simulation results by transforming the multi-dimensional performance indicators of simulation results into a single reward signal, enabling large language models to stably converge to the parameter solution with optimal overall performance under complex constraints.

[0070] In physical simulation, each simulation call is computationally expensive, and the agent's ability to read context information is limited. To maximize simulation performance within a limited simulation budget, embodiments of this invention also propose using a truncated field-of-view training strategy to limit the number of simulation calls.

[0071] In some embodiments, the agent 130 can iteratively execute the simulation parameter generation process up to a predetermined time step.

[0072] As an example, the maximum number of times agent 130 calls physics simulator 140 in each training round can be limited to a fixed value. In each training round, agent 130 executes the first time step up to the [missing information]. The process of generating simulation parameters for time steps.

[0073] Some embodiments of the present invention achieve efficient policy learning under strictly limited simulation budget conditions by employing a truncated vision training strategy. This enables large language models to learn effective local optimization directions with very few simulation calls, significantly improving sample efficiency and avoiding the problem that traditional reinforcement learning methods are difficult to train in expensive environments.

[0074] During the simulation parameter generation process, adjustments to the next simulation parameters generated by the large language model relative to the current simulation parameters in different directions and with different magnitudes can lead to varying degrees of optimization or degradation of the simulation results. The trend of the simulation results may be a continuous increase in performance, or a decrease followed by an increase, with the final simulation result showing the best performance; or it may be an increase followed by a decrease, with the intermediate simulation results showing the best performance. To balance the contribution of the final state performance improvement with the optimal performance during the exploration process, embodiments of this invention also propose a new round reward for reinforcement learning.

[0075] In some embodiments, agent 130 may also use reinforcement learning to train large language model 120 based on the following steps.

[0076] First, agent 130 can calculate the final performance improvement reward based on the difference between the final performance score corresponding to the final simulation result at the predetermined time step and the initial performance score corresponding to the initial simulation result at the first time step.

[0077] Then, agent 130 can calculate the performance-optimal reward based on the difference between the highest performance score across all time steps and the initial performance score.

[0078] Finally, agent 130 can determine the round reward based on the final state performance improvement reward and the process performance optimal reward. The parameters of the large language model 120 can be adjusted based on the round reward.

[0079] As an example, when the predetermined time step of the simulation parameter generation process is a fixed value. At that time, the round reward for reinforcement learning It can be represented as: (5) in, This represents the initial simulation parameters corresponding to the first time step. This represents the parameters of the last simulation corresponding to the predetermined time step. This represents the initial performance score corresponding to the initial simulation results at the first time step. This represents the final performance score corresponding to the final simulation result at the predetermined time step. This represents the highest performance score across all time steps. and These are the weighting coefficients.

[0080] Although specific formulas have been given above to describe how round rewards are calculated, these formulas are only an example of how to calculate round rewards, and the embodiments of the present invention are not limited thereto.

[0081] Some embodiments of the present invention determine round rewards based on final state performance improvement rewards and process performance optimization rewards, balancing the contribution of final state performance improvement and optimal performance during exploration, enabling reinforcement learning of large language models to improve the performance of simulation results in different aspects.

[0082] Due to the cost constraints caused by the extremely high computational cost of simulation calls, in order to reduce the waste of computational resources caused by simulation failures, embodiments of the present invention also propose a retrieval enhancement feasibility filtering mechanism used in the inference phase.

[0083] In some embodiments, the process of generating simulation parameters performed by the intelligent agent 130 may further include the following steps.

[0084] First, agent 130 can calculate the distance between the next simulation parameter associated with the next time step and the historical simulation parameters in the historical parameter set. The historical parameter set includes historical simulation parameters and corresponding historical simulation results.

[0085] Then, when the historical simulation results corresponding to the nearest predetermined number of historical simulation parameters in the historical parameter set do not converge, the agent 130 can regenerate the next simulation parameters associated with the next time step based on the current simulation parameters and historical interaction information.

[0086] As an example, historical parameter set It can be represented as: (6) in, Representing the One historical simulation parameter, Indicates the first The historical simulation results corresponding to each simulation parameter include multiple performance indicators when the simulation converges and when the simulation fails to converge.

[0087] In generating new simulation parameters Then, calculate its distance from the historical simulation parameters in the historical parameter set: (7) When a predetermined number of simulation parameters in the historical parameter set all correspond to simulation failures (i.e., the simulation does not converge), it can be determined that the new simulation parameter is located in a high-failure-risk region. Therefore, the simulation call for that parameter can be skipped, allowing the large language model to regenerate the simulation parameter, thereby reducing the waste of computational resources caused by simulation processes that may result in simulation failures.

[0088] Although specific formulas are given above to describe how to measure the distance between the current simulation parameters and historical simulation parameters, these formulas are only an example of distance calculation. The embodiments of the present invention are not limited to this, but other distance measurement methods or similar sample selection strategies can be used to adapt to different parameter space structures and failure modes.

[0089] Some embodiments of the present invention actively avoid parameter regions with high failure probability by skipping simulation calls for simulation parameters that may lead to simulation failure. This reduces the consumption of ineffective computing resources at the system level, and allows limited computing resources to be used to explore parameter schemes with higher success rates, thereby improving the performance of simulation results under limited computing resources.

[0090] In practical reasoning applications, the limited context window length of large language models leads to insufficient long-term optimization capabilities. Therefore, embodiments of this invention also propose a block-based iterative reasoning method.

[0091] In some embodiments, agent 130 may be configured to invoke large language model 120 to perform the following steps. In a first process, agent 130 may iteratively execute a simulation parameter generation process up to a predetermined time step. In a second process, agent 130 may further iteratively execute the simulation parameter generation process up to a predetermined time step. The current simulation parameter associated with the first time step of the second process may be the simulation parameter with the highest performance score in the first process.

[0092] As an example, the total iteration budget for physics simulator 140 is: The complete optimization process can be divided into multiple blocks (i.e., inference blocks) by iteration count (i.e., the simulator's budget execution count for each inference block). Each block executes independently within a fixed number of iterations, and the simulation parameter with the highest performance score in the current block is selected as the initial simulation parameter for the next block. It can be represented as: (8) As a result, the contextual information of the large language model is significantly reduced, and equivalent long-range parameter optimization can be achieved without increasing the contextual burden of a single inference.

[0093] As an example, when The 12 iterations can be divided into 4 blocks, each containing 3 iterations. In the first process, the simulation parameter generation process can be executed iteratively 3 times, and if the second simulation parameter has the highest performance score, it is used as the simulation parameter associated with the first time step in the second process. In the second process, the simulation parameter generation process can be executed iteratively 3 times, and if the first simulation parameter has the highest performance score, it is used as the simulation parameter associated with the first time step in the third process. In the third process, the simulation parameter generation process can be executed iteratively 3 times, and if the third simulation parameter has the highest performance score, it is used as the simulation parameter associated with the first time step in the fourth process.

[0094] Although specific values ​​for the number of iterations and the number of blocks have been given above, the embodiments of the present invention are not limited to these values ​​and can be adjusted according to specific task requirements.

[0095] Some embodiments of the present invention overcome the limitation of the context length of large language models by using a block-based iterative inference method, and achieve an effective connection between short-vision training and long-vision optimization without increasing the context burden of a single inference.

[0096] Figure 3 This is a schematic diagram of the training and inference process of a large language model for a computing device used in a physics simulator according to some embodiments of the present invention.

[0097] During the training and inference process of the large language model 120, the agent can invoke the large language model 120 based on the current simulation parameters and historical interaction information associated with the current time step. To generate the next simulation parameters associated with the next time step. .

[0098] In some embodiments, optionally, the agent may invoke the large language model 120 to execute a truncated vision training strategy, iteratively executing the simulation parameter generation process for a predetermined time step. .

[0099] During the training of the large language model 120, after receiving the current simulation parameters and current simulation results associated with the current time step, a performance score 304 can be calculated based on multiple performance indicators 302 in the simulation results, and a round reward 306 can be calculated based on the performance scores 304 of all iterations in a round. Finally, reinforcement learning is used to adjust the parameters of the large language model 120 based on the round reward 306.

[0100] In some embodiments, optionally, the agent may invoke the large language model 120 to perform retrieval enhancement feasibility filtering 308.

[0101] During training, the physical simulator is invoked at each time step (310). The agent can invoke the physical simulator to execute the physical simulator (312) to obtain multiple performance metrics (302) and provide them to the agent.

[0102] During the inference process, the physical simulator is invoked at each time step (310), and the agent (130) can access the historical parameter set. The agent retrieves historical simulation parameters and corresponding historical simulation results that are similar to the latest simulation parameters. When the historical simulation results for similar historical simulation parameters all indicate non-convergence, the agent can skip the execution of the physics simulator based on the latest simulation parameters (312) and regenerate the next simulation parameters. When the historical simulation results for at least one similar historical simulation parameter indicate convergence, the agent can execute the physics simulator based on the latest simulation parameters (312). The physics simulator obtains multiple performance indicators (302) and provides them to the agent.

[0103] In some embodiments, a physical simulator can be used to perform semiconductor device simulation. Semiconductor device simulation can be a Technology Computer-Aided Design (TCAD) semiconductor simulation. Simulation parameters may include at least one of the following: gate length, source / drain extension length, source / drain length, monolithic nanosheet width, monolithic nanosheet thickness, nanosheet vertical spacing, gate oxide thickness, source / drain doping concentration, source / drain extension doping concentration, and substrate doping concentration.

[0104] In semiconductor device simulation, performance metrics may include, for example, off-state current Ioff, on-state current Ion, and subthreshold swing SS. In one example, the performance metric in the first iteration is Ioff = 11.67. A / um, Ion=2.68 A / um, SS=60.94mV / dec, the performance index in the second iteration is Ioff=11.56 A / um, Ion=2.54 With A / um and SS=61.06mV / dec, compared to the first iteration, the second iteration shows a decrease in Ioff (performance improvement), a decrease in Ion (performance degradation), and an increase in SS (performance degradation). Therefore, these performance indicators are mutually constrained rather than increasing or decreasing in the same direction, which exacerbates the complexity of the simulation parameter optimization scheme.

[0105] As an example, the following is an example of an agent calling a large language model to generate simulation parameters in semiconductor device simulation: # Prompt You are a senior semiconductor device design engineer.

[0106] Your task is to optimize some device parameters of a 2nm GAAFET through iterative simulation and parameter tuning to meet the device performance requirements.

[0107] Device performance requirements

[0108] On-state current Ion > 2.31e-03 A / um, off-state current Ioff < 8.00e-09 A / um, SS < 61.00mV / dec

[0109] [List and range of adjustable parameters]

[0110] Lg: 0.024 (0.01, 0.028) # Gate length (µm)

[0111] Lext: 0.008, (0.004, 0.015) # Source / drain extension length (µm)

[0112] Lsd: 0.009, (0.006, 0.014) # Source-drain length (um)

[0113] NSwt: 0.029, (0.026, 0.032) # Width of a single nanosheet (µm)

[0114] NSt: 0.007, (0.002, 0.01) # Thickness of a single nanosheet (µm)

[0115] NSbuf: 0.008, (0.002, 0.008) # Vertical spacing of nanosheets (um)

[0116] Tox: 0.0011, (0.001, 0.003) # Gate oxide thickness (µm)

[0117] SD_Doping: 1.413e+21, (6e+19, 6e+21) # Source / drain doping concentration (cm⁻³)

[0118] SDE_Doping: 2.125e+20, (5e+18, 5e+20) # Source / drain extension doping concentration (cm⁻³)

[0119] Sub_Doping: 2.765e+17, (1e+16, 1e+18) # Substrate doping concentration (cm⁻³)

[0120] Heuristic parameter tuning rules

[0121] {rules}

[0122] Before each use of the tool, please clearly explain your analysis logic and the reasons for parameter adjustments, and describe your thought process using " <analyze> " and"< / analyze> The tag is complete.

[0123] # Agent Analysis and Tool Calling

[0124] ## Analysis (turn 1)

[0125] ```text

[0126] <analyze>

[0127] First, I need to perform a benchmark simulation using the given parameters to obtain initial Ion, Ioff, and SS values. Based on the results, I will evaluate which performance metrics are not meeting the requirements and make targeted adjustments according to the parameters.

[0128] Current parameter settings: - Lg: 0.024 um (close to the upper limit) - Tox: 0.0011 um (close to the lower limit, indicating good electrostatic control) - NSwt: 0.029 um, NSt: 0.007 um, NSbuf: 0.008 um - SD_Doping: 1.413e+21 cm⁻³ (Above average) - SDE_Doping: 2.125e+20 cm-3 (Medium) - Sub_Doping: 2.765e+17 cm⁻³ (Moderately low) Let me perform the first simulation to obtain baseline performance data.

[0129] < / analyze>

[0130] I will now use the current parameters to perform the first simulation and obtain initial performance data.

[0131] ```

[0132] ## Tool Calling (turn 1)

[0133] ```text

[0134] "tool_calls": [

[0135] {

[0136] "id": "functions.TCAD_simulate:0", "function": { "arguments": { "variables": { "Lg":0.024, "Lext":0.008, "Lsd":0.009, "NSwt":0.029, "NSt":0.007, "NSbuf":0.008, "Tox":0.0011, "SD_Doping": 1.413e+21, "SDE_Doping":2.125e+20, "Sub_Doping": 2.765e+17 } }, "name": "TCAD_simulate" }, "type": "function", "index": 0 } ] ``` ## Emulator Feedback (turn 1) ```text For the TCAD simulation: The simulation results are: (1) simulationreturn: simulation success (2) simulation metric: Ioff=1.20e-08 A / um, Ion=2.56e-03 A / um, SS=60.72 mV / dec" ``` ## Analysis (turn 2) ```text <analyze> Analysis of the first simulation results: - Ion = 2.56e-03 A / um (Meets requirements > 2.31e-03) - Ioff = 1.20e-08 A / um (Does not meet the requirements of < 8.00e-09, exceeds the limit by 1.5 times) - SS = 60.72 mV / dec (Meets requirement < 61.00) Adjustment plan (conservative adjustment): - Tox: 0.0011 → 0.0014 (an increase of 27%, but still within the range of 0.001-0.003) - Lg: 0.024 → 0.026 (increase by 8%) - Sub_Doping: 2.765e+17 → 5e+17 (an 80% increase, but still within the range of 1e+16-1e+18) < / analyze> The first simulation results showed that Ion and SS met the requirements, but Ioff slightly exceeded the limit.

[0137] ```

[0138] ## Tool Calling (turn 2)

[0139] ```text

[0140] "tool_calls": [

[0141] {

[0142] "id": "functions.TCAD_simulate:0", "function": { "arguments": { "variables": { "Lg":0.026, "Lext":0.008, "Lsd":0.009, "NSwt":0.029, "NSt":0.007, "NSbuf":0.008, "Tox":0.0014, "SD_Doping": 1.413e+21, "SDE_Doping":2.125e+20, "Sub_Doping":5.000e+17 } }, "name": "TCAD_simulate" }, "type": "function", "index": 0 } ] ``` ## Emulator Feedback (turn 2) ```text For the TCAD simulation: The simulation results are: (1) simulationreturn: simulation success (2) simulation metric: Ioff=5.07e-09 A / um, Ion=1.76e-03 A / um, SS=60.77 mV / dec ``` In the example above, in the first iteration, the large language model analyzes the first simulation result returned by the physics simulator, gives the next simulation parameters, and then provides the next simulation parameters to the physics simulator for the next physics simulation process.

[0143] In some embodiments, the physical simulator 140 can be used for simulation scenarios such as material design, drug molecule structure optimization, complex engineering structure design, and process parameter tuning.

[0144] According to another aspect of the present invention, a computing system for physical simulation is provided.

[0145] Figure 4 This is a block diagram of a first computing system 400 for physical simulation according to some embodiments of the present invention.

[0146] The first computing system 400 may include a computing device 100 and a physics simulator 140. The computing device 100 is coupled to the physics simulator 140. The computing device 100 may be a combination of... Figures 1 to 3 The details of the computing device 100 described herein will not be elaborated here to avoid redundancy.

[0147] According to another aspect of the present invention, a computing system for physical simulation is provided.

[0148] Figure 5 This is a block diagram of a second computing system 500 for physical simulation according to some embodiments of the present invention.

[0149] In some embodiments, the second computing system 500 may include computing resources 110, a large language model 120, an agent 130, and a physics simulator 140. In some embodiments, the large language model 120 may also be external to the second computing system 500 (e.g., remotely to the second computing system 500).

[0150] The physics simulator 140 may be coupled to the computing resource 110. The physics simulator 140 may be configured to receive simulation parameters and perform simulations associated with physical processes based on the simulation parameters to generate simulation results.

[0151] The agent 130 can be configured to invoke the large language model 120 to perform corresponding functions. The large language model 120 can be coupled to the computing resource 110.

[0152] Computing resources 110, large language model 120, intelligent agent 130, and physics simulator 140 can be combined Figures 1 to 3 The details of the computing resources 110, large language model 120, intelligent agent 130, and physical simulator 140 described are not elaborated here to avoid redundancy.

[0153] In some embodiments, the agent 130 may receive initial simulation parameters associated with a physical process input by a user; based on the initial simulation parameters, iteratively execute a simulation parameter generation process, including performing the following processes for each time step: receiving current simulation parameters and current simulation results associated with the current time step from a physical simulator, wherein the current simulation results are generated by the physical simulator based on the current simulation parameters, wherein the initial simulation parameters are the current simulation parameters associated with a first time step; and generating next simulation parameters associated with the next time step based on the current simulation parameters and historical interaction information, wherein the historical interaction information includes the current simulation parameters, the current simulation results, previous simulation parameters associated with all previous time steps, and corresponding previous simulation results; providing the next simulation parameters to the physical simulator; and outputting all simulation parameters and corresponding simulation results obtained by executing the simulation parameter generation process. The agent 130 may train the large language model 120 using reinforcement learning based on the following steps: calculating a performance score based on multiple performance metrics and simulation convergence in the simulation results, wherein at least two of the multiple performance metrics are mutually constrained, and the performance score reflects the degree of deviation of each performance metric from the corresponding target value; and determining a reward function based on the performance score, wherein the parameters of the large language model are adjusted based on the reward function.

[0154] In some embodiments, in response to determining that the simulation convergence indicator has converged, calculating the performance score may include: calculating the deviation of each of a plurality of performance metrics from its corresponding target value; mapping the deviation to a satisfaction score, wherein the satisfaction score decreases as the deviation increases; and calculating the performance score as a geometric aggregation of the satisfaction scores.

[0155] In some embodiments, in response to determining that the simulation convergence indicator has not converged, calculating the performance score may include: calculating the performance score as zero.

[0156] In some embodiments, agent 130 iteratively executes the simulation parameter generation process up to a predetermined time step, wherein agent 130 may also train large language model 120 using reinforcement learning based on the following steps: calculating a final-state performance improvement reward based on the difference between the final performance score corresponding to the final simulation result at the predetermined time step and the initial performance score corresponding to the initial simulation result at the first time step; calculating a process performance optimal reward based on the difference between the highest performance score and the initial performance score at all time steps; and determining a round reward based on the final-state performance improvement reward and the process performance optimal reward, wherein the parameters of the large language model are adjusted based on the round reward.

[0157] In some embodiments, the iterative execution of the simulation parameter generation process by the agent 130 may further include: calculating the distance between the next simulation parameter associated with the next time step and the historical simulation parameters in the historical parameter set, wherein the historical parameter set includes historical simulation parameters and corresponding historical simulation results; when the historical simulation results corresponding to the nearest predetermined number of historical simulation parameters in the historical parameter set indicate non-convergence, regenerating the next simulation parameter associated with the next time step based on the current simulation parameters and historical interaction information.

[0158] The intelligent agent-large language model framework according to embodiments of the present invention has been experimentally verified in a real simulation environment, demonstrating the feasibility and effectiveness of its technical solution. As an example, embodiments of the present invention were systematically experimentally verified in a semiconductor device simulation environment based on a real physical model. The physical simulator used was industrial-grade physical simulation software, used to simulate the electrical performance of nanoscale devices under different structural and process parameter conditions. Experimental results show that, under the same simulation budget, the large language model trained using embodiments of the present invention can significantly reduce the number of simulation failures and obtain high-quality parameter solutions that satisfy multiple performance index constraints with fewer simulation calls. Compared with existing methods that rely solely on prompting engineering, embodiments of the present invention show a significant advantage in the comprehensive optimization effect of multiple performance indicators, with more stable optimization results and insensitivity to initial parameter conditions. Further comparative experiments also show that simply increasing the model size does not significantly improve the optimization effect, while using the reinforcement learning training and inference framework of embodiments of the present invention, even with a limited model size, optimization results close to or even reaching industrial target requirements can be obtained. These experimental results fully verify the feasibility and practical value of embodiments of the present invention in real engineering simulation environments.

[0159] Table 1 presents experimental results for an agent-large language model framework based on some embodiments of the present invention. The results include scores and performance metrics for various methods, electrical metrics, multiple general models, and embodiments of the present invention.

[0160] Table 1: Experimental results of the agent-large language model framework;

[0161] The IRDS specification is the International Roadmap for Devices and Systems specification, providing a basis for electrical performance requirements. AgenticTCAD represents a multi-agent framework for generating semiconductor TCAD simulation code and optimizing devices. T represents the simulator's budgeted number of executions for each inference block, and K represents the number of inference blocks.

[0162] As can be seen from the table above, a higher score indicates better performance. The scores of the embodiments according to the present invention are significantly higher than those of other methods and general models. For the on-state current Ion, which is better when the value is larger, the off-state current Ioff, which is better when the value is smaller, and the subthreshold swing SS, which is better when the value is smaller, the performance indicators of the embodiments according to the present invention are also significantly better than those of other methods and general models.

[0163] Embodiments of the invention have been described with reference to the accompanying drawings. These embodiments are illustrative and not restrictive.

Claims

1. A computing device for a physics simulator, characterized in that, include: Computational resources are configured for use by the large language model; An intelligent agent, coupled to a physics simulator coupled to the computing device, the physics simulator being configured to receive simulation parameters and perform simulations associated with physical processes based on the simulation parameters to generate simulation results, the intelligent agent being configured to invoke a large language model to perform the following steps: S1. Receive initial simulation parameters associated with the physical process input by the user; S2. Based on the initial simulation parameters, iteratively execute the simulation parameter generation process, including performing the following process for each time step: S21. Receive from the physical simulator current simulation parameters and current simulation results associated with the current time step, wherein the current simulation results are generated by the physical simulator based on the current simulation parameters, and wherein the initial simulation parameters are the current simulation parameters associated with the first time step; and S22. Based on the current simulation parameters and historical interaction information, generate the next simulation parameters associated with the next time step, wherein the historical interaction information includes the current simulation parameters, the current simulation result, the previous simulation parameters associated with all previous time steps, and the corresponding previous simulation results; S23. Provide the next simulation parameters to the physical simulator; and S3. Output all simulation parameters and corresponding simulation results obtained by executing the simulation parameter generation process; The agent trains the large language model using reinforcement learning based on the following steps: S4. Based on multiple performance indicators and simulation convergence in the simulation results, calculate a performance score, wherein at least two of the multiple performance indicators are mutually constrained, and the performance score reflects the degree of deviation of each performance indicator from its corresponding target value; and S5. Determine a reward function based on the performance score, wherein the parameters of the large language model are adjusted based on the reward function.

2. The computing device as described in claim 1, characterized in that, In response to determining that the simulation convergence indicator has converged, S4 includes: S41. Calculate the deviation of each of the plurality of performance indicators from the corresponding target value; S42. Map the deviation amount to a satisfaction score, wherein the satisfaction score decreases as the deviation amount increases; and S43. Calculate the performance score as a geometric aggregation result of the satisfaction score.

3. The computing device as described in claim 1, characterized in that, In response to determining that the simulation convergence indicator has not converged, S4 includes: S44. Set the performance score to zero.

4. The computing device as claimed in claim 1, characterized in that, S2 is executed iteratively for a predetermined number of time steps.

5. The computing device as described in claim 4, characterized in that, The agent also trains the large language model using reinforcement learning based on the following steps: S6. Calculate the final performance improvement reward based on the difference between the final performance score corresponding to the final simulation result corresponding to the predetermined time step and the initial performance score corresponding to the initial simulation result corresponding to the first time step. S7. Calculate the process performance-optimal reward based on the difference between the highest performance score across all time steps and the initial performance score; as well as S8. Based on the final state performance improvement reward and the process performance optimization reward, determine the round reward, wherein the parameters of the large language model are adjusted based on the round reward.

6. The computing device as claimed in claim 1, characterized in that, S2 further includes: S24. Calculate the distance between the next simulation parameter associated with the next time step and the historical simulation parameters in the historical parameter set, wherein the historical parameter set includes historical simulation parameters and corresponding historical simulation results; S25. When the historical simulation results corresponding to the nearest predetermined number of historical simulation parameters in the historical parameter set do not converge, the next simulation parameters associated with the next time step are regenerated based on the current simulation parameters and the historical interaction information.

7. The computing device as claimed in claim 1, characterized in that, The agent is configured to invoke the large language model to: In the first process, the simulation parameter generation process is executed iteratively for a predetermined time step; as well as In the second process, the simulation parameter generation process is further iteratively executed up to the predetermined time step, wherein the current simulation parameter associated with the first time step of the second process is the simulation parameter with the highest performance score in the first process.

8. The computing device as claimed in claim 1, characterized in that, The physical simulator is used to perform semiconductor device simulation, and the simulation parameters include at least one of the following: gate length, source / drain extension length, source / drain length, monolithic nanosheet width, monolithic nanosheet thickness, nanosheet vertical spacing, gate oxide thickness, source / drain doping concentration, source / drain extension doping concentration, and substrate doping concentration.

9. A computational system for physical simulation, characterized in that, include: The computing device as described in any one of claims 1-8; as well as The physical simulator is coupled to the computing device.

10. A computational system for physical simulation, characterized in that, include: Computing resources; A physics simulator, coupled to the computing resources, is configured to: Receive simulation parameters; as well as Based on the simulation parameters, a simulation associated with the physical process is performed to generate simulation results; An intelligent agent is configured to invoke a large language model coupled to the computing resources to perform the following steps: S1. Receive initial simulation parameters associated with the physical process input by the user; S2. Based on the initial simulation parameters, iteratively execute the simulation parameter generation process, including performing the following process for each time step: S21. Receive from the physical simulator current simulation parameters and current simulation results associated with the current time step, wherein the current simulation results are generated by the physical simulator based on the current simulation parameters, and wherein the initial simulation parameters are the current simulation parameters associated with the first time step; and S22. Based on the current simulation parameters and historical interaction information, generate the next simulation parameters associated with the next time step, wherein the historical interaction information includes the current simulation parameters, the current simulation result, the previous simulation parameters associated with all previous time steps, and the corresponding previous simulation results; S23. Provide the next simulation parameters to the physical simulator; and S3. Output all simulation parameters and corresponding simulation results obtained by executing the simulation parameter generation process; The agent trains the large language model using reinforcement learning based on the following steps: S4. Based on multiple performance indicators and simulation convergence in the simulation results, calculate a performance score, wherein at least two of the multiple performance indicators are mutually constrained, and the performance score reflects the degree of deviation of each performance indicator from its corresponding target value; and S5. Determine a reward function based on the performance score, wherein the parameters of the large language model are adjusted based on the reward function.

11. The computing system as described in claim 10, characterized in that, In response to determining that the simulation convergence indicator has converged, S4 includes: S41. Calculate the deviation of each of the plurality of performance indicators from the corresponding target value; S42. Map the deviation amount to a satisfaction score, wherein the satisfaction score decreases as the deviation amount increases; and S43. Calculate the performance score as a geometric aggregation result of the satisfaction score; In response to determining that the simulation convergence indicator has not converged, S4 includes: S44. Set the performance score to zero.

12. The computing system as described in claim 10, characterized in that, The agent iteratively executes S2 for a predetermined number of time steps, wherein the agent further trains the large language model using reinforcement learning based on the following steps: S6. Calculate the final performance improvement reward based on the difference between the final performance score corresponding to the final simulation result corresponding to the predetermined time step and the initial performance score corresponding to the initial simulation result corresponding to the first time step. S7. Calculate the process performance-optimal reward based on the difference between the highest performance score across all time steps and the initial performance score; and S8. Based on the final state performance improvement reward and the process performance optimization reward, determine the round reward, wherein the parameters of the large language model are adjusted based on the round reward.

13. The computing system as described in claim 10, characterized in that, S2 further includes: S24. Calculate the distance between the next simulation parameter associated with the next time step and the historical simulation parameters in the historical parameter set, wherein the historical parameter set includes historical simulation parameters and corresponding historical simulation results; S25. When the historical simulation results corresponding to the nearest predetermined number of historical simulation parameters in the historical parameter set do not converge, the next simulation parameters associated with the next time step are regenerated based on the current simulation parameters and the historical interaction information.

Images