A digital chip dynamic simulation verification method and system based on a large language model
By introducing a large language model into the dynamic simulation of digital chips for natural language determination, generating structured intermediate representations and performing real-time inference, the rigidity of traditional methods in out-of-order output and semantic dependency scenarios is solved, achieving efficient verification results and improved coverage.
Patent Information
- Application Number
- CN202611562388.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-10
- Publication Date
- 2026-11-06
AI Technical Summary
Traditional digital chip verification methods are rigid in handling out-of-order output, packet loss, and intermittent output, and have difficulty handling scenarios where the output form is correct but semantics depend on preconditions. Furthermore, the golden reference model and scoreboard have high maintenance costs, and existing large language model solutions lack real-time coupling and auditable decision-making.
During the dynamic simulation of digital chips, natural language judgment is performed through a large language model to generate structured intermediate representations. Preconditions and correctness constraints are extracted in combination with design specifications, and judgment conclusions are generated in real time through reasoning. A confidence fusion mechanism and dual-track verification are adopted to ensure the flexibility and reliability of verification.
It improves the flexibility and coverage of digital chip verification, mitigates the risk of misjudgment caused by model illusion, enables effective judgment of scenarios such as out-of-order and conditional output, and works in conjunction with traditional verification processes.
Smart Images
Figure CN123050299A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of digital integrated circuit design and verification technology, and in particular to a method and system for dynamic simulation verification of digital chips based on a large language model. Background Technology
[0002] In digital chip front-end verification, traditional methods primarily rely on a scoreboard mechanism: a reference model generates "golden" data, and the scoreboard then compares the output of the design under test (DUT) with the golden data item by item or transaction by transaction to determine whether the DUT's behavior is correct. This mechanism works well in scenarios with fixed output formats and regular timing, but it exhibits significant limitations in the following scenarios: First, it is relatively rigid in handling out-of-order output, packet loss, and intermittent output, requiring the writing of a large amount of customized comparison logic; second, it is difficult to express scenarios where "the output format is correct but semantically dependent on preconditions" (e.g., no output is allowed in a specific state machine state, or the result needs to be reinterpreted by field under a specific configuration) using fixed numerical comparisons; third, when design specifications are frequently iterated, the maintenance cost of the golden reference model and the scoreboard is high.
[0003] In recent years, large language models (LLMs) have demonstrated strong capabilities in natural language understanding and reasoning. Some solutions have attempted to use them for verification purposes, such as test scenario generation, SVA assertion generation, or failure log diagnosis.
[0004] However, existing solutions mostly treat large models as offline auxiliary tools, lack real-time coupling with the simulation timeline event loop, and do not provide an engineering closed loop to alleviate model illusion and ensure the auditability of judgments.
[0005] Therefore, there is an urgent need for a technical solution that can embed large model reasoning into the dynamic simulation process and work in conjunction with the traditional verification process to solve the above-mentioned technical defects. Summary of the Invention
[0006] The purpose of this invention is to provide a method and system for dynamic simulation verification of digital chips based on a large language model, so as to perform real-time, re-feedbackable and verifiable correctness judgment on the behavior of DUT by combining natural language judgment with large model reasoning during dynamic simulation.
[0007] To achieve the above objectives, this invention provides a dynamic simulation verification method for digital chips based on a large language model, comprising: In the dynamic simulation environment of the design under test (DUT) running in the electronic design automation simulator, key signals and transaction information are extracted according to the simulation time point to generate a structured intermediate representation; Extract the preconditions and correctness constraints according to the design specifications, and serialize them into natural language judgment expressions; The natural language decision expression is filled with the structured intermediate representation to obtain the text to be inferred; Within the transaction or phase window triggered by the simulation event, the Large Language Model (LLM) is invoked to reason about the text to be judged, so as to output a structured reasoning result containing the judgment conclusion, confidence level and basis fragments. The structured reasoning results are written back to the simulator as assertion events to participate in the result determination of coverage statistics and regression verification.
[0008] Preferably, the structured intermediate representation includes a set of key-value pairs extracted and serialized from the interface signals, internal state machine codes, and transaction-level monitoring information of the Design Under Test (DUT), rather than a direct transmission of the original simulation logs.
[0009] Preferably, the natural language judgment expression is derived from the design specifications using a rule template, and the rule template includes a precondition field, a set of allowed behaviors field, and a correctness constraint field.
[0010] Preferably, the step of calling the large language model to perform reasoning on the text to be reasoned includes: dividing the simulation time axis into reasoning windows according to transaction boundaries or stage boundaries to achieve real-time reasoning coupled with the event loop of the electronic design automation simulator.
[0011] Preferably, the confidence level is obtained by weighted fusion of four types of signals: semantic consistency confidence level, rule coverage confidence level, inference stability confidence level, and context integrity confidence level.
[0012] Preferably, the semantic consistency confidence is obtained by reading the softmax probability distribution of the large language model when outputting the decision word or the negative log-likelihood of the corresponding word token; The rule coverage confidence score is determined based on the ratio of the number of constraint rules explicitly referenced in the current natural language decision to the total number of constraint rules in the current simulation scenario. The inference stability confidence is obtained by repeatedly calling the large language model LLM N times with the same structured input within the same simulation event window and counting the consistency ratio of the output judgment results N times. The confidence level of context integrity is determined based on the sampling integrity of key signals within the current simulation event window. When a transaction is detected to cross the window boundary or a key signal is lost, the confidence level of context integrity is reduced accordingly.
[0013] Preferably, a dual-track review process is also included. When the judgment conclusion of the Large Language Model (LLM) is failure or the confidence level is between a preset low threshold and a preset high threshold, a review based on the Golden Reference Model (RM) or a formal checker is triggered. Only when the dual-track conclusions of the Large Language Model (LLM) and the Golden Reference Model (RM) or the formal checker are consistent, the corresponding event is counted as a confirmed defect. If the dual-track conclusions are inconsistent, the process is reverted to the traditional scoring board for comparison or suspended for manual confirmation.
[0014] Preferably, the method also includes a confidence fallback process: when the confidence level is higher than a preset high threshold, the judgment conclusion of the Large Language Model (LLM) is directly written back to the simulator as a simulation assertion event; when the confidence level is between a preset low threshold and a preset high threshold, the Golden Reference Model (RM) or a formal checker is triggered for dual-track verification; when the confidence level is lower than a preset low threshold or the judgment conclusion of the Large Language Model (LLM) is undeterminable, the method falls back to the traditional scoring board for comparison or is suspended for manual confirmation.
[0015] Preferably, the electronic design automation simulator includes an EDA simulator; the simulation time points include clock edges, transaction start and end boundaries, or coverage event trigger points; the deployment method of the large language model (LLM) includes localized or controlled private deployment, and the simulation data does not go abroad to the public cloud during the call process.
[0016] This invention also provides a dynamic simulation and verification system for digital chips based on a large language model, used to execute the dynamic simulation and verification method for digital chips based on a large language model as described above, including: The monitoring and acquisition module is used to extract key signals and transaction information of the design under test (DUT) according to the simulation time point; The structured serialization module is used to generate structured intermediate representations; The rule template storage module is used to store rule template modules. According to the design specifications, the preconditions and correctness constraints are extracted through the rule template modules and serialized into natural language judgment expressions. The Large Language Model Inference Agent Module is used to invoke the deployed Large Language Model (LLM) to perform inference on the text to be inferred within the transaction or stage window triggered by the simulation event, and output a structured inference result containing the judgment conclusion, confidence level and basis fragments. The result feedback and dual-track verification module is used to write the structured reasoning results back to the simulator as assertion-type events. The coverage and regression statistics module is used to summarize assertion-type events and participate in the result determination of coverage statistics and regression validation.
[0017] Compared with the prior art, the present invention has the following advantages: This invention introduces a natural language inference and judgment mechanism coupled with EDA simulator events, in addition to traditional scoreboard numerical comparison. During simulation, the monitoring module extracts key signals and transaction information of the Design Under Test (DUT) at simulation time points and generates a structured intermediate representation. The rule template module extracts preconditions and correctness constraints according to design specifications and serializes them into natural language judgment expressions. The inference agent calls a local or controlled deployed large language model within the transaction / stage window to infer whether the current DUT behavior satisfies the judgment expression, and outputs a structured result containing the judgment conclusion, confidence level, and basis fragments. The result write-back module writes the inference event back to the simulator to participate in assertion, coverage, and regression statistics. In other words, by cyclically coupling large model inference with simulation events and forming an engineering closed loop through mechanisms such as rule templates, result backfeeding, dual-track verification, and confidence level rollback, this method can improve verification flexibility and coverage in scenarios such as out-of-order processing, conditional suppression, and semantic dependency pre-configuration, while mitigating the risk of misjudgment caused by model illusion. It can also be mixed with traditional scoreboard scheduling and is compatible with existing UVM verification environments. Attached Figure Description
[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0019] Figure 1 A schematic diagram of the overall architecture of the digital chip dynamic simulation verification method based on a large language model provided in the embodiments of the present invention.
[0020] Figure 2 This is a flowchart illustrating the dynamic simulation verification method for digital chips based on a large language model, provided in an embodiment of the present invention.
[0021] Figure 3 This is a schematic diagram of the result backfilling and dual-track verification sub-process provided in an embodiment of the present invention. Detailed Implementation
[0022] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] like Figure 1As shown in the embodiment of the present invention, a dynamic simulation verification system for digital chips based on a large language model includes the following overall architecture: an EDA simulator and a design under test (DUT), a monitoring and acquisition module, a structured serialization module, a rule template storage module, a large language model inference agent module, a result feedback and dual-track verification module, and a coverage and regression statistics module. Each module is connected via a controlled internal communication link. The large language model is deployed locally or privately under controlled conditions, and simulation data is not exported to the public cloud. The monitoring and acquisition module is used to extract key signals and transaction information of the design under test (DUT) according to the simulation time point; The structured serialization module is used to generate structured intermediate representations; The rule template storage module is used to store rule template modules. According to the design specifications, the preconditions and correctness constraints are extracted through the rule template modules and serialized into natural language judgment expressions. The Large Language Model Inference Agent Module is used to invoke the deployed Large Language Model (LLM) to perform inference on the text to be inferred within the transaction or stage window triggered by the simulation event, and output a structured inference result containing the judgment conclusion, confidence level and basis fragments. The result feedback and dual-track verification module is used to write the structured reasoning results back to the simulator as assertion-type events. The coverage and regression statistics module is used to summarize assertion-type events and participate in the result determination of coverage statistics and regression validation.
[0024] like Figure 2 As shown in the figure, the present invention provides a dynamic simulation verification method for digital chips based on a large language model, which specifically includes the following steps: Step S201: In the dynamic simulation environment of the DUT running in an EDA simulator (such as VCS, Xcelium, Questa, GalaxSim, etc.), the monitoring and acquisition module extracts key signals and transaction information of the DUT according to simulation time points (such as clock edges, transaction start / end boundaries, or coverage event trigger points) to generate a structured intermediate representation. The structured intermediate representation is preferably a set of key-value pairs, such as {state:"IDLE", configuration:"MODE_A", payload:0x12, valid flag signal:1}, rather than the direct transmission of the original simulation log.
[0025] In step S202, the rule template module extracts the preconditions and correctness constraints according to the design specification (Spec) and serializes them into natural language judgment expressions. The rule template includes a precondition field, a set of allowed behaviors field, and a correctness constraint field. For example, a rule template derived from a Spec entry can be: "When the state is IDLE and the configuration is MODE_A, if valid is high, the DUT should output the result corresponding to the payload within the same transaction window, and should not generate unexpected error flags."
[0026] Step S203: Fill the natural language decision expression with structured intermediate representation to obtain the text to be inferred and divided into inference windows according to transaction boundaries or stage boundaries to achieve coupling with the simulator event loop.
[0027] In step S204, within the inference window, the Large Language Model Inference Agent module calls the controlled deployment of the Large Language Model (LLM) to infer the text to be inferred and outputs the structured inference results, including the judgment conclusion (pass / fail / unknown), confidence level, and supporting fragments.
[0028] In step S205, the result feedback and dual-track verification module writes the structured inference results back to the simulator as assertion events to participate in coverage statistics and regression verification result determination. When the result is determined to be a failure or the confidence level is between the preset low threshold and the preset high threshold, the Golden Reference Model (RM) or formal checker is triggered for verification. Only when the dual tracks are consistent is it counted as a confirmed defect. When the confidence level is insufficient, it reverts to the traditional scoreboard for comparison or suspends manual verification.
[0029] The confidence level is obtained by weighted fusion of four types of signals: semantic consistency confidence, rule coverage confidence, inference stability confidence, and context integrity confidence. For ease of understanding, a specific example of confidence level calculation is given below. Suppose that within a certain simulation event window, in the softmax probability distribution of the Large Language Model (LLM) output for the decision word, "pass" corresponds to a probability of 0.91, thus obtaining a semantic consistency confidence level C1 = 0.91. In the current scenario, eight constraint rules are defined in the design specifications, and the current natural language decision explicitly references six of them (including the crucial "out-of-order allowed" rule), resulting in a rule coverage confidence level C2 = 6 / 8 = 0.75. Within the same window, with the same structured input and temperature > 0 (using a non-zero temperature sampling configuration), the LLM model is called N = 5 times, and the result is "pass" in all 5 calls, resulting in an inference stability confidence level C3 = 5 / 5 = 1.0. Since the key signals in the current window are fully sampled and the transaction does not cross the window boundary, the context integrity confidence level C4 = 1.0. Taking calibrable weighting coefficients w1=0.35, w2=0.25, w3=0.25, and w4=0.15, the confidence level after weighted fusion is: C=w1·C1+w2·C2+w3·C3+w4·C4 =0.35×0.91+0.25×0.75+0.25×1.0+0.15×1.0≈0.894; If C is higher than a preset high threshold (e.g., 0.85), the model "pass" decision is directly written back to the simulator as an assertion event. If C is between a preset low threshold (e.g., 0.6) and a preset high threshold, the Golden Reference Model (RM) dual-track verification is triggered. If C is lower than a preset low threshold or the LLM model outputs "unknown," the process reverts to traditional scoreboard comparison or suspends manual confirmation. All four types of signals are automatically generated by the simulation environment without relying on manual annotation, thus engineering and quantifying the "confidence level" indicator and introducing it into the verification process scheduling.
[0030] For ease of understanding, Table 1 below provides a set of typical scenarios with examples of input, decision, and output.
[0031] Table 1. Scenarios for out-of-order output, conditional suppression of output, and verification of output semantic dependency pre-configuration. Scene Structured intermediate representation (example) Natural Language Decision Formula (Example) Out-of-order output Status: RUN, Request ID: req_id=3, Response ID: resp_id=3, Payload: 0xAB When the state is RUN and the req_id matches the resp_id, the response payload should be identical to the request payload. Out-of-order delivery is allowed, but mismatches are not permitted. Conditional inhibition State = WAIT, valid flag = 0, error flag = 0 When state is WAIT, valid is allowed to be 0 and no output is allowed, but the error flag must not be set. Configuration dependencies Configure cfg=mode MODE_B, field selection configuration bit field_sel=1, output value out=0x10 When cfg is MODE_B and field_sel=1, the output should be interpreted according to the semantics of MODE_B, with the lower 8 bits being valid. like Figure 3As shown, a decision-making process is presented, which integrates four types of signals—semantic consistency, rule coverage, inference stability, and contextual integrity—to obtain confidence levels and performs dual-track verification and confidence level rollback accordingly. The dual-track verification process includes: when the Large Language Model (LLM) concludes as a failure or the confidence level falls between a preset low threshold and a preset high threshold, a verification based on the Golden Reference Model (RM) or a formal checker is triggered; only when the dual-track conclusions of the LLM and the RM or formal checker are consistent is the corresponding event counted as a confirmation defect; if the dual-track conclusions are inconsistent, the process rolls back to a traditional scoring board for comparison or is suspended for manual verification. The confidence rollback process includes: when the confidence level is higher than a preset high threshold, the judgment conclusion of the Large Language Model (LLM) is directly written back to the simulator as a simulation assertion event; when the confidence level is between a preset low threshold and a preset high threshold, the Golden Reference Model (RM) or a formal checker is triggered for dual-track verification; when the confidence level is lower than a preset low threshold or the judgment conclusion of the Large Language Model (LLM) is undeterminable, the process rolls back to the traditional scoring board for comparison or suspends the process for manual confirmation.
[0032] In summary, this invention is particularly suitable for verification scenarios that traditional scoreboards struggle to handle, such as out-of-order output, conditionally suppressed output, and output semantic dependency pre-configuration. It can also alleviate model illusion through dual-track verification and confidence backoff mechanisms, thereby improving verification flexibility and coverage.
[0033] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.
Claims
1. A large language model-based digital chip dynamic simulation verification method, characterized in that, include: In the dynamic simulation environment of the design under test (DUT) running in the electronic design automation simulator, key signals and transaction information are extracted according to the simulation time point to generate a structured intermediate representation; Extract the preconditions and correctness constraints according to the design specifications, and serialize them into natural language judgment expressions; The natural language decision expression is filled with the structured intermediate representation to obtain the text to be inferred; Within the transaction or phase window triggered by the simulation event, the Large Language Model (LLM) is invoked to reason about the text to be judged, so as to output a structured reasoning result containing the judgment conclusion, confidence level and basis fragments. The structured reasoning results are written back to the simulator as assertion events to participate in the result determination of coverage statistics and regression verification.
2. The large language model-based digital chip dynamic simulation verification method according to claim 1, wherein, The structured intermediate representation contains a set of key-value pairs extracted and serialized from the interface signals, internal state machine codes, and transaction-level monitoring information of the Design Under Test (DUT), rather than a direct transmission of the original simulation logs.
3. The method for dynamic simulation verification of digital chips based on a large language model as described in claim 1, characterized in that, The natural language decision expression is derived from the design specifications using a rule template. The rule template includes a precondition field, a set of allowed behaviors field, and a correctness constraint field.
4. The method for dynamic simulation verification of digital chips based on a large language model as described in claim 1, characterized in that, The invocation of the large language model to perform reasoning on the text to be reasoned includes: dividing the simulation time axis into reasoning windows according to transaction boundaries or stage boundaries to achieve real-time reasoning coupled with the event loop of the electronic design automation simulator.
5. The method for dynamic simulation verification of digital chips based on a large language model as described in claim 1, characterized in that, The confidence level is obtained by weighted fusion of four types of signals: semantic consistency confidence level, rule coverage confidence level, inference stability confidence level, and context integrity confidence level.
6. The method for dynamic simulation verification of digital chips based on a large language model as described in claim 5, characterized in that, The semantic consistency confidence is obtained by reading the softmax probability distribution of the large language model when outputting decision words or the negative log-likelihood of the corresponding word token; The rule coverage confidence score is determined based on the ratio of the number of constraint rules explicitly referenced in the current natural language decision to the total number of constraint rules in the current simulation scenario. The inference stability confidence is obtained by repeatedly calling the large language model LLM N times with the same structured input within the same simulation event window and counting the consistency ratio of the output judgment results N times. The confidence level of context integrity is determined based on the sampling integrity of key signals within the current simulation event window. When a transaction is detected to cross the window boundary or a key signal is lost, the confidence level of context integrity is reduced accordingly.
7. The method for dynamic simulation verification of digital chips based on a large language model as described in claim 1, characterized in that, It also includes a dual-track review process. When the judgment conclusion of the Large Language Model (LLM) is failure or the confidence level is between the preset low threshold and the preset high threshold, a review based on the Golden Reference Model (RM) or a formal checker is triggered. Only when the dual-track conclusions of the Large Language Model (LLM) and the Golden Reference Model (RM) or the formal checker are consistent, the corresponding event is counted as a confirmed defect. If the dual-track conclusions are inconsistent, it is reverted to the traditional scoring board for comparison or suspended for manual confirmation.
8. The method for dynamic simulation verification of digital chips based on a large language model as described in claim 1, characterized in that, It also includes a confidence fallback process. When the confidence level is higher than a preset high threshold, the judgment conclusion of the Large Language Model (LLM) is directly written back to the simulator as a simulation assertion event. When the confidence level is between a preset low threshold and a preset high threshold, the Golden Reference Model (RM) or a formal checker is triggered for dual-track verification. When the confidence level is lower than a preset low threshold or the judgment conclusion of the Large Language Model (LLM) is undeterminable, the process falls back to the traditional scoring board for comparison or is suspended for manual confirmation.
9. The method for dynamic simulation verification of digital chips based on a large language model as described in claim 1, characterized in that, The electronic design automation simulator includes an EDA simulator; the simulation time points include clock edges, transaction start and end boundaries, or coverage event trigger points; the deployment method of the large language model (LLM) includes localized or controlled private deployment, and the simulation data does not leave the public cloud during the call process.
10. A dynamic simulation and verification system for digital chips based on a large language model, used to execute the dynamic simulation and verification method for digital chips based on a large language model as described in any one of claims 1 to 9, characterized in that, include: The monitoring and acquisition module is used to extract key signals and transaction information of the design under test (DUT) according to the simulation time point; The structured serialization module is used to generate structured intermediate representations; The rule template storage module is used to store rule template modules. According to the design specifications, the preconditions and correctness constraints are extracted through the rule template modules and serialized into natural language judgment expressions. The Large Language Model Inference Agent Module is used to invoke the deployed Large Language Model (LLM) to perform inference on the text to be inferred within the transaction or stage window triggered by the simulation event, and output a structured inference result containing the judgment conclusion, confidence level and basis fragments. The result feedback and dual-track verification module is used to write the structured reasoning results back to the simulator as assertion-type events. The coverage and regression statistics module is used to summarize assertion-type events and participate in the result determination of coverage statistics and regression validation.