An soc verification artifact generation method and system based on a large model and mutation feedback

Abstract

The application provides a SOC verification workpiece generation method and system based on a large model and mutation feedback, relates to the technical field of SOC chip verification, and comprises the following steps: obtaining a multi-modal design specification document of a design under test, generating a structured intermediate verification contract by means of semantic analysis of a large language model, generating a verification workpiece based on the intermediate verification contract, applying the verification workpiece to the design under test carrying a mutant to carry out mutation testing, calculating a mutation defect detection rate, constructing a feedback signal based on the detection rate, driving the large model to iteratively correct the intermediate verification contract and the verification workpiece, and outputting a target verification workpiece after the detection rate meets the standard. The beneficial effect is that the mutation test detection rate is used as the model iteration feedback basis, the verification workpiece defect detection capability is quantified, the verification blind area is located and optimized by means of the surviving mutants, the invalid workpiece and the constant true assertion defects caused by the traditional scheme relying on compilation and simulation logs are avoided, the verification workpiece generation quality is improved, and the cost of manual writing and auditing is reduced.

Description

Technical Field

[0001] This invention relates to the field of SOC chip verification technology, and in particular to a method and system for generating SOC verification artifacts based on large models and mutation feedback. Background Technology

[0002] With the continuous growth in the scale of system-on-a-chip (SoC), functional verification workload now accounts for more than 60% of the overall R&D investment. Currently, register-transfer level (RTL) functional verification mainly relies on verification engineers manually writing SystemVerilog assertions (SVA), universal verification methodology (UVM) stimulus sequences, and functional coverage points based on design specifications.

[0003] In recent years, the automatic generation of verification artifacts from design specifications using large language models has gradually become a technological trend. However, the self-correction feedback mechanisms of existing large language model generation schemes generally rely solely on whether the syntax compilation passes and simple simulation error logs. A syntactically correct verification artifact that passes simulation may not effectively constrain the critical behaviors of the design; a tautologically correct or weak assertion can still pass simulation smoothly, but it lacks real defect detection capabilities. This lack of objective measurement of the defect detection capability of the generated verification artifacts leads to inconsistent quality, easily including a large number of invalid artifacts, and still requires a significant amount of manual review. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a method for generating SOC verification artifacts based on a large model and mutation feedback, comprising the following steps: Step S1, obtaining a multimodal design specification document of the design under test, and using a large language model to perform semantic parsing on the multimodal design specification document to generate a structured intermediate verification contract; Step S2, generating a corresponding verification artifact based on the intermediate verification contract; Step S3, applying the verification artifact to the design under test injected with mutants for mutation testing, and statistically analyzing the proportion of mutants detected by the verification artifact to obtain the mutation test defect detection rate; Step S4, generating a feedback signal based on the mutation test defect detection rate, and using the feedback signal to drive the large language model to perform self-correction iteration on the intermediate verification contract and / or the verification artifact until the mutation test defect detection rate reaches a preset threshold, and outputting the target verification artifact.

[0005] Preferably, before step S2, the method further includes performing two-layer verification on the intermediate verification contract: a first-layer verification: performing a static syntax and logical validity check on the intermediate verification contract; a second-layer verification: verifying the completeness and consistency between the intermediate verification contract and the multimodal design specification document; wherein, only after passing the first-layer verification and the second-layer verification, the intermediate verification contract is used as the unique truth source for generating the verification artifact, and the verification artifact is generated based on the unique truth source.

[0006] Preferably, the first layer of verification includes: verifying the existence and bit width matching of signals and registers referenced in the intermediate verification contract in the symbol table of the design under test; verifying the legality of timing intervals in the intermediate verification contract; and using a solver to detect mutual exclusion contradictions between multiple intermediate verification contracts.

[0007] Preferably, the second layer of verification includes: constructing a bidirectional traceability matrix between the numbered requirement items in the multimodal design specification document and the intermediate verification contract to verify completeness; reversing the intermediate verification contract into natural language text according to a preset template, and using a second major language model to determine whether the natural language text and the corresponding requirement items are semantically equivalent to verify consistency; performing a three-stage processing on the results of the second layer of verification: directly releasing intermediate verification contracts that are determined to be semantically equivalent and have no coverage gaps; automatically reverting intermediate verification contracts that are determined to have semantic conflicts or coverage gaps for re-parsing or regeneration; submitting intermediate verification contracts that are determined to be questionable to the reviewers, and backfilling the review conclusions into incremental annotation data.

[0008] Preferably, step S2 specifically comprises: step S21, constraining the generation logic of the large language model based on a preset rule template, and setting a reserved slot in the rule template; step S22, using the large language model to fill only the reserved slot with signals and parameters derived from the intermediate verification contract, generating at least one verification artifact including SVA assertions, UVM stimulus sequences and functional coverage points.

[0009] Preferably, in step S4, the feedback signal driving the self-correction iteration further includes formal model verification feedback. The process of obtaining the formal model verification feedback is as follows: the generated SVA assertion is input into the formal verification engine with the design under test for attribute verification. When the SVA assertion does not conform to the intermediate verification contract or there is a tautology problem, the corresponding counterexample alarm information or tautology alarm information is obtained. The counterexample alarm information or tautology alarm information and the mutation test defect detection rate together constitute a dual feedback signal, and the dual feedback signal is used to drive the large language model to perform the self-correction iteration on the intermediate verification contract and / or the verification artifact.

[0010] Preferably, in steps S3 and S4: the mutant type injected into the design under test includes at least one of bit flip, operator substitution, timing offset and branch negation; when the defect detection rate of the mutation test does not reach a preset threshold, the test blind area not covered by the verification workpiece is located in reverse according to the undetected surviving mutants, and the test blind area is converted into structured prompt information and fed back to the large language model.

[0011] Preferably, the method further includes step S5, which is executed after step S4, and includes: step S51, obtaining the intermediate representation of the real driver operation from the driver multiplexing stimulus library and the running trajectory collected from the on-chip hardware monitoring unit; step S52, performing a three-source consistency comparison between the intermediate verification contract, the intermediate representation of the real driver operation and the running trajectory, and when there is an inconsistency, feeding back the inconsistency information to step S1 and / or step S2 to correct the intermediate verification contract and / or the verification artifact.

[0012] This invention also provides a SOC verification artifact generation system based on a large model and mutation feedback. Applying the aforementioned SOC verification artifact generation method, the system includes: a multimodal specification parsing and contract generation module, used to acquire multimodal design specification documents of the design under test, and to perform semantic parsing of the multimodal design specification documents using a large language model to generate a structured intermediate verification contract; a verification artifact generation module, used to generate corresponding verification artifacts based on the intermediate verification contract; a mutation testing module, used to apply the verification artifacts to the design under test injected with mutants for mutation testing, and to statistically analyze the proportion of mutants detected by the verification artifacts to obtain a mutation test defect detection rate; and a dual-feedback self-correction engine, used to generate a feedback signal based on the mutation test defect detection rate, and to use the feedback signal to drive the large language model to perform self-correction iteration on the intermediate verification contract and / or the verification artifact until the mutation test defect detection rate reaches a preset threshold, and then output the target verification artifact.

[0013] The above technical solution has the following advantages or beneficial effects: By parsing the multimodal design specification document into a structured intermediate verification contract, and introducing the mutation test defect detection rate as the core feedback signal driving the self-correction iteration of the large language model, the actual defect detection capability of the generated verification artifact is directly quantitatively evaluated and targetedly enhanced. Surviving mutants can accurately expose verification blind spots and drive model reinforcement, fundamentally eliminating the invalid artifacts and tautological assertion problems caused by existing technologies relying solely on syntax compilation and simulation error logs. This significantly improves the quality of the generated verification artifact and significantly reduces the workload of manual writing and review. Attached Figure Description

[0014] Figure 1 A flowchart illustrating a SOC verification artifact generation method based on a large model and mutation feedback, as a preferred embodiment of the present invention. Figure 2 A schematic diagram of a two-layer automated verification mechanism for the Contract Verification (VC) is shown in a preferred embodiment of the present invention. Figure 3 This is a schematic diagram of a sub-process of step S2 in a preferred embodiment of the present invention; Figure 4 This is a schematic diagram of the mutation test defect detection rate feedback mechanism in a preferred embodiment of the present invention. Detailed Implementation

[0015] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment; other embodiments that conform to the spirit of the present invention may also fall within the scope of the present invention.

[0016] In this article, the following abbreviations are defined: SOC stands for System on Chip; RTL stands for Register Transfer Level; LLM stands for Large Language Model; SVA stands for SystemVerilog Assertions; UVM stands for Universal Verification Methodology; DUT stands for Design Under Test; AST stands for Abstract Syntax Tree; SAT / SMT stands for Satisfiability / Satisfiability Modulo Theories; and NLI stands for Natural Language Inference.

[0017] In a preferred embodiment of the present invention, based on the above-mentioned problems existing in the prior art, a method for generating SOC verification artifacts based on large models and mutation feedback is provided, such as... Figure 1As shown, the process includes the following steps: Step S1, obtaining the multimodal design specification document of the design under test, performing semantic parsing on the multimodal design specification document using a large language model, and generating a structured intermediate verification contract; Step S2, generating the corresponding verification artifact based on the intermediate verification contract; Step S3, applying the verification artifact to the design under test injected with mutants for mutation testing, statistically analyzing the proportion of mutants detected by the verification artifact, and obtaining the mutation test defect detection rate; Step S4, generating a feedback signal based on the mutation test defect detection rate, and using the feedback signal to drive the large language model to perform self-correction iteration on the intermediate verification contract and / or verification artifact until the mutation test defect detection rate reaches a preset threshold, and outputting the target verification artifact.

[0018] Specifically, in this embodiment, the multimodal design specification document of the design under test includes functional descriptions in natural language, register description tables, and timing waveform diagrams, among other information for various modalities.

[0019] In step S1, a large language model is invoked for semantic parsing, the register description table is structurally extracted, and the timing waveform is image-recognized to output a unified set of design intent elements, including signals, register bit fields, timing relationships, cross-IP interactions, and functional or security intents. Subsequently, the set of design intent elements is converted into a structured intermediate verification contract. The intermediate verification contract is an intermediate representation between natural language and target code, with restricted syntax and well-defined fields. Its structured fields include at least a contract identifier, associated signals or registers, timing obligations (including preconditions, post-responses, and clock intervals), cross-IP dependencies, intent type (e.g., functional or security), expected coverage targets, and specification source number.

[0020] In step S2, the corresponding verification artifact is derived based on the intermediate verification contract. The verification artifact includes at least one of SVA assertion, UVM stimulus sequence and functional coverage point.

[0021] In steps S3 and S4, the mutation test feedback process is executed. A batch of semantic mutants is automatically injected into the RTL of the design under test (DUT), and the generated verification artifact is used to perform simulation verification on the DUT with injected mutants. The number of detected mutants is recorded as follows: The total number of injected mutants is denoted as The formula for calculating the defect detection rate in mutation testing is:

[0022] Mutation test defect detection rate This directly quantifies the ability of generated validation artifacts to capture real defects. Undetected surviving mutants precisely pinpoint design behavior blind spots not covered by the current validation artifact.

[0023] When the mutation test defect detection rate does not reach the preset threshold, the list of surviving mutants and their corresponding uncovered design behaviors are converted into structured prompt information and fed back to step S1 or step S2 to drive the large language model to be re-parsed or regenerated in a targeted manner, forming a closed loop iteration until the mutation test defect detection rate meets the preset threshold requirement.

[0024] This embodiment solves the technical problem in the prior art that relying solely on syntax compilation or simulation run logs leads to a single feedback dimension and makes it impossible to evaluate the actual defect detection capability of the verification workpiece. By using the defect detection rate of mutation testing as the core indicator for evaluation and error correction feedback, this embodiment ensures that the generated verification workpiece has extremely high effectiveness and defect capture capability, greatly reducing the workload of engineers' manual review.

[0025] In another preferred embodiment of the present invention, based on the above-described SOC verification artifact generation method, before step S2, the method further includes performing two-layer verification on the intermediate verification contract: First-layer verification: performing static syntax and logical validity checks on the intermediate verification contract; Second-layer verification: verifying the completeness and consistency between the intermediate verification contract and the multimodal design specification document; wherein, in step S2, the intermediate verification contract after passing the first-layer verification and the second-layer verification is used as the unique truth source for generating the verification artifact, and the verification artifact is generated based on the unique truth source.

[0026] Specifically, in this embodiment, the two-layer verification automatically intercepts phantom errors in specification parsing before any target code is generated. Only intermediate verification contracts that have completely passed the first-layer syntax and logic validity check and the second-layer consistency and completeness check are allowed to serve as the sole source of truth for the subsequent generation of SVA assertions, UVM stimuli, and functional coverage points.

[0027] This embodiment solves the technical problem that large language models are prone to semantic illusions and lack independent verification intermediates when generating code directly from natural language by applying deterministic two-layer automated verification before code generation. It ensures that the intermediate verification contract becomes a highly reliable and unique source of truth, intercepting parsing errors in the early stages.

[0028] In another preferred embodiment of the present invention, based on the above-described SOC verification workpiece generation method, the first layer of verification includes: verifying the existence and bit width matching of signals and registers referenced in the intermediate verification contract in the design symbol table under test; verifying the legality of timing intervals in the intermediate verification contract; and using a solver to detect mutual exclusion contradictions between multiple intermediate verification contracts.

[0029] Specifically, the first layer of static checks in this embodiment is as follows: Figure 2 As shown, the verification toolchain automatically performs deterministic execution: Step 1, Signal and Register Existence Verification: This involves parsing the abstract syntax tree and symbol reference list of the intermediate verification contract using a parser generator (such as ANTLR, Lark, or PLY) or a structured schema verification tool (such as JSON Schema or Pydantic) on the front end; simultaneously, an RTL parsing tool (such as Verible or Yosys) is used to export the symbol table of the design under test. The signals, registers, and bit fields referenced by the intermediate verification contract are compared one by one with the symbol table; if a reference is missing or the name does not match, it is considered an error.

[0030] Step 2, Bit Width and Bit Field Matching Verification: The bit widths of constants, comparisons, and assignments involved in the intermediate verification contract are automatically compared with the bit widths declared in the design to detect whether there are out-of-bounds errors or bit width mismatches.

[0031] Step 3, Temporal Interval Validation: Traverse and check the abstract syntax tree of the temporal obligations, requiring that the temporal interval constraints satisfy... (For example, represented as) And ensure that the clock domain is consistent and there are no empty intervals.

[0032] Step 4, Contract self-consistency test: Directly call the solver (such as Z3, CVC5, Yices, etc. SAT / SMT solvers) to automatically detect whether there are mutual contradictions among multiple contracts acting on the same signal. For example, two contracts require the same signal to take opposite values ​​under the same conditions.

[0033] This embodiment ensures the syntactic rigor and logical consistency of the intermediate verification contract itself without human intervention through deterministic static analysis based on symbol table comparison, abstract syntax tree traversal, and solver proof.

[0034] In another preferred embodiment of the present invention, based on the above-described SOC verification artifact generation method, the second layer of verification includes: constructing a bidirectional traceability matrix between the numbered requirement items in the multimodal design specification document and the intermediate verification contract to verify completeness; reversing the intermediate verification contract into natural language text according to a preset template, and using a second large language model to determine whether the natural language text and the corresponding requirement item are semantically equivalent to verify consistency; performing a three-stream processing on the verification results of the intermediate verification contract: directly releasing intermediate verification contracts that are determined to be semantically equivalent and have no coverage gaps as the sole source of truth; automatically reverting intermediate verification contracts that are determined to have semantic conflicts or coverage gaps for re-parsing or regeneration; submitting intermediate verification contracts that are determined to be questionable to the reviewers, and backfilling the review conclusions into incremental annotation data.

[0035] Specifically, in this embodiment, the second layer of verification targets the consistency and completeness between the contract and the specification document, and the mechanism is as follows: Figure 2As shown: In step S1, the specification document is split into atomic requirement items with unique numbers. The second-level verification automatically constructs a bidirectional traceability matrix between the requirement items and intermediate verification contracts. The program automatically locates requirement items that are not covered by any contract (coverage gaps) or contracts whose specification source cannot be found (suspected parsing illusions).

[0036] Meanwhile, the structured intermediate verification contract is automatically reversed and restated into natural language text according to a fixed template, and then handed over to an independent second large language model instance for natural language reasoning (NLI) semantic equivalence determination.

[0037] The verification results adopt a three-stream mechanism: those determined to be semantically equivalent and without gaps are directly released as the sole source of truth; those determined to be semantically conflicting or with coverage gaps are automatically reverted to step S1 for re-parsing or regeneration; only a small number of entries determined to be questionable are submitted to engineers for exception review, and the review conclusions are backfilled as incremental labeled data to continuously reduce the subsequent questionable rate.

[0038] This embodiment achieves automated verification of the completeness and consistency between specifications and contracts through traceability matrix, reverse paraphrasing, and three-way flow processing. Humans only need to make exceptions for a few questionable items, which greatly reduces the burden of manual review.

[0039] In another preferred embodiment of the present invention, based on the above-described SOC verification workpiece generation method, such as... Figure 3 As shown, step S2 includes: step S21, constraining the generation logic of the large language model based on a preset rule template, and setting a reserved slot in the rule template; step S22, using the large language model to fill only the reserved slot with signals and parameters derived from the intermediate verification contract, generating at least one verification artifact including SVA assertions, UVM stimulus sequences and functional coverage points.

[0040] Specifically, this embodiment employs a rule template constraint decoding mechanism during the verification artifact generation stage. The rule template fixes the grammatical framework and code paradigm of SVA assertions, UVM stimulus sequences, or functional coverage points, and reserves slots in the template. The generation logic of the large language model is strictly constrained, and is only allowed to fill the reserved slots with signal names, register bit fields, and parameters derived from the verified intermediate verification contract.

[0041] This embodiment constrains the generation freedom of the large language model through rule templates, which solves the technical problem that general large language models are prone to syntax errors and code structure illusions when generating complex verification code, and ensures the syntactic validity of the generated verification artifact and its strict correspondence with the intermediate verification contract.

[0042] In another preferred embodiment of the present invention, based on the above-described SOC verification artifact generation method, in step S4, the feedback signal driving the self-correction iteration further includes formal model verification feedback. The process of obtaining formal model verification feedback is as follows: the generated SVA assertion is input into the formal verification engine with the design under test for attribute verification. When the SVA assertion does not conform to the intermediate verification contract or there is a tautology problem, the corresponding counterexample alarm information or tautology alarm information is obtained. The counterexample alarm information or tautology alarm information and the mutation test defect detection rate together constitute a dual feedback signal, which is used to drive the large language model to perform self-correction iteration on the intermediate verification contract and / or verification artifact in step S4.

[0043] Specifically, in this embodiment, the dual feedback mechanism is as follows: Figure 4 As shown, the generated SVA assertions and RTL design are fed into the formal verification engine for property proof. During the proof process, if an assertion is found to be true because its preconditions are never met, or if an assertion fails to meet the design behavior, resulting in a property counterexample, counterexample alarm information or true property alarm information is extracted. The counterexample alarm information or true property alarm information, together with the mutation test defect detection rate, constitute a dual feedback signal, which is fed back to the large language model to drive the self-correction iteration in step S4.

[0044] This embodiment solves the technical problem that single simulation feedback cannot discover constant assertions or logical blind spots by introducing a dual feedback mechanism consisting of formal model verification feedback and mutation test defect detection rate. It ensures the rigor of the verified workpiece from two dimensions: mathematical proof and experimental injection.

[0045] In another preferred embodiment of the present invention, based on the above-described SOC verification workpiece generation method, in steps S3 and S4: the mutant type injected into the design under test includes at least one of bit flip, operator replacement, timing offset and branch negation; when the mutation test defect detection rate does not reach a preset threshold, the test blind area not covered by the verification workpiece is located in reverse according to the undetected surviving mutants, and the test blind area is converted into structured prompt information and fed back to the large language model.

[0046] Specifically, this embodiment uses the verification of a single-IP APB timer as an example to illustrate the mutation testing and defect blind zone feedback process. The design under test is an APB timer. The multimodal design specification document contains a natural language description: "When the control register CTRL.EN is set to 1, the timer starts counting within 3 clock cycles; when the count value reaches the value of the LOAD register, the interrupt flag STAT.INT is set and irq is pulled high." The register description table gives the bit field definitions of CTRL (address 0x00), LOAD (address 0x04), and STAT (address 0x08).

[0047] In step S1, the following intermediate verification contract is generated: contract :EN rising edge ##[1:3] counting == 1; contract :cnt == LOAD ##1 STAT.INT == 1 && irq == 1.

[0048] Before step S2, the first-level static check confirms that counting, STAT.INT, irq, cnt, and LOAD all exist in the RTL symbol table and have matching bit widths, and the interval ##[1:3] is valid (satisfying the condition). ), and the contract 1 and The solver determined that there were no mutually exclusive contradictions; the second-level verification confirmed that all specification requirements were accepted and semantically equivalent, and allowed it to be the only source of truth.

[0049] In steps S3 and S4, mutants are injected into the APB timer RTL. Injection types include replacing the counting comparison operator with >= and deleting IRQ assignment statements. The defect detection rate obtained from the mutation test statistics of the initially generated verification artifact is 82%. Based on the undetected surviving mutants, a test blind spot where the current verification artifact has an IRQ that is one beat short at the LOAD boundary is identified. This test blind spot is converted into structured prompt information and fed back to the large language model to drive its contract reinforcement. Boundary assertions increase the defect detection rate of mutation testing to 100% and eliminate tautological assertions.

[0050] This embodiment solves the technical problem of inaccurate blind zone positioning during the verification workpiece generation process by refining the mutant injection and surviving mutant reverse positioning mechanism, and realizes the targeted self-evolution of the large language model and workpiece reinforcement.

[0051] In another preferred embodiment of the present invention, based on the above-described SOC verification artifact generation method, the method further includes step S5, which is executed after step S4, including: step S51, obtaining the intermediate representation of the real driver operation from the driver multiplexing excitation library and the running trajectory collected from the on-chip hardware monitoring unit; step S52, performing a three-source consistency comparison between the intermediate verification contract, the intermediate representation of the real driver operation and the running trajectory, and when there is an inconsistency, feeding back the inconsistency information to step S1 and / or step S2 to correct the intermediate verification contract and / or verification artifact.

[0052] Specifically, in steps S51 and S52 of the three-source cross-verification mechanism in this embodiment, the intermediate verification contract generated in step S1, the intermediate representation of the actual driver operation exported from the software driver source code, and the running trajectory collected from the on-chip hardware monitoring unit are uniformly named and mapped, and then a consistency comparison is performed. The comparison checks whether the access order, key data, and timing logic of the three to the same peripheral, register, or interrupt are consistent.

[0053] In a system-level cross-IP application scenario (DMA and UART collaboration), the specification describes that "the CPU configures the DMA to move the memory buffer to the UART transmit FIFO. After the move is complete, the DMA generates a completion interrupt, and then the UART transmits byte by byte." In step S1, an intermediate verification contract containing cross-IP dependencies is generated (constraining DMA.done to precede UART.tx_start). In step S5, when cross-referencing the contract with the actual DMA-UART execution trajectory, a FIFO full-pressure behavior not mentioned in the specification is exposed. This inconsistency is fed back to step S1 to correct the intermediate verification contract and generation logic.

[0054] This embodiment solves the technical problem of existing generation schemes being disconnected from actual chip operation data and unable to use actual operation trajectories to verify the correctness of generated workpieces by introducing specification-driver-monitoring three-source cross-verification, and establishes a more reliable verification closed loop.

[0055] In another preferred embodiment of the present invention, the verification workpiece generation and collaborative verification process are specifically described using a system-level cross-IP DMA and UART collaborative verification scenario as an example.

[0056] In this embodiment, the input multimodal design specification document includes a natural language description of "CPU configures DMA to move data from memory buffer to UART transmit FIFO. After the data is moved, DMA generates a completion interrupt, and then UART triggers the transmission process", and also includes bit field definitions of DMA control register and UART status register.

[0057] In step S1, the above specifications are parsed using a large language model, and relevant signals and registers of the two IPs, DMA and UART (such as DMA channel configuration, DMA.done, UART transmit FIFO and tx_start, etc.) and their cross-IP timing dependencies are extracted, and a unified set of design intent elements is output; then, a structured intermediate verification contract containing cross-IP timing dependencies (i.e., constraining the DMA transfer completion signal to precede the UART transmit start signal) is extracted.

[0058] Before step S2, the correctness and completeness of the intermediate verification contract are confirmed through first-layer and second-layer checks: The first-layer static check automatically compares the RTL symbol tables of the DMA and UART IPs to confirm that signals such as DMA.done, UART.tx_start, and FIFO write pointer exist and their bit widths match. It also verifies that the clock domain labels of the signals involved in the cross-IP timing obligations are consistent (if the two are asynchronous, the intermediate verification contract is required to explicitly declare cross-clock domain constraints to avoid implicit synchronization assumptions), and each contract is determined by the SAT / SMT solver to have no mutual exclusion contradictions. The second-layer check automatically confirms by the requirement traceability matrix that all requirements of the specification (transfer, completion interruption, byte-by-byte transmission, and their order) are covered by the intermediate verification contract without any coverage gaps. The cross-IP timing contract is then reverse-paraphrased into natural language and determined by an independent large language model to be semantically equivalent to the original specification. Only after both layers of checks have passed automatically is the intermediate verification contract released as the sole truth source for generating the verification artifact.

[0059] In steps S2, S3, and S4, corresponding cross-IP SVA assertions and UVM system-level stimulus sequences are generated, and cross-IP interactive mutants are injected into the system-level RTL. Based on the feedback of the defect detection rate of the mutation test, the large language model is supplemented with constraints to ensure that the defect detection rate of the system-level verification workpiece reaches a preset threshold.

[0060] In step S5, a three-source consistency comparison is performed between the intermediate verification contract, the intermediate operation representation from the DMA driver function in the driver multiplexing stimulus library, and the actual bus and FIFO operation trajectory collected from the on-chip hardware monitoring unit. During the comparison, the hardware operation trajectory reveals a hardware behavior not explicitly described in the specification document—when the UART transmit FIFO reaches the near-full threshold, a backpressure pull-down DMA request is triggered.

[0061] When an inconsistency is detected among the three sources, the inconsistency information is fed back to steps S1 and S2, driving the large language model to automatically supplement the timing wait clause under the FIFO backpressure condition in the intermediate verification contract, and update the corresponding system-level SVA assertion and UVM response sequence, thus completing the closed-loop correction of the generated verification artifact and intermediate verification contract based on the actual running trajectory.

[0062] This embodiment demonstrates the value of two-layer automated verification and three-source cross-verification of system-level cross-IP intent expression and cross-IP contract.

[0063] In another preferred embodiment of the present invention, a SOC verification artifact generation system based on large model and mutation feedback is provided, applying the SOC verification artifact generation method of any of the above embodiments. The system includes: a multimodal specification parsing and contract generation module, comprising a multimodal specification parsing module and a verification contract generation module, used to obtain the multimodal design specification document of the design under test, and use a large language model to perform semantic parsing on the multimodal design specification document to generate a structured intermediate verification contract; and a verification artifact generation module, used to generate corresponding verification artifacts based on the intermediate verification contract. The mutation testing module applies the verification artifact to the design under test (DUT) injected with mutants for mutation testing, statistically analyzes the proportion of mutants detected in the verified artifact, and obtains the mutation test defect detection rate. The dual-feedback self-correction engine generates a feedback signal based on the mutation test defect detection rate, and uses this feedback signal to drive a large language model to perform self-correction iterations on intermediate verification contracts and / or verification artifacts until the mutation test defect detection rate reaches a preset threshold, at which point the target verification artifact is output. The collaborative verification docking module acquires the intermediate representation of real driver operations from the driver reuse stimulus library and the running trajectory collected from the on-chip hardware monitoring unit. It performs a three-source consistency comparison between the intermediate verification contract, the intermediate representation of real driver operations, and the running trajectory. When inconsistencies exist, it provides feedback to correct the parsing results or contract content, and finally imports the verified artifact that has passed the final approval into the simulation regression platform.

[0064] This embodiment of the system achieves full-process automation and highly reliable self-correction from multimodal specification input to high-quality SOC verification workpiece output through the coordinated cooperation between various modules, solving the technical problems of low efficiency and difficulty in quantifying quality in traditional manual writing of verification workpieces.

[0065] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made using the content of this specification and illustrations should be included within the protection scope of the present invention.

Claims

1. A method for generating SOC verification artifacts based on large models and mutation feedback, characterized in that, Includes the following steps: Step S1: Obtain the multimodal design specification document of the design under test, and use a large language model to perform semantic parsing on the multimodal design specification document to generate a structured intermediate verification contract. Step S2: Generate a corresponding verification artifact based on the intermediate verification contract; Step S3: Apply the verification artifact to the design under test injected with mutants for mutation testing, and count the proportion of mutants detected by the verification artifact to obtain the mutation test defect detection rate; Step S4: Generate a feedback signal based on the mutation test defect detection rate, and use the feedback signal to drive the large language model to perform self-correction iteration on the intermediate verification contract and / or the verification artifact until the mutation test defect detection rate reaches a preset threshold, and output the target verification artifact.

2. The SOC verification workpiece generation method based on large model and mutation feedback according to claim 1, characterized in that, Before step S2, the method further includes performing two layers of verification on the intermediate verification contract: a first layer of verification: performing static syntax and logical validity checks on the intermediate verification contract itself; a second layer of verification: verifying the completeness and consistency between the intermediate verification contract and the multimodal design specification document; wherein, only after passing the first layer of verification and the second layer of verification, the intermediate verification contract is used as the unique truth source for generating the verification artifact, and the verification artifact is generated based on the unique truth source.

3. The SOC verification workpiece generation method based on large model and mutation feedback according to claim 2, characterized in that, The first layer of verification includes: verifying the existence and bit width matching of signals and registers referenced in the intermediate verification contract in the symbol table of the design under test; verifying the validity of timing intervals in the intermediate verification contract; and using a solver to detect mutual exclusion contradictions between multiple intermediate verification contracts.

4. The SOC verification workpiece generation method based on large model and mutation feedback according to claim 2, characterized in that, The second layer of verification includes: constructing a bidirectional traceability matrix between the numbered requirement items in the multimodal design specification document and the intermediate verification contract to verify completeness; reversing the intermediate verification contract into natural language text according to a preset template, and using a second major language model to determine whether the natural language text and the corresponding requirement items are semantically equivalent to verify consistency; performing a three-stage processing on the results of the second layer of verification: allowing intermediate verification contracts that are determined to be semantically equivalent and have no coverage gaps to be released directly; automatically reverting intermediate verification contracts that are determined to have semantic conflicts or coverage gaps for re-parsing or regeneration; and submitting intermediate verification contracts that are determined to be questionable to the reviewers, and backfilling the review conclusions into incremental annotation data.

5. The SOC verification workpiece generation method based on large model and mutation feedback according to claim 1, characterized in that, Step S2 specifically comprises: Step S21, constraining the generation logic of the large language model based on a preset rule template, and setting a reserved slot in the rule template; Step S22, using the large language model to fill only the reserved slot with signals and parameters derived from the intermediate verification contract, generating at least one verification artifact including SVA assertions, UVM stimulus sequences and functional coverage points.

6. The SOC verification workpiece generation method based on large model and mutation feedback according to claim 1, characterized in that, In step S4, the feedback signal driving the self-correction iteration also includes formal model verification feedback. The process of obtaining the formal model verification feedback is as follows: the generated SVA assertion is input into the formal verification engine with the design under test for attribute verification. When the SVA assertion does not conform to the intermediate verification contract or there is a tautology problem, the corresponding counterexample alarm information or tautology alarm information is obtained. The counterexample alarm information or tautology alarm information and the mutation test defect detection rate together constitute a dual feedback signal, and the dual feedback signal is used to drive the large language model to perform the self-correction iteration on the intermediate verification contract and / or the verification artifact.

7. The SOC verification workpiece generation method based on large model and mutation feedback according to claim 1, characterized in that, In steps S3 and S4: the mutant types injected into the design under test include at least one of bit flip, operator substitution, timing offset, and branch negation; when the defect detection rate of the mutation test does not reach a preset threshold, the test blind area not covered by the verification workpiece is located in reverse according to the undetected surviving mutants, and the test blind area is converted into structured prompt information and fed back to the large language model.

8. The SOC verification workpiece generation method based on large model and mutation feedback according to claim 1, characterized in that, The method further includes step S5, which is executed after step S4, and includes: step S51, obtaining the intermediate representation of the real driver operation from the driver multiplexing stimulus library and the running trajectory collected from the on-chip hardware monitoring unit; step S52, performing a three-source consistency comparison between the intermediate verification contract, the intermediate representation of the real driver operation and the running trajectory, and when there is an inconsistency, feeding back the inconsistency information to step S1 and / or step S2 to correct the intermediate verification contract and / or the verification artifact.

9. A SOC verification workpiece generation system based on large model and mutation feedback, characterized in that, The SOC verification artifact generation method according to any one of claims 1-8 includes: a multimodal specification parsing and contract generation module, used to acquire a multimodal design specification document of the design under test, and to perform semantic parsing of the multimodal design specification document using a large language model to generate a structured intermediate verification contract; a verification artifact generation module, used to generate a corresponding verification artifact based on the intermediate verification contract; a mutation testing module, used to apply the verification artifact to the design under test injected with mutants for mutation testing, and to count the proportion of mutants detected by the verification artifact to obtain a mutation test defect detection rate; and a dual-feedback self-correction engine, used to generate a feedback signal based on the mutation test defect detection rate, and to use the feedback signal to drive the large language model to perform self-correction iteration on the intermediate verification contract and / or the verification artifact until the mutation test defect detection rate reaches a preset threshold, and to output the target verification artifact.

Images