A method and system for inference enhancement for automated RTL debugging

Abstract

The application discloses a kind of inference enhancement method and system for automated RTL debugging, method includes the following steps: by the mixed error injection mechanism of rule and LLM combination constructs diversified debugging sample, and utilizes GPT-4 to step level error positioning annotation to error CoT;Model training adopts gradual optimization process, so that it has the ability to evaluate the correctness of each step reasoning;In the reasoning phase, the candidate repair path of multiple strip CoT is generated in parallel by the generation model;Process reward model then carries out reliability scoring to each step of each path, and selects the path with the highest comprehensive score as the final output;The present application realizes the explicit supervision of intermediate reasoning, ensures that the debugging process is not only correct in results, but also reliable and traceable in logic, greatly improves the debugging success rate in complex RTL scenario.

Description

Technical Field

[0001] This invention relates to the field of large model technology, and in particular to an inference enhancement method and system for automated RTL debugging. Background Technology

[0002] Existing automated RTL debugging solutions are mainly divided into two categories: traditional automated tool-based methods (such as RTL-Repair, Strider) and LLM-based intelligent methods (such as MEIC, VeriDebug, Lik, etc.). The limitations of tool-based solutions are: (1) severely limited generalization ability: relying on predefined rules and templates, it is difficult to cover diverse and unseen error types, especially in scenarios involving complex control flow and data flow interaction. (2) high expansion and maintenance costs: requiring domain experts to continuously maintain and expand the rule base, making it difficult to adapt to rapidly iterating design environments and new error patterns. The limitations of LLM-based solutions are: (1) lack of process supervision and intermediate inference reliability: existing methods generally take the correctness of the final repair as the optimization goal, ignoring the step-by-step supervision of the debugging inference chain, which makes the model prone to "illusion" in intermediate steps, and the error propagates along the thought chain, seriously affecting the debugging reliability. (2) insufficient training data and lack of step-level annotation: currently there is a lack of debugging process datasets containing error location and correct / error step annotations, which limits the model from learning interpretable and verifiable inference patterns. (3) Weak understanding of complex hardware behavior: When faced with hardware-specific behaviors such as multi-cycle timing and concurrent signals, the inference accuracy of LLM drops significantly, and the generated repair solutions often do not meet hardware semantics or verification requirements. Currently, there is a lack of frameworks that can provide fine-grained, supervised inference processes while maintaining the generalization ability of LLM. Existing methods either rely too much on rules and lose flexibility, or the lack of process control leads to unreliable outputs. Summary of the Invention

[0003] Purpose of the Invention: The purpose of this invention is to provide an inference enhancement method for automated RTL debugging. It employs a process reward model (PRM) to provide a fine-grained, step-level supervision agent framework for multi-step RTL debugging inference, aiming to achieve highly reliable and interpretable automated RTL error localization and repair, thereby solving the problems in the background art.

[0004] Technical Solution: To achieve the above objectives, the present invention provides the following technical solution:

[0005] In a first aspect, the present invention provides an inference enhancement method for automated RTL debugging, comprising the following steps:

[0006] Step 1: Construct samples containing register-transfer level design flaws through an automated process, and obtain corresponding debug inference chains containing multi-step inference processes for each sample; verify the correctness of the debug inference chains, and automatically label each step of the debug inference chain that fails verification, so as to form a debug training dataset containing step-level correctness labels.

[0007] Step 2: Use the debug training dataset to perform the first training on the generative model to obtain the mapping ability from design defects and design specifications to the correct reasoning chain; perform the second training on the generative model to enhance the ability to distinguish between correct and incorrect reasoning; use the step-level correctness labels in the debug training dataset to train the process reward model to obtain the ability to perform fine-grained evaluation of reasoning steps.

[0008] Step 3: Based on the design defects and design specifications to be addressed, drive the trained generative model to generate multiple candidate debugging paths containing inference steps in parallel; call the trained process reward model to evaluate the correctness of each inference step in each candidate debugging path and obtain the confidence evaluation of each step; based on the confidence evaluation of all steps in each candidate debugging path, select the one with the best overall inference quality from the multiple candidate debugging paths as the final debugging result and output it.

[0009] In one possible implementation, step 1 includes:

[0010] Step 11: Obtain the functionally correct register-transfer stage design and its textual specifications. Use a combination of rule-based error injection and intelligent error injection to modify the functionally correct register-transfer stage design and generate diverse register-transfer stage design defect samples.

[0011] Step 12: Input the design defect samples of each register transfer stage and their corresponding specifications into multiple heterogeneous language models to obtain candidate responses generated by each language model, which include step-by-step debugging reasoning processes and corresponding fix codes;

[0012] Step 13: Perform functional equivalence verification on the repair code in each candidate response. If the repaired design function is equivalent to the original correct design, mark the complete inference chain in the candidate response as the correct inference chain; otherwise, mark it as the incorrect inference chain.

[0013] In one possible implementation, each step of the debugging inference chain that fails verification is automatically labeled, including: for a reasoning chain marked as erroneous, using an evaluation model to evaluate each inference step in the chain sequentially to estimate the probability that each inference step is the first step to encounter an error; selecting the inference step with the highest probability as the hypothetical error starting point, extracting the part of the inference chain before the error starting point, and regenerating the subsequent inference content and repair code based on this part of the inference chain; if the regenerated repair code passes the functional equivalence verification, then all inference steps before the hypothetical error starting point are labeled as correct, and the hypothetical error starting point and all inference steps thereafter are labeled as erroneous.

[0014] In one possible implementation, in step 2, the first training of the generative model is supervised fine-tuning training, which uses the debug inference chains marked as correct in the debug training dataset as the learning target, so that the generative model learns to generate correct inference patterns that conform to hardware debugging logic.

[0015] In one possible implementation, the second training of the generative model is direct preference optimization training, which uses preference data pairs consisting of correct and incorrect inference chains to train the generative model to form a probability distribution difference between generating correct and incorrect inference chains, thereby suppressing the generation of outputs that do not conform to the hardware logic during the inference process.

[0016] In one possible implementation, in step 2, the training process reward model includes: taking register transfer level design specifications, design defects, and a debug inference chain containing step-level correctness labels from the debug training dataset as input, and the training process reward model represents the output of each step in the inference chain as a confidence value of the probability of correctness.

[0017] In one possible implementation, step 3 includes: controlling the generative model to generate multiple candidate debugging paths for the same register-transfer level design defect and design specification in sampling mode, each candidate debugging path containing complete, step-by-step debugging inference content; sequentially inputting the step-by-step inference content in each candidate debugging path into the trained process reward model to obtain the correctness confidence for each step of inference; comprehensively calculating the correctness confidence of all steps in each candidate debugging path, selecting the candidate debugging path with the best calculation result as the final debugging scheme for output, the final debugging scheme containing a traceable complete inference process and corresponding register-transfer level design repair code.

[0018] In one possible implementation, the comprehensive calculation of the correctness confidence of all steps includes calculating the average confidence of all steps in the candidate debug path.

[0019] In one possible implementation, the debugging inference chain includes textual analysis, inference, and localization steps of signal dependencies, timing behavior, concurrent execution logic, and error propagation paths in the hardware design.

[0020] In a second aspect, the present invention provides an inference enhancement system for automated RTL debugging, comprising a multi-agent model of a generative model and a process reward model, for performing the method described in any of the first aspects above.

[0021] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: Firstly, this invention decomposes the RTL debugging task into two specialized roles: a generative model and a process reward model, which collaborate to complete the task. The generative model explores multiple debugging paths in parallel, while the process reward model performs fine-grained evaluation of the inference steps of each path, achieving explicit and quantitative supervision of the debugging process. This fundamentally solves the problem of low reliability caused by intermediate inference "illusions" in existing methods. Secondly, it proposes a fully automated step-level annotation data generation pipeline, overcoming the industry challenge of scarce process supervision data. Thirdly, this invention creatively designs a data augmentation process that requires no manual intervention. It generates diverse defects through hybrid error injection, automatically filters samples using functional equivalence verification, and uses LLM to automatically locate and annotate error steps in the error thought chain, thereby constructing the first process reward model training dataset suitable for RTL debugging, laying the data foundation for the realization of process-level supervision. Finally, it designs a progressive specialized training strategy to achieve deep alignment between model capabilities and domain knowledge. To address the specialized requirements of hardware debugging, this invention does not employ general hint engineering or simple fine-tuning. Instead, it designs a multi-stage progressive strategy that includes supervised fine-tuning, direct preference optimization, and specialized training of the process reward model. This strategy enables the generative model not only to learn the correct debugging patterns but also to learn from errors to differentiate between superior and inferior reasoning. Simultaneously, it allows the process reward model to accurately identify specific logical vulnerabilities in the hardware domain. This deep, specialized training ensures the model's superior performance in complex hardware scenarios. Attached Figure Description

[0022] Figure 1 A flowchart illustrating an inference enhancement method for automated RTL debugging provided by the present invention;

[0023] Figure 2 This is a schematic diagram of an inference enhancement method for automated RTL debugging provided by the present invention. Detailed Implementation

[0024] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Furthermore, in the description of the embodiments of this application, the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. The terms "comprising" and "having," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusion.

[0026] In one feasible implementation, the inference enhancement method for automated RTL debugging provided by this invention can be executed independently by an electronic device, such as a terminal or a server. The terminal can include, but is not limited to, mobile phones, computers, smart wearable devices, smart in-vehicle devices, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, big data, and artificial intelligence platforms, etc., without limitation. In another feasible implementation, the inference enhancement method for automated RTL debugging provided by this invention can also be executed collaboratively by multiple electronic devices, such as a terminal and a server. For ease of explanation, this invention will subsequently use execution by an electronic device as an example.

[0027] Based on the above description, the following describes the relevant process of an inference enhancement method for automated RTL debugging provided by the invention.

[0028] In one exemplary embodiment, taking the electronic device performing alone as an example, please refer to... Figure 1 The inference enhancement method for automated RTL debugging provided in this embodiment of the invention may include the following steps:

[0029] Step 1: Construct samples containing register-transfer level design flaws through an automated process, and obtain corresponding debug inference chains containing multi-step inference processes for each sample; verify the correctness of the debug inference chains, and automatically label each step of the debug inference chain that fails verification, so as to form a debug training dataset containing step-level correctness labels.

[0030] The debugging inference chain includes a thought process that can be used to debug register-transfer level design flaws. In one feasible implementation, the debugging inference chain may include textual analysis, inference, and localization steps of signal dependencies, timing behavior, concurrent execution logic, and error propagation paths in the hardware design.

[0031] In one feasible implementation, samples containing register-transfer level design flaws are constructed through an automated process, and a corresponding debug inference chain containing multi-step inference processes is obtained for each sample; during the verification of the correctness of the debug inference chain, the following steps can be performed:

[0032] Step 11: Obtain the functionally correct register-transfer level (RTL) design and its textual specifications. Modify the functionally correct RTL design using a combination of rule-based and intelligent error injection methods to generate diverse RTL design defect samples. Step 12: Input each RTL design defect sample and its corresponding specifications into multiple heterogeneous language models to obtain candidate responses generated by each language model, containing a step-by-step debugging inference process and corresponding repair code. Step 13: Perform functional equivalence verification on the repair code in each candidate response. If the repaired design is functionally equivalent to the original correct design, mark the complete inference chain in the candidate response as a correct inference chain (i.e., correctness verification passed); otherwise, mark it as an incorrect inference chain (i.e., correctness verification failed).

[0033] In one feasible implementation, the following steps can be performed during the automatic annotation of each step in the debugging inference chain that fails verification:

[0034] For a reasoning chain marked as erroneous, an evaluation model is used to evaluate each reasoning step in the chain sequentially to estimate the probability that each reasoning step is the first step to be erroneous. The reasoning step with the highest probability is selected as the hypothetical starting point of the error. The part of the reasoning chain before the starting point of the error is extracted, and the subsequent reasoning content and repair code are regenerated based on this part of the reasoning chain. If the regenerated repair code passes the functional equivalence verification, all reasoning steps before the hypothetical starting point of the error are marked as correct, and the hypothetical starting point of the error and all reasoning steps after it are marked as erroneous.

[0035] Step 2: Use the debug training dataset to perform the first training on the generative model to obtain the mapping ability from design defects and design specifications to correct reasoning chains; perform the second training on the generative model to enhance the ability to distinguish between correct and incorrect reasoning; use the step-level correctness labels in the debug training dataset to train the process reward model to obtain the ability to perform fine-grained evaluation of reasoning steps.

[0036] In one feasible implementation, the first training of the generative model is supervised fine-tuning training, which uses the debug inference chains marked as correct in the debug training dataset as the learning target, so that the generative model learns to generate correct inference patterns that conform to hardware debugging logic.

[0037] In one feasible implementation, the second training of the generative model is direct preference optimization training, which uses preference data pairs consisting of correct and incorrect inference chains to train the generative model to form a probability distribution difference between generating correct and incorrect inference chains, thereby suppressing the generation of outputs that do not conform to hardware logic during the inference process.

[0038] In one feasible implementation, the training process reward model includes: taking register transfer level design specifications, design defects, and a debug inference chain containing step-level correctness labels from the debug training dataset as input, and the training process reward model characterizes the output of each step in the inference chain as a confidence value of the probability of correctness.

[0039] Step 3: Based on the design defects and design specifications to be addressed, drive the trained generative model to generate multiple candidate debugging paths containing inference steps in parallel; call the trained process reward model to evaluate the correctness of each inference step in each candidate debugging path and obtain the confidence evaluation of each step; based on the confidence evaluation of all steps in each candidate debugging path, select the one with the best overall inference quality from the multiple candidate debugging paths as the final debugging result and output it.

[0040] In one feasible implementation, the following steps can be taken: The generative model, operating in sampling mode, generates multiple candidate debugging paths for the same register-transfer level design defect and specification. Each candidate debugging path contains complete, step-by-step debugging reasoning. The step-by-step reasoning from each candidate debugging path is sequentially input into the trained process reward model to obtain the correctness confidence for each step of the reasoning. The correctness confidence of all steps in each candidate debugging path is comprehensively calculated, and the candidate debugging path with the optimal calculation result is selected as the final debugging scheme and output. The final debugging scheme includes a traceable complete reasoning process and corresponding register-transfer level design repair code.

[0041] In one feasible implementation, the comprehensive calculation of the correctness confidence of all steps includes calculating the average confidence of all steps in the candidate debug path.

[0042] This invention constructs a debugging training dataset containing step-level correctness labels through automatic annotation. This dataset is used to train both a generative model and a process reward model. The generative model acquires the ability to map design flaws and specifications to correct debugging inference chains and to differentiate inference methods, while the process reward model gains the ability to accurately evaluate the correctness of single-step inference. During the inference phase, the generative model explores multiple candidate debugging paths in parallel, while the process reward model scores the confidence of each step on each path. Based on this, the system selects the path with the best overall inference quality as the final output. This achieves explicit and quantitative supervision of the debugging process, fundamentally solving the problem of low reliability caused by intermediate inference "illusions" in existing methods.

[0043] In the automatic annotation process, this invention adopts a combination of rule-based error injection and intelligent error injection to generate diverse register-transfer level design defect samples, thereby improving data diversity and further enhancing the training effect of the generative model and process reward model trained based on the corresponding data.

[0044] In the model training process, this invention uses supervised fine-tuning of the generative model with correct inference chain data to master the correct debugging inference pattern; it then retrains using preference data. This strategy enables the generative model not only to learn the correct debugging pattern but also to learn from errors to distinguish between superior and inferior inference, thus improving the model training effect. For the process reward model, this invention uses step-level labeled data to train it to perform binary classification on each step in the debugging inference chain, outputting a confidence value representing the probability of the step being correct. This allows the process reward model to accurately master the ability to identify specific logical vulnerabilities in the hardware domain.

[0045] During actual debugging, for design flaws and specifications in the input, the trained generative model generates multiple candidate debugging paths in parallel under sampling mode. Each path contains complete, step-by-step debugging inference content and repair code. The process reward model evaluates each step of the inference in each path and gives a correctness confidence score. The confidence scores of all steps in each path are comprehensively calculated, and the path with the highest comprehensive score is output as the final debugging result. This enables explicit supervision of intermediate inference, ensuring that the debugging process is not only correct in terms of results but also logically reliable and traceable, greatly improving the debugging success rate in complex RTL scenarios.

[0046] In yet another exemplary embodiment, taking the electronic device performing alone as an example, please refer to... Figure 2 The inference enhancement method for automated RTL debugging provided in this embodiment of the invention may include the following steps:

[0047] Step S1: Diverse defect injection and thought chain generation.

[0048] The goal of this phase is to create a large number and diverse sample "defective design-specification-thought chain-repair" quadruples. The process begins with a dataset containing a functionally correct RTL design (e.g., an RTL design implemented in Verilog code) and its textual specification. To simulate real-world error scenarios, this invention employs two complementary error injection strategies: (a) Rule-based error injection: Automated modification of the originally correct RTL design using predefined syntactic and semantic error templates (e.g., changing the operator + to -, altering signal bit width, incorrect connection ports, adjusting sensitivity list edges). This method can quickly and cost-effectively generate a large number of defect variants known in error type and location. (b) LLM-based error injection: To further enhance the complexity and realism of the defects, multiple high-performance large-scale language models (which can be selected as needed, and this invention does not impose restrictions) are prompted to introduce a functional error by modifying a single line of code. This method generates more challenging defects that better reflect human design error patterns. All generated defective designs must undergo syntax checking using standard compilers such as Icarus Verilog to filter out uncompileable samples, ensuring that subsequent debugging tasks focus on functional issues. Subsequently, for each "defective design-specification" pair that passes the inspection, multiple heterogeneous large language models are invoked in parallel (heterogeneous large language models can be selected according to needs, and this invention does not impose any restrictions), requiring them to generate repaired code and output a complete, step-by-step debugging process (i.e., the CoT) for the defective RTL design. Optionally, it can be required to output in a structured format (e.g., placed in...). <think>(Within the tag) Output CoT, placing the final fix code in <answer>Within the tags. This step aims to identify multiple potential debugging paths and solutions for the same RTL debugging problem.

[0049] Step S2: Mind chain verification and dataset organization.

[0050] The goal of this stage is to verify the correctness of the generated thought chains and categorize them into different training datasets. Due to the subjectivity of natural language thought chains, directly evaluating their correctness is not feasible. Therefore, this invention uses the functional equivalence of RTL design as the verification criterion: simulation tools are used to verify the repair design generated by LLM. Only when the function of the repair design is completely equivalent to the original correct design is the corresponding entire thought chain judged as correct; otherwise, it is judged as incorrect. Based on this verification result, two core datasets are constructed: (c) Supervised fine-tuning dataset: containing all verified correct samples. The input of each sample is "specification" and "defective design", and the output is the corresponding "correct thought chain" and "correct repair code". This dataset is used to train the generative model to learn standard, correct debugging reasoning patterns. (d) Direct preference optimization dataset: for the same RTL debugging problem, "correct thought chains" are paired with "incorrect thought chains" to form <correct, incorrect> preference pairs. This dataset is used in subsequent training to teach the model to distinguish between high-quality and low-quality reasoning, thereby suppressing its tendency to generate illusions.

[0051] Step S3: Automated annotation of incorrect steps in the thought chain.

[0052] This is a crucial step in implementing process-level supervision, aiming to label each step in an "erroneous" thought chain as either "correct" or "erroneous." The specific process is as follows: (e) Error localization: For an erroneous thought chain (i.e., an erroneous inference chain), first, the advanced large model is invoked to examine each inference step, assessing the likelihood that the step is the first erroneous step and assigning a confidence score. (f) Hypothesis verification and labeling: The step with the highest confidence score is selected as the erroneous starting point of the hypothesis. All content before this step is retained, prompting the large model to regenerate subsequent inference chains and corrective code from this step. (g) Labeling confirmation: If the newly generated corrective code passes the functional equivalence verification, the original hypothesis is proven to be true. At this point, all steps before the erroneous step are labeled "correct," and this step and all subsequent steps are labeled "erroneous." If the verification fails, the next step with high confidence is selected, and the process is repeated.

[0053] In a more specific and feasible implementation, the electronic device can invoke an LLM (Logical Reasoning Model) to perform process-level correctness annotation on erroneous thought chains. Specifically, for any erroneous thought chain, the electronic device can input the erroneous thought chain and its corresponding correct design (i.e., defect-free hardware code) into the LLM, and instruct the LLM to attempt to locate the first erroneous step multiple times. Specifically, the LLM can be prompted to score each step in the thought chain in each attempt (e.g., a score range of 0 to 1, 0-10, etc., not limited in this invention), indicating the probability that the step is the first erroneous step. The electronic device selects the step with the highest score as a candidate erroneous starting point and extracts all steps before that step as a correct prefix. Subsequently, the electronic device invokes the LLM again, using this prefix as context, prompting the LLM to regenerate subsequent reasoning steps and the corrected design code based on this prefix. Next, the electronic device invokes a formal equivalence verification tool (e.g., VC Formal) to perform a functional equivalence check between the newly generated design and the golden design (i.e., the correct design). If the two are equivalent, all steps in the prefix are considered correct, while the candidate step and all subsequent steps are considered incorrect. The electronic device then marks the prefix step as "correct" and the candidate step and subsequent steps as "incorrect," ending the labeling process. If they are not equivalent, it indicates that the guess about the first incorrect step was inaccurate. The electronic device retains the original incorrect inference chain and proceeds to the next attempt, i.e., it re-calls the LLM to score the steps and selects another candidate step, repeating the prefix generation and verification process. If a split point that can pass equivalence verification is not found within the maximum number of attempts, the electronic device marks all steps in that inference chain as "unknown," indicating that their correctness cannot be determined.

[0054] Through the above automated process, precise step-level labels can be added to a large number of flawed thought chains. Finally, the correct thought chains, where all steps are marked as "correct," constitute the training dataset for the process reward model. Each sample in this dataset includes: task context (specifications, flawed design, complete thought chain) and the corresponding sequence of step-level correctness labels.

[0055] Step S4: Multi-stage training strategy

[0056] To fully utilize the enhanced dataset and incorporate this knowledge into the model parameters, this invention employs a progressive multi-stage strategy, separately training the generative model and the process reward model to ensure synergy between the two in debugging tasks. The training of the generative model consists of two stages:

[0057] Phase 1: Supervised Fine-Tuning (SFT). In this phase, the base model is fine-tuned using a supervised fine-tuning dataset. This dataset contains a large amount of validated correct thought processes. Specifically, a functionally correct hardware design and its specifications are taken as input, and the corresponding correct thought processes and fixes are taken as output, providing supervision signals during training. During training, the model learns these inference steps and fixes using an autoregressive approach and is optimized using a cross-entropy loss function, thereby mastering the inference process and results of RTL debugging and enabling the model to possess basic RTL debugging capabilities.

[0058] Phase Two: Direct Preference Optimization (DPO). While the SFT phase teaches the model the correct debugging patterns, it may still produce incorrect reasoning in complex scenarios because it has only seen correct debugging methods and cannot distinguish between incorrect ones. To further improve the reliability of the model's debugging process, Direct Preference Optimization (DPO) is introduced. This phase uses a Direct Preference Optimization dataset, where each sample contains both correct (preferred) and incorrect (rejected) thought chains for the same problem. During training, the model learns to adjust its parameters so that the probability of generating correct thought chains is significantly higher than the probability of generating incorrect thought chains. Through this contrastive learning, the model internalizes the difference between high-quality and low-quality reasoning, reduces the tendency to generate illusions in ambiguous or complex situations, and improves the robustness of the output.

[0059] In one feasible implementation, for the same design problem, if it is determined that there is at least one correct reasoning chain and one incorrect reasoning chain, training samples for DPO can be constructed based on the correct and incorrect reasoning chains. Specifically, each training sample contains three parts: the problem itself (including design specifications and defect codes), a correct reasoning chain and its fix as the preferred answer, and an incorrect reasoning chain and its fix as the rejected answer. This set of samples constitutes the DPO training set. An exemplary training sample is as follows:

[0060] Question (query):

[0061] Design specifications: A 4-bit counter, triggered by the rising edge of the clock, active high for asynchronous reset, and automatically reset to zero after counting to 15.

[0062] Defective code (Verilog snippet) (Defect: Lack of logic to reset the count to zero when it reaches 15, causing the count to exceed 15):

[0063] always @(posedge clk or posedge reset) begin

[0064] if (reset)

[0065] count <= 0;

[0066] else

[0067] count <= count + 1;

[0068] End

[0069] The preferred answer, i.e., the correct reasoning chain and its repair:

[0070] Step 11: The reset logic is correct, and the count is cleared to zero during reset.

[0071] Step 12: Observe the increment logic: count is incremented by 1 on each clock edge, but there is no upper limit judgment.

[0072] Step 13: According to the specification, the count should be reset to zero after reaching 15. Therefore, the next value needs to be set to 0 when count equals 15.

[0073] Step 14: Modify the else branch: if (count == 4'd15) count <= 0; else count <= count + 1; (Corrected)

[0074] Rejection, i.e., a flawed chain of reasoning and its correction:

[0075] Step 21: Reset logic is correct.

[0076] Step 22: Suspect that the clock edge is used incorrectly. The specification requires the rising edge, but the design has already used posedge, so it may not be a problem.

[0077] Step 23: Try changing the clock to a falling edge (negedge clock) to solve the problem.

[0078] Step 24: Change the always block to @(negedge clk or posedge reset) (to fix bugs)

[0079] In one feasible implementation, the generative model trained by SFT is retrained using the constructed DPO training set. The retraining adopts the DPO training method, which enables the model to distinguish between correct and incorrect reasoning and debugging methods. The loss function is as follows:

[0080]

[0081] In the formula, This indicates an input query, which consists of a hardware design specification description document and defect codes. It is the preferred answer, consisting of the correct reasoning chain and the correct repair; It is a rejection of the answer, composed of flawed reasoning and incorrect fixes. (Model) The training objective is to train on the preference dataset. Above, by continuously adjusting the model parameters This allows for querying given input. Maximize output probability And reduce output probability This preference is expressed as the difference between two probabilities. To achieve this. At the same time, to ensure model training converges and prevent misfitting caused by outliers, parameters are used. To control the sensitivity of preferences, and use the sigmoid function. The preference loss during training is mapped to the interval (0,1). Finally, the optimal model parameters are fine-tuned. Should be in the dataset Minimize the loss function At the same time, achieve the maximum mathematical expectation. .

[0082] The Process Reward Model (PRM) is specifically designed to evaluate the correctness of each step in a thought chain. During training, it uses a dataset with step-level annotations, where each sample contains the complete thought chain and a correct / incorrect label for each step. PRM is also trained using SFT. During training, the model takes an RTL design specification description, incorrect RTL designs, and the labeled thought chain as input. After feature extraction by the encoder, it performs a binary classification prediction for each step in the thought chain, outputting whether it considers that step correct or incorrect. Through training on a large amount of data, PRM learns to identify logical fallacies, domain knowledge errors, and inconsistencies in reasoning steps, gaining the ability to accurately evaluate the quality of reasoning.

[0083] The training data for the process reward model consists of step-level labeled data. The dataset comprises two parts: a "query" and an "answer," formatted as follows: (Query) A design specification description, faulty hardware design code, and the large model inference process during debugging of this code. (Answer) A list where each element is either "correct" or "incorrect," representing whether each inference step in the large model inference process was correct. For example:

[0084] (Query):

[0085] Design Specifications: Implement a 4-bit counter that increments on the rising edge of the clock. The counter is reset to zero when the reset signal rst is high; the counter remains unchanged when the enable signal en is low.

[0086] Hardware design code:

[0087] module counter(

[0088] input clk,

[0089] input rst,

[0090] input en,

[0091] output reg [3:0] count );

[0093] always @(posedge clk) begin

[0094] if (rst)

[0095] count <= 4'b0001; / / bug: should be 0000

[0096] else if (en)

[0097] count <= count + 1;

[0098] end

[0099] Endmodule

[0100] Large model inference process:

[0101] Step 1: This module is a synchronous 4-bit counter that increments. Step 2: When the reset signal rst is high, the code sets count to 0001. Step 3: According to specifications, the counter should be cleared to zero upon reset; therefore, the reset assignment is incorrect. Step 4: The error lies in the reset value setting not conforming to design specifications; it should be 0000 instead of 0001. Step 5: Solution: Change the assignment in the reset branch from count <= 4'b0001 to count <= 4'b0000.

[0102] (Answer): [Correct, Correct, Correct, Correct, Correct]

[0103] PRM Training Process: In this stage, the process reward model is also trained using supervised fine-tuning (SFT) to enable it to judge the correctness of each step in the inference process. Specifically, the training data consists of design specification descriptions, faulty hardware design code, and the corresponding large model debugging inference process, which is further organized into step-level labeled data, that is, each inference step is labeled as "correct" or "incorrect".

[0104] During training, this invention takes the design specifications, RTL code, and the current inference step and its previous historical steps as input, and uses the correctness label corresponding to that step as a supervision signal. By learning these labeled inference processes, the model gradually learns how to determine whether each inference step conforms to the design specifications and whether it helps to locate and correct errors. The loss function is as follows:

[0105]

[0106] The input query consists of a hardware design specification description file, hardware code containing design flaws, and the reasoning process of the large model when fixing the flaws; It is the output target of the model, which is a variable of length . A list, consisting of binary labels "correct" and "incorrect", symbolically represented as... . The PRM model is used in a given query and the location of reasoning steps At that time, predict the first Each reasoning step is tagged as The probability. The training goal of the model is to continuously update the model. The parameters enable step-level annotation of the dataset. Minimize cross-entropy loss To minimize the loss function At the same time, obtain the maximum mathematical expectation. Through this training, the PRM model can accurately identify the location of errors in the inference steps.

[0107] Step S5: Collaborative Reasoning Mechanism

[0108] During the RTL debugging phase, the trained generative model and process reward model can work together according to the generation-evaluation-selection collaborative process, with the following specific steps:

[0109] S51, Parallel Generation. When the electronic device receives a new debugging task (containing design specifications and error codes), it can invoke a trained generative model to run in sampling mode, generating N candidate debugging paths in parallel based on the same input (N=20 in this invention). Each path contains a complete thought chain and a repair design proposal. The sampling temperature can be set according to requirements; this invention does not impose limitations. For example, to balance output quality and diversity, the sampling temperature can be set to 0.6. This step fully utilizes the creativity of the generative model, exploring various possible error localization methods and repair strategies, providing rich candidate solutions for subsequent evaluation.

[0110] As we know, during the inference phase, given a hardware design containing defects and its specifications, the generative model is required to find and resolve these defects. The model generates multiple candidate solutions simultaneously with a degree of randomness; this mode is called sampling mode. This mode is achieved by setting a hyperparameter called "temperature." Higher temperatures make the model more likely to produce diverse outputs, while lower temperatures result in more deterministic outputs but may lack exploratory capabilities. The generation of each candidate solution is independent; the model performs twenty samplings in parallel, generating 20 solutions that may be identical or different. The "temperature" and "number of samplings" can be customized by the user. Each sampling starts from the beginning and outputs the inference chain and the final fix code step by step. The inference chain is required to be organized according to a specified format: "step 1: xxx; step 2: xxx ...", with clear step numbering for easy parsing. After generation, the twenty candidate solutions are fed into a process reward model for scoring. The reward model judges the correctness of each step of the reasoning in each solution and then calculates the average accuracy of the entire inference chain as the final score for that solution. The solution with the highest score is selected as the final output.

[0111] S52, Fine-grained process evaluation. All generated candidate thought chains are sequentially fed into a trained process reward model for evaluation. For each thought chain, the process reward model independently evaluates each step, outputting a confidence score for that step as correct using a softmax function. These scores reflect the logical rigor and domain knowledge compliance of each step of the reasoning. By evaluating all steps of the complete thought chain, a detailed reliability analysis of the reasoning path is obtained.

[0112] S53, Reasoning Reliability Scoring and Selection. Based on the fine-grained evaluation results, the system calculates a comprehensive reliability score for each candidate path. Specifically, the average confidence score of all steps in the path's thought process chain is taken as the path's total score. This scoring mechanism ensures a quantitative assessment of the overall quality of the reasoning process, considering the reliability of each step while avoiding evaluation bias caused by a single abnormal step. After scoring all paths, the system performs optimal path selection, choosing the path with the highest comprehensive score as the final output solution.

[0113] This invention, for the first time, decomposes the RTL debugging task into two specialized roles: a generative model and a process reward model, which collaborate to complete the task. The generative model explores multiple debugging paths in parallel, while the process reward model performs fine-grained evaluation of the inference steps of each path, achieving explicit and quantitative supervision of the debugging process. This fundamentally solves the reliability problem caused by the "illusion" of intermediate inference in existing methods. A fully automated step-level annotation data generation pipeline is proposed, overcoming the industry challenge of scarce process supervision data. This invention creatively designs a data augmentation process that requires no manual intervention. It generates diverse defects through hybrid error injection, automatically filters samples using functional equivalence verification, and uses LLM to automatically locate and annotate error steps in the error thought chain. This constructs the first process reward model training dataset suitable for RTL debugging, laying the data foundation for process-level supervision. A progressive specialized training strategy is designed to achieve deep alignment between model capabilities and domain knowledge. Addressing the specialized requirements of hardware debugging, this invention does not adopt general hint engineering or simple fine-tuning, but instead designs a multi-stage progressive strategy that includes supervised fine-tuning, direct preference optimization, and specialized training of the process reward model. This strategy enables the generative model not only to learn the correct debugging patterns but also to learn from errors to differentiate between superior and inferior reasoning; simultaneously, it allows the process reward model to accurately identify hardware-specific logical vulnerabilities. This deep specialization training ensures the model's superior performance in complex hardware scenarios.

[0114] In one exemplary embodiment, the present invention also provides an inference enhancement system for automated RTL debugging, comprising a multi-agent model of a generative model and a process reward model, for performing the method described in any one of the claims.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0116] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0117] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0118] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.< / answer> < / think>

Claims

1. A method for enhancing inference for automated RTL debugging, characterized in that, Includes the following steps: Step 1: Construct samples containing register-transfer level design flaws through an automated process, and obtain a corresponding debug inference chain containing multi-step inference processes for each sample; The correctness of the debugging inference chain is verified, and each step of the debugging inference chain that fails verification is automatically labeled to form a debugging training dataset containing step-level correctness labels. Step 2: Use the debug training dataset to perform the first training on the generative model to obtain the mapping ability from design flaws and design specifications to the correct inference chain; The generative model is trained a second time to enhance its ability to distinguish between correct and incorrect reasoning. By training a process reward model using step-level correctness labels from a debug training dataset, we can gain the ability to perform fine-grained evaluation of inference steps. Step 3: Based on the design defects and design specifications to be addressed, drive the trained generative model to generate multiple candidate debugging paths containing inference steps in parallel; The trained process reward model is invoked to evaluate the correctness of each step of reasoning in each candidate debugging path and obtain the confidence evaluation of each step. Based on the confidence evaluation of all steps in each candidate debugging path, the path with the best overall reasoning quality is selected from multiple candidate debugging paths as the final debugging result and output.

2. The inference enhancement method for automated RTL debugging according to claim 1, characterized in that, Step 1 includes the following steps: Step 11: Obtain the functionally correct register-transfer stage design and its textual specifications. Use a combination of rule-based error injection and intelligent error injection to modify the functionally correct register-transfer stage design and generate diverse register-transfer stage design defect samples. Step 12: Input the design defect samples of each register transfer stage and their corresponding specifications into multiple heterogeneous language models to obtain candidate responses generated by each language model, which include step-by-step debugging reasoning processes and corresponding fix codes; Step 13: Perform functional equivalence verification on the repair code in each candidate response. If the repaired design function is equivalent to the original correct design, mark the complete inference chain in the candidate response as the correct inference chain; otherwise, mark it as the incorrect inference chain.

3. The inference enhancement method for automated RTL debugging according to any one of claims 1 or 2, characterized in that, Automatically annotate each step of the debugging inference chain that fails verification, including: For a reasoning chain marked as erroneous, an evaluation model is used to evaluate each reasoning step in the chain sequentially to estimate the probability that each reasoning step is the first step to be erroneous. The reasoning step with the highest probability is selected as the hypothetical starting point of the error. The part of the reasoning chain before the starting point of the error is extracted, and the subsequent reasoning content and repair code are regenerated based on this part of the reasoning chain. If the regenerated repair code passes the functional equivalence verification, all reasoning steps before the hypothetical starting point of the error are marked as correct, and the hypothetical starting point of the error and all reasoning steps after it are marked as erroneous.

4. The inference enhancement method for automated RTL debugging according to claim 1, characterized in that, In step 2, the first training of the generative model is supervised fine-tuning training. The training uses the debug inference chains marked as correct in the debug training dataset as the learning target, so that the generative model learns to generate correct inference patterns that conform to hardware debugging logic.

5. A method for enhancing inference for automated RTL debugging according to any one of claims 1 or 4, characterized in that, The second training of the generative model is direct preference optimization training. The training uses preference data pairs consisting of correct and incorrect inference chains. The training model forms a probability distribution difference between generating correct and incorrect inference chains, thereby suppressing the generation of outputs that do not conform to the hardware logic during the inference process.

6. The inference enhancement method for automated RTL debugging according to claim 1, characterized in that, In step 2, the training process reward model includes: The debug training dataset contains register-transfer level design specifications, design defects, and debug inference chains with step-level correctness labels as input. During training, the reward model represents the output of each step in the inference chain as a confidence value of the probability of correctness.

7. The inference enhancement method for automated RTL debugging according to claim 1, characterized in that, Step 3 includes: In sampling mode, the control generation model generates multiple candidate debugging paths for the same register-transfer level design defect and design specification. Each candidate debugging path contains complete, step-by-step debugging reasoning. The step-by-step reasoning from each candidate debugging path is sequentially input into the trained process reward model to obtain the correctness confidence for each step of reasoning. The correctness confidence of all steps in each candidate debugging path is comprehensively calculated, and the candidate debugging path with the best calculation result is selected as the final debugging scheme and output. The final debugging scheme contains a traceable complete reasoning process and corresponding register-transfer level design repair code.

8. The inference enhancement method for automated RTL debugging according to claim 7, characterized in that, The overall calculation of the correctness confidence of all steps includes calculating the average confidence of all steps in the candidate debugging path.

9. The inference enhancement method for automated RTL debugging according to claim 1, characterized in that, The debugging inference chain includes textual analysis, inference, and localization steps of signal dependencies, timing behavior, concurrent execution logic, and error propagation paths in the hardware design.

10. An inference enhancement system for automated RTL debugging, characterized in that, A multi-agent model including a generative model and a process reward model, for performing the method described in any one of claims 1-9.

Images