Structure-aware two-stage reinforcement learning SAT solving method and related device

Abstract

The application discloses a structure-aware two-stage reinforcement learning SAT solving method and related device, and belongs to the field of chip design. In the variable initialization stage, chip signal polarity and logic consistency information are introduced to construct a structure-aware initial feature vector, breaking through the limitation of traditional graph modeling which can only represent local logical relationship. A two-stage reinforcement learning framework composed of a main path and a sampling path is designed. The sampling path is used to explore the chip logic solution space, and the main path is used to stabilize the optimal verification strategy, effectively alleviating problems such as unstable training convergence and single strategy. Based on the reward-guided feature update mechanism, the dynamic evolution of the chip signal feature is explicitly bound with the reinforcement feedback, improving the solver interpretability and narrowing the gap between neural inference and chip logic symbolic inference. Based on the GNN multi-round message passing result, signal assignment decisions are executed and verification trajectories are continuously optimized, without relying on artificial heuristics to obtain efficient chip design verification capability.

Description

Technical Field

[0001] This invention relates to the field of chip design, and more particularly to a structure-aware two-stage reinforcement learning SAT solution method and related apparatus. Background Technology

[0002] The Boolean satisfiability problem (SAT) is a technique for determining the satisfiability of Boolean logic formulas. It determines whether there exists a set of Boolean variable values ​​that makes the overall logic formula true. In the context of the rapid development of artificial intelligence and integrated circuits, the Boolean satisfiability problem has become a crucial fundamental issue in chip design verification, software verification, planning, and constraint solving. Specifically, in chip design verification scenarios, Boolean variables correspond to the level states of signals within the chip. SAT is used to determine whether these signal levels can satisfy the logic formula derived from the chip's functional and structural constraints. Therefore, the efficiency of SAT solving directly affects the chip verification cycle and verification quality.

[0003] Conjunctive Normal Form (CNF) is the standard representation used by SAT in chip verification. It consists of multiple logical constraints (clauses) connected by a logical AND, and each clause is composed of one or more signal level states (literals) connected by a logical OR. A literal represents a specific signal level value, including high and low levels. When all constraints in the CNF are satisfied, the chip logic is considered to be in a satisfyable state, meaning the corresponding signal level combination meets the chip design requirements.

[0004] In chip design verification, as integration density increases and logic structures become increasingly complex, functional verification requires determining whether logic circuits composed of billions of transistors meet preset requirements. The core of this process is to transform logic constraints into CNF formulas and solve them using SAT to determine if a signal configuration satisfies all constraints. However, traditional Conflict-Driven Clause Learning (CDCL) solvers, which rely on heuristic rules, are prone to efficiency degradation, weak transferability, and difficulty in adapting to different module structures (such as CPU cores, memory controllers, and interface circuits) when handling CNF in very large-scale chips. In recent years, neural networks, especially graph neural networks (GNNs), have been used to learn variable selection strategies from data to overcome the performance bottlenecks of artificial heuristics. However, they still face many challenges in chip verification scenarios: CNFs are typically modeled as variable-clause bipartite graphs, which have limited expressive power and struggle to capture high-order logical connections across modules; reinforcement learning rewards are extremely sparse, only generating feedback when a satisfactory solution is found, leading to unstable training and slow optimization; policy exploration paths are singular, easily getting trapped in local optima, resulting in pseudo-verification; the iterative updates of variable feature vectors are semantically opaque, hindering interpretation and control, and making it difficult to meet verification traceability requirements. Against this backdrop, there is an urgent need for a new SAT solution that can fully utilize the deep structure of chip CNFs, obtain stable and effective reinforcement signals during training, and improve chip design verification efficiency, robustness, and generalization ability. Summary of the Invention

[0005] The purpose of this invention is to overcome the problems of the prior art and provide a structure-aware two-stage reinforcement learning SAT solution method and related apparatus.

[0006] The objective of this invention is achieved through the following technical solution: a structure-aware two-stage reinforcement learning method for solving the SAT, comprising the following steps:

[0007] Obtain the conjunctive normal form formula for the chip design verification task, and construct a literal-clause bipartite graph based on the conjunctive normal form formula; the Boolean variables of the conjunctive normal form formula correspond to chip signal variables, and the clauses correspond to chip logic constraint clauses; each chip signal variable in the literal-clause bipartite graph is split into positive literal nodes and negative literal nodes, and each chip logic constraint clause is treated as an independent node. Based on whether the literal appears in the chip logic constraint clause, a connection relationship is established between the corresponding literal node and the clause node to form an adjacency matrix.

[0008] The polarity score and consistency score of each chip signal variable are calculated based on the adjacency matrix. An initial feature vector of the chip signal variable is generated based on the polarity score and consistency score. An initial feature vector of the chip logic constraint clause is generated based on the initial feature vector of the chip signal variable and the adjacency matrix. The polarity score is used to indicate whether the chip signal variable tends to be high or low level in the logic constraint, and the consistency score is used to indicate the degree of tendency to be high or low level.

[0009] Perform multiple rounds of iterative solution, each round of iteration including the following sub-steps:

[0010] Based on the current chip signal variable assignments, the satisfaction status of each chip logic constraint clause is determined. For chip logic constraint clauses that are not satisfied, noise perturbation is injected into their feature vectors. Then, the feature vectors of the clauses are aggregated to literal nodes using a literal-clause bipartite graph, yielding the aggregated message. The aggregated message, the current feature vector of the chip signal variable, and random noise are fused to generate a query input vector. This query input vector is then input into a variable query network to generate a variable query vector.

[0011] Based on the current chip logic constraint clause loss and the gradient information contained in the variable query vector, the feature vector of the chip logic constraint clause is dynamically updated; the feature vector of the chip signal variable is dynamically updated using the sampling reward and main path reward generated in the previous iteration.

[0012] Parallel execution of the sampling exploration path and the main path utilization path: In the sampling exploration path, a policy distribution is constructed based on the query vector, and sampling assignments are obtained by random sampling according to the policy distribution. The satisfaction status of chip logic constraint clauses is determined based on the sampling assignments, and a sampling exploration reward is generated. The policy loss of the sampling exploration path stage is calculated based on the selection probability of the sampling assignments in the policy distribution and the sampling exploration reward. In the main path utilization path, the estimated loss caused by the assignment of different chip signal variables is evaluated based on the query vector, and the assignment with the minimum loss is selected as the main path assignment. The satisfaction status of chip logic constraint clauses is determined based on the main path assignments, and a main path utilization reward is generated. The policy loss of the main path stage is calculated based on the selection probability of the main path assignments in the policy distribution and the main path utilization reward.

[0013] The main loss is calculated based on the constraint loss evaluated in the history of multiple iterations. The parameters of the solver are optimized in conjunction with the main loss, the sampling exploration path stage strategy loss, and the main path stage strategy loss. The iteration is terminated when it is detected that all chip logic constraint clauses are satisfied.

[0014] After the iteration is terminated, the output chip signal variable assignment results and the chip logic constraint satisfaction determination conclusion are presented.

[0015] In one example, after constructing the literal-clause bipartite graph, the following is also included:

[0016] Based on the connection relationship between text nodes and clause nodes, a clause relationship graph and a variable relationship graph are constructed; the edges of the clause relationship graph connect two chip logic constraint clause nodes that share at least one text, and the edges of the variable relationship graph connect two chip signal variable nodes that appear in at least one identical chip logic constraint clause.

[0017] In one example, generating the initial feature vector of the chip logic constraint clause based on the initial feature vector of the chip signal variables and the adjacency matrix includes:

[0018] The initial feature vector of the chip signal variable is expanded into positive and negative text features corresponding to the chip signal variable. The text features associated with each chip logic constraint clause are aggregated through the adjacency matrix, thereby generating the initial feature vector of the chip logic constraint clause.

[0019] In one example, fusing the message aggregated to the text node, the current feature vector of the chip signal variable, and random noise to generate the query input vector includes:

[0020] The messages aggregated to the text nodes are split into positive text messages and negative text messages according to the text polarity. The positive text messages, negative text messages, the current feature vector of the chip signal variables, and random noise are concatenated to obtain the query input vector.

[0021] In one example, the dynamic updating of the feature vector of the chip signal variable using the sampling reward and main path reward generated in the previous iteration includes:

[0022] The sampling reward is concatenated with the current feature vector of the chip signal variable and input into the first update gating network to generate a feature offset. The current feature vector of the chip signal variable is initially updated based on the first feature offset to obtain the initial updated feature vector.

[0023] The initial updated feature vector, positive and negative text loss signals, and gradient information related to chip signal variables are concatenated and input into the update gate network for feature transformation to generate directional feature vectors. These directional feature vectors are then normalized to obtain the directional vector. The directional adjustment amount is generated by combining the main path reward generated in the previous iteration with the dynamic decay coefficient. The directional adjustment amount is then used to optimize the directional feature vector along the directional vector. Finally, the optimization result is combined with the current feature vector through residual connections to obtain the updated chip signal variable feature vector.

[0024] In one example, the step of determining the satisfaction status of chip logic constraint clauses based on sampled assignments and generating sampling exploration rewards includes:

[0025] The difference between the number of unsatisfied chip logic constraint clauses before and after the sampling assignment is calculated to obtain the graph-level reward of the sampling path;

[0026] Identify chip signal variables whose values ​​change during sampling and assignment;

[0027] The graph-level reward of the sampling path is allocated to the chip signal variables whose values ​​change during the sampling assignment, thereby generating a sampling exploration reward;

[0028] Based on the main path assignment, determine the satisfaction status of the chip logic constraint clauses, and generate the main path utilization reward, including:

[0029] The graph-level reward for the main path is obtained by calculating the difference between the number of chip logic constraint clauses that are not satisfied before and after the main path is assigned.

[0030] Identify chip signal variables whose values ​​change during the main path assignment;

[0031] The graph-level reward of the main path is allocated to the chip signal variables whose values ​​change during the main path assignment, thereby generating the main path utilization reward.

[0032] In one example, the parameters of the co-optimization solver, based on the main loss, the sampling exploration stage policy loss, and the main path stage policy loss, include:

[0033] The total strategy loss is obtained by combining the strategy loss during the sampling exploration phase and the strategy loss during the main path phase.

[0034] Based on the current training progress, dynamically adjust the weights of the main loss and the total policy loss in gradient updates;

[0035] The gradients of the main loss and the total policy loss after weight updates are merged, and the parameters of the solver are updated using the merged gradients.

[0036] It should be further noted that the technical features corresponding to the above examples can be combined or replaced to form new technical solutions.

[0037] The present invention also includes a computer program product comprising a computer program that, when executed by a processor, implements the steps of the structure-aware two-stage reinforcement learning SAT solving method formed by any or a combination of the above examples.

[0038] The present invention also includes a storage medium storing computer instructions that, when executed, perform the steps of the structure-aware two-stage reinforcement learning SAT solving method formed by any or more of the above examples.

[0039] The present invention also includes a terminal comprising a memory and a processor, the memory storing computer instructions executable on the processor, wherein the processor, when executing the computer instructions, performs the steps of the structure-aware two-stage reinforcement learning SAT solving method formed by any or more of the above examples.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] 1. By constructing a literal-clause bipartite graph to perform fine-grained structure-aware modeling of chip logic constraints, this invention addresses the limitations of traditional variable-clause graphs in terms of expressive power and the difficulty in capturing high-order cross-module relationships. By introducing a dynamic information filtering mechanism based on clause state noise injection and a reward generation mechanism based on satisfied states, it effectively addresses the challenges of extremely sparse rewards and unstable training in reinforcement learning's SAT problem solving. Through a two-stage reinforcement learning mechanism that combines parallel sampling exploration and main path utilization, it balances solution space exploration and optimal policy utilization, avoiding the problems of single policy and easy getting trapped in local optima. Furthermore, by using a reward-guided feature update strategy, the evolution of variable feature vectors is explicitly correlated with reinforcement learning feedback, significantly improving the interpretability and traceability of the solver's decision-making process, meeting the transparency requirements of chip verification. This invention's overall method breaks through the reliance on manual heuristics, achieving end-to-end efficient solving of the chip design verification SAT problem, and improving chip design verification efficiency, robustness, and generalization ability.

[0042] 2. The clause relationship diagram explicitly depicts the coupling relationship between different logical constraints, which is conducive to cross-constraint loss or information propagation in subsequent steps and improves the synergy of decision-making; the variable relationship diagram describes the logical co-occurrence relationship between signal variables, providing a structured basis for subsequent implementation of reward allocation that considers the mutual influence between variables, making optimization more in line with the actual logical topology of chip design.

[0043] 3. By calculating the change in the number of unsatisfied clauses as the graph-level reward for each step, the originally sparse final success reward is transformed into dense feedback, effectively solving the problems of sparse training signals and slow optimization. The reward is accurately allocated to the variable whose actual value changes, which can accurately learn the contribution of variable assignment adjustment to improving the validation state, thereby more effectively guiding the direction of policy optimization. Attached Figure Description

[0044] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The accompanying drawings are provided to provide a further understanding of the present application and constitute a part of the present application. The same reference numerals are used in these drawings to denote the same or similar parts. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application.

[0045] Figure 1 A flowchart illustrating a method provided as an example of the present invention;

[0046] Figure 2 A flowchart of a preferred example of the present invention is provided. Detailed Implementation

[0047] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0049] In one example, such as Figure 1 As shown, a structure-aware two-stage reinforcement learning SAT solution method is applied to a RISC-V processor data path verification scenario. The method includes the following steps:

[0050] S1: Obtain the conjunctive normal form formula for the chip design verification task, and construct a literal-clause bipartite graph based on the conjunctive normal form formula; each chip signal variable in the literal-clause bipartite graph is split into positive literal nodes and negative literal nodes, each chip logic constraint clause is treated as an independent node, and a connection relationship is established between the corresponding literal node and the clause node based on whether the literal appears in the chip logic constraint clause, forming an adjacency matrix.

[0051] Among them, CNF graph structure representation is a way to transform CNF formulas into graph structures to adapt to the modeling requirements of graph neural networks (GNNs). It mainly includes four types: (1) Literal-Clause Graph (LCG): with literals and clauses as nodes, edges represent the appearance of literals in corresponding clauses, reflecting the direct relationship between the two; (2) Variable-Clause Graph (VCG): with variables and clauses as nodes, edges represent the appearance of variables in corresponding clauses, but do not distinguish between the positive and negative polarities of variables; (3) Variable Interaction Graph (VIG): with variables as nodes, edges represent the appearance of two variables in the same clause, which can capture the logical relationship between variables; (4) Literal Interaction Graph (LIG): with literals separated by positive and negative polarities as nodes, edges represent the appearance of two literals in the same clause, which can reflect the cooperative or conflicting relationship between literals. Graph neural networks are neural network models designed specifically for processing graph structure data. They can capture the topological and semantic relationships between nodes through mechanisms such as feature vectors, message passing, and feature updates. In chip design verification, GNN is used to model graph structures represented by CNF, realize the representation learning of signal nodes and constraint nodes, and is the core node feature core technology component of neural SAT solvers.

[0052] Step S1's core is to transform the unstructured CNF formula into a learnable "text-clause" association structure for the GNN, and generate a key matrix to aid information transmission, laying the data foundation for subsequent feature learning and policy optimization. For the given CNF formula (corresponding to the Verilog code logic constraint transformation result of the RISC-V processor data path), its core parameters include the total number of variables. Total number of clauses The variables represent Boolean hardware signals in the processor's data path, such as data bus bit signals, register read / write control signals, and arithmetic logic unit (ALU) operation control signals. The clauses represent constraints in the chip's functional logic, such as data transmission timing constraints, register port mutual exclusion constraints, and ALU operation consistency constraints.

[0053] After determining the number of variables and clauses, the adjacency relationships of the literal-clause bipartite graph are constructed. Unlike traditional variable-clause bipartite graphs that directly use variables as nodes, this invention constructs the literal-clause bipartite graph based on the conjunctive normal form. In the literal-clause bipartite graph, each chip signal variable is explicitly split into positive literal nodes (representing a high-level active state) and negative literal nodes (representing a low-level active state), with each constraint acting as an independent node. Furthermore, depending on whether a literal appears in a clause, connecting edges are established between the corresponding literal node and clause node, forming an adjacency matrix.

[0054] S2: Calculate the polarity score and consistency score of each chip signal variable based on the adjacency matrix, and generate the initial feature vector of the chip signal variable based on the polarity score and consistency score; generate the initial feature vector of the chip logic constraint clause based on the initial feature vector of the chip signal variable and the adjacency matrix.

[0055] In step S2, the frequency of positive and negative literals corresponding to each variable in all constraints is counted based on the adjacency matrix, and two polarity scores and a consistency score are calculated accordingly. The polarity score indicates whether the chip signal variable tends to be high or low in the logic constraints, while the consistency score quantifies the degree of explicitness of the tendency towards high or low levels.

[0056] Preferably, the polarity score and consistency score are standardized and nonlinearly transformed. The features obtained after standardization and nonlinear transformation are then input into a dedicated variable initialization network, outputting a high-dimensional initial feature vector of the chip signal variables that incorporates structural semantics. Further, based on the initial feature vector of the chip signal variables and the adjacency matrix of the literal-clause bipartite graph, the features of all literals belonging to the same clause are aggregated, and the aggregated features are optimized to generate the initial feature vector of the chip logic constraint clause.

[0057] S3: Perform multiple rounds of iterative solution, each round of iteration including the following sub-steps:

[0058] S31: Determine the satisfaction status of each chip logic constraint clause based on the current chip signal variable assignment. Inject noise perturbation into the feature vector of chip logic constraint clauses that are not satisfied. Then, aggregate the feature vector of the clause to the literal node through the literal-clause bipartite graph to obtain the message aggregated to the literal node. Fuse the message aggregated to the literal node, the current feature vector of the chip signal variable, and random noise to generate a query input vector. Input the query input vector into the variable query network to generate a variable query vector.

[0059] In step S31, the satisfaction status of each constraint clause is determined based on the current assignment result. Unsatisfiable constraints are perturbed to weaken their noise impact, while retaining the original features of satisfiable constraints. Then, through a literal-clause bipartite graph, the constraint features are passed to the signal nodes. The literal node messages obtained from the aggregation of signal nodes, the current state features of chip signal variables (such as delay and stability), and the noise introduced for exploration randomness are fused to generate a variable query vector. Based on the query vector, the constraint loss is calculated and the signal gradient is derived in reverse for subsequent signal feature optimization.

[0060] Step S31 completes the state awareness and encoding within the reinforcement learning framework. In the chip-verified reinforcement learning environment, the environment is the logical structure of the CNF formula, and the state is the current constraint satisfaction status. Step S31 encodes this state into an information-rich query vector through dynamic noise injection and structure-aware aggregation, which serves as the basis for subsequent agent (solver) decisions.

[0061] S32: Based on the current chip logic constraint clause loss and the gradient information contained in the variable query vector, dynamically update the feature vector of the chip logic constraint clause; utilize the sampling reward and main path reward generated in the previous iteration to dynamically update the feature vector of the chip signal variable.

[0062] In step S32, during the feature update phase, both the feature vectors of constraints and signals are updated. When updating constraint features, depending on whether the message passing mechanism is enabled, the current constraint state (satisfied / unsatisfied) is integrated with the constraint loss. This is then split into signal loss and a new constraint state via the feature transformation module. After normalizing the new constraint state, it is updated by superimposing it with the original state through residual connections, ensuring that the constraint features evolve reasonably as the verification progresses. Signal feature updates are divided into two phases: In the sampling exploration phase, the normalized sampling reward is combined with the signal features, and an adjustment offset is generated through a gating network. Preliminary optimization results are obtained under the action of a dynamic attenuation coefficient, followed by further integration of signal gradient and positive / negative level loss information for optimization. In the main path utilization phase, the initially optimized signal features are directionally normalized, and a directional adjustment amount is generated based on the main path reward for a second update. The final signal features are obtained through normalization and residual connections, while limiting the gradient propagation scale to avoid gradient explosion or vanishing.

[0063] Step S32 implements the co-evolution of feature representations based on reinforcement learning feedback. It not only utilizes constraint loss to optimize clause features, but also directly uses the reward signals generated by reinforcement learning (sampling reward and main path reward) to update the guiding variable features.

[0064] S33: Parallel execution of the sampling exploration path and the main path utilization path fully realizes the decision-making, evaluation, and policy optimization loop of reinforcement learning: In the sampling exploration path, a policy distribution is constructed based on the query vector, and sampling assignments are obtained by random sampling according to the policy distribution. The satisfaction status of chip logic constraint clauses is judged based on the sampling assignments, and a sampling exploration reward is generated; the policy loss of the sampling exploration path stage is calculated based on the selection probability of the sampling assignments in the policy distribution and the sampling exploration reward; In the main path utilization path, the estimated loss caused by the assignment of different chip signal variables is evaluated based on the query vector, and the assignment with the minimum loss is selected as the main path assignment; the satisfaction status of chip logic constraint clauses is judged based on the main path assignments, and a main path utilization reward is generated; the policy loss of the main path stage is calculated based on the selection probability of the main path assignments in the policy distribution and the main path utilization reward.

[0065] In step S33, the sampling exploration phase generates a strategy distribution based on signal features. It explores the chip logic solution space by randomly sampling and assigning signal levels. Based on the change in constraint satisfaction after the current assignment and the previous result, it marks the changed signal nodes and calculates the graph-level reward corresponding to the change in the number of unsatisfied constraints. This reward is then allocated to the changed signal nodes, and the sampling strategy loss is calculated accordingly. In the main path utilization phase, the constraint loss is calculated based on the prediction results and aggregated into a graph-level loss. The assignment with the minimum graph-level loss is selected as the current optimal action, and the main path reward and utilization phase strategy loss are generated in the same way.

[0066] In step S33, the solver, acting as a reinforcement learning agent, learns and optimizes its policy through interaction with the environment (CNF constraints). In the chip design verification scenario of this application, the agent's action is to assign a value to the chip signal level, and the reward is the reduction of logical conflicts after the corresponding assignment action is executed, thereby guiding the solver to gradually generate more effective signal level configurations. The sampling exploration path explores the solution space by randomly sampling and executing trial actions. Furthermore, policy gradient loss is a core loss metric used in reinforcement learning to optimize the policy. By calculating the product of the logarithmic probability of the sampled action in its policy distribution and the reward obtained for that action, the policy parameters are adjusted through backpropagation, optimizing the policy towards "increasing the probability of selecting high-reward actions and decreasing the probability of selecting low-reward actions," which is the core optimization objective of policy gradient algorithms.

[0067] S34: Calculate the main loss based on the constraint loss evaluated in the multi-round iteration history. Optimize the solver parameters together with the main loss, the policy loss in the sampling exploration stage, and the policy loss in the main path stage. Terminate the iteration when all chip logic constraint clauses are satisfied.

[0068] In step S34, the policy loss from the sampling exploration stage and the policy loss from the main path stage are combined according to preset weights to form the total policy loss, achieving a balance between exploration and utilization. Furthermore, the gradient weights of the main loss and the total policy loss are dynamically adjusted according to the training progress, and after merging, all parameters of the solver are updated to ensure that parameter optimization both meets the requirements for reducing chip logic errors and adapts to the direction of signal assignment strategy optimization. Finally, at the end of each iteration, it is checked whether all constraints are satisfied. If all are satisfied, it indicates that an effective signal configuration satisfying the chip logic has been found, and the iteration can be terminated early to improve verification efficiency; otherwise, the next iteration continues and the loss of this round is recorded.

[0069] S4: After terminating the iteration, output the chip signal variable assignment results and the chip logic constraint satisfaction determination conclusion.

[0070] After the iteration terminates, the output signal level prediction results (corresponding to the feasible operating configuration of the chip), cumulative loss, policy loss, and iteration steps are assigned to provide a basis for solver parameter optimization and the satisfaction determination of the chip's CNF formula. The solver includes GNN and related network models under the two-stage reinforcement learning framework.

[0071] This invention introduces chip signal polarity and logic consistency scores during the variable initialization phase to construct a structure-aware initial feature vector, overcoming the limitation of traditional graph modeling which can only represent local logical relationships. A two-stage reinforcement learning framework consisting of a main path and a sampling path is designed. The sampling path is used to explore the chip logic solution space, while the main path is used to stably utilize the optimal verification strategy, effectively alleviating problems such as unstable training convergence and single strategy. Furthermore, a reward-guided feature update mechanism is proposed, explicitly binding the dynamic evolution of chip signal features with reinforcement feedback, improving the solver's interpretability and narrowing the gap between neural inference and chip logic symbolic inference. During the solution process, the solver performs signal assignment decisions and continuously optimizes the verification trajectory based on the results of multiple rounds of message passing in the GNN. It achieves efficient chip design verification capabilities without relying on manual heuristics, providing an efficient and reliable implementation scheme for solving the SAT (Simplified Design Assay) for chip design verification using a hybrid neural-symbolic optimization approach.

[0072] The method of this invention is essentially a general combinatorial optimization and constraint satisfaction problem solving engine, which can be widely applied to any field that can be reduced to a SAT problem or expressed as finding feasible / optimal solutions under constraints, such as software vulnerability analysis, resource allocation and scheduling, antenna array deployment, etc.

[0073] In one example, step S1 constructs a literal-clause bipartite graph, including:

[0074] S11: Parsing of raw CNF data.

[0075] First, define the set of variables for the conjunctive normal form formula. set of clauses :

[0076] ;

[0077] in, , All are indexes; each variable The value is or These correspond to the low and high levels of processor hardware signals, respectively (0 represents low level, 1 represents high level). Each clause It is a decomposition of several literals, representing a processor logic constraint; each literal can be a variable. Or its negation .

[0078] To construct the adjacency matrix required for the graph structure, all adjacency matrices need to be recorded. The appearance of each alphanumeric character (positive and negative) in the clause:

[0079] ;

[0080] For each character clause Record their relationships, for example: , respectively representing "body text" Appears in constraint clauses "and negative text" Appears in constraint clauses These relationships form the basis for the subsequent adjacency matrix and graph structure.

[0081] S12: Graph Structure Construction and Normalization: Explicitly Capturing CNF Structures.

[0082] To explicitly express the structural information of CNF formulas obtained from the Verilog logic transformation of the RISC-V processor data path and to support subsequent GNN message passing, a literal-clause bipartite graph is constructed to characterize the relationship between chip signals (modeled using Boolean variables) and their respective logic constraints. Furthermore, based on whether a literal appears in a chip logic constraint clause, connections are established between the corresponding literal node and the clause node, thereby constructing a literal-clause adjacency matrix. The matrix rows correspond to the text (previous row). The behavior is a literal text indicating that the signal is in an active level state, followed by... The behavior represents a negative literal indicating that the signal is in an inactive level state; the matrix columns correspond to constraint clauses of processor functions or timing logic. If the literal... Appears in constraint clauses In the middle, then ,otherwise At the same time, for Transpose to obtain the clause-literal adjacency matrix This is used for message passing from subsequent clauses to text. The adjacency matrix represents the combination of chip signal states and logical constraints as edges of a graph, enabling GNNs to directly learn local logical dependencies and global structural features, avoiding the limitations of traditional SAT solutions that rely on manual rule extraction.

[0083] Preferably, after constructing the text-clause bipartite graph, the method further includes:

[0084] Based on the connection relationship between text nodes and clause nodes, construct clause relationship graphs and variable relationship graphs.

[0085] In this context, the edges in the clause relationship graph connect two chip logic constraint clause nodes that share at least one literal. The clause relationship graph is used to characterize the logical connections between processor constraint clauses and construct the clause relationship matrix. If two clauses share at least one literal (i.e., depend on a common chip signal state), then they are considered to be logically coupled. Otherwise, the value is 0, and the clause relation matrix is ​​normalized row-wise.

[0086]

[0087] in, Represents the normalized clause relation graph adjacency matrix; Representation of the adjacency matrix of clause relation graph The Row vectors; Representation of the adjacency matrix of clause relation graph No. line, number The element values ​​of the column.

[0088] Furthermore, the edges in the variable relationship graph connect two chip signal variable nodes that appear in at least one of the same chip logic constraint clauses. The variable relationship graph describes whether two chip signals (Boolean variables) jointly participate in a certain logic constraint. Construct the variable relationship matrix. If two variables appear simultaneously in at least one constraint clause (i.e., both participate in a logical condition or functional constraint), then , For example, if an ALU opcode bit and a data select signal frequently appear together in a functional constraint, then there is an edge connection between them. Similarly, row-wise normalization is performed.

[0089]

[0090] in, Represents the adjacency matrix of the normalized variable relationship graph; Adjacency matrix representing variable relationship graph The Row vectors; Adjacency matrix representing variable relationship graph No. line, number The element values ​​of the column. The normalized variable relationship graph can prevent highly coupled signal nodes from having too much weight in message propagation, thus keeping the learning process stable.

[0091] This example demonstrates how constructing text-clause diagrams, clause relationship diagrams, and variable relationship diagrams can express the logical dependencies between chip signals and the coupling relationships between constraints, thereby preserving the original hardware logic structure features during the learning process.

[0092] In one example, step S2 generates the structure-aware initial feature vector, which includes:

[0093] S21: Structural feature calculation: Extract the core semantics of CNF formula (corresponding to chip signal behavior features).

[0094] Based on adjacency matrix Statistically count the number of times the text corresponding to each chip signal variable appears in the constraint clause, and calculate the polarity score and consistency score that reflect the signal behavior pattern.

[0095] The polarity score measures whether a signal more frequently participates in logic as a "valid state" or a "non-valid state." In CNF, positive and negative literals represent logical assertions of "true" or "false" for a variable, respectively. When mapping these to hardware signals in the RISC-V processor data path, the meaning of the valid level needs to be determined by considering the specific signal's active-high or active-low definition. By statistically analyzing the frequency of positive and negative literals in clauses, we can characterize the signal's more frequent level state (valid or invalid) in constraint inference within the chip's logic, thus providing a hardware semantic basis for interpreting the polarity score and consistency score.

[0096] First, count the number of times each character appears. The expression is:

[0097]

[0098] right Decompose to obtain the degree of the main text (The first n elements) and negative literal degree (The last n elements) Represent the set of real numbers, and then calculate the polarity score using the following formula. :

[0099]

[0100] in, To avoid a denominator of 0, the result range is... A polarity score greater than 0.5 indicates that the variable... In logical constraints, it tends to accept the truth value, and vice versa.

[0101] Furthermore, the consistency score measures the balance of positive and negative literals in a variable, reflecting the reliability of the polarity score. (Consistency score) The calculation formula is as follows:

[0102]

[0103] Calculation result range The higher the score, the better the variable. The more evenly positive and negative characters appear, the weaker the polarity score; the lower the score, the more significant the polarity score, providing a basis for subsequent feature adjustments.

[0104] S22: Perform feature standardization on polarity scores and consistency scores to enhance feature discriminative power and stability.

[0105] Preferably, to avoid interference from fluctuations in the original feature values ​​on the solver's learning, a learnable parameter and nonlinear mapping processing step is introduced after step S21 to standardize the structural features:

[0106] Define learnable parameters The initial value is set to 5.0, and it is adaptively adjusted during training to amplify feature differences and enhance the solver's sensitivity to structural information. Simultaneously, through... The function maps features to The range is determined by dynamically adjusting the weights of polarity features in conjunction with the consistency score.

[0107] ;

[0108] ;

[0109] in, Indicates a centralized polarity characteristic; This represents the centralized consistency characteristic. The centralized consistency characteristic calculation formula enables the suppression of polarity under high consistency conditions and the preservation of polarity under low consistency conditions, ensuring that the initial characteristic accurately reflects the "level assertion preference" and "logic participation mode" of the actual chip signal.

[0110] In steps S21-S22, by extracting structural features such as polarity and consistency of CNF formulas, and combining normalization and feature transformation, initial feature vectors of variables and clauses are generated. This allows the solver to capture the deep structural information of processor logic constraints in the early stages of the solution process, avoiding the limitations of traditional chip verification methods that rely on manual heuristic analysis.

[0111] S23: Initial feature vector generation: Constructing a high-dimensional feature representation that can express the chip logic structure.

[0112] By employing techniques such as feature concatenation, MLP mapping, gating optimization, and normalization, the aforementioned discrete structural features (polarity features, consistency features, level assertion biases, etc.) are transformed into high-dimensional continuous feature vectors that can be processed by GNNs, enabling the solver to capture the deep semantics of the processor's logical constraints from the initial stages of the solution process. This process comprises two parts: the generation of initial feature vectors for variables and the generation of initial feature vectors for clauses.

[0113] Initial feature vector generation for each chip signal variable: For each chip signal variable, the structural features are generated. , The features are concatenated, and then the input variables are used to initialize the gating layer to optimize the feature representation, resulting in the final initial feature vector of the variables. , The feature vector dimension encodes, in hardware semantics, the frequency characteristics of high / low level signals participating in logic, level assertion bias, the stability of the signal's role in the circuit as reflected by the consistency score, and the importance of the signal when it appears together with other logic constraints. This enables the solver to distinguish different types of hardware behavior patterns from the outset, such as data line signals, control signals, and inverting control signals.

[0114] Initial feature vector generation for clauses: First, transpose the adjacency matrix of the literal and clause to obtain the structure of the literal pointed to by the clause. Preferably, the initial feature vector of the chip signal variable is concatenated with its opposite sign to form a feature vector set containing positive and negative literals. The initial input features of the clause are obtained by aggregating the positive and negative literal feature vectors associated with the constraint through an adjacency matrix. This aggregation process is semantically equivalent in hardware to: pooling the features of all chip signals involved in a logical constraint to express the combinational logic semantics of that constraint. After obtaining the initial input of the clause, it is optimized by the clause initialization gating layer, and finally normalized based on the clause relationship graph to obtain the initial feature vector of the clause. During the initial feature vector generation process of the clause, the semantic distinction between positive and negative signal states is preserved by splitting the message by polarity. Combined with the current variable features and random noise, the generated query vector can more comprehensively encode the positive and negative uncertainties faced by each variable in the current graph state, thereby improving the accuracy of the assignment and evaluation in the subsequent reinforcement learning stage.

[0115] In one example, step S31 establishes a bidirectional correlation mechanism between chip signal features and logical constraint loss through iterative parameter initialization, clause noise processing based on satisfiability feedback, structure-aware graph message propagation, and query vector construction, providing a stable and reliable computational platform for subsequent two-stage reinforcement learning. In this example, generating the variable query vector includes the following sub-steps:

[0116] S311: Iteration parameter initialization, including:

[0117] S3111: Iteration Rounds and State Initialization: Setting the Training Iteration Rounds Test iteration rounds ; This corresponds to the solver's optimization cycle on the RISC-V processor's data path logic. The initialization clause satisfies the state. (0 indicates not satisfied, 1 indicates satisfied), assign values ​​to initialization variables (chip signals). (Initially set to true); Initialize sampling phase reward Main path stage rewards (No initial reward).

[0118] S3112: Hyperparameter Setting: Introducing Dynamic Attenuation Coefficient Each round according to renew, The current round is defined, and T represents the total number of rounds. Set the loss weights for the two-stage strategy: sampling phase... Main path utilization phase This ensures an effective balance between "exploring different signal combinations" and "converging to the optimal signal assertion configuration".

[0119] S312: Clause noise import based on satisfiability feedback: constructing structure-aware positive and negative samples.

[0120] To improve the solver's ability to distinguish the semantic contribution of clauses, clause satisfaction labels are introduced to construct positive and negative sample pairs. First, the original clause satisfaction vector... Extend it to a semantic mask and define its broadcast form in the feature space. ,in This indicates that the corresponding clause can be satisfied at present (positive sample). This indicates that the corresponding clause is not satisfied (negative sample). Based on this label, the current feature vector of the clause... Perform a calculation as follows:

[0121] ;

[0122] in, This represents the updated clause feature vector. In the above update calculation expression, clauses that satisfy the condition retain their original semantic expression, while clauses that do not satisfy the condition are injected with perturbations. This weakens the information contribution of unsatisfied clauses in the message passing process. This mechanism breaks the pattern of indiscriminate propagation of clause semantics and explicitly constructs a structural bias in which "reliable clauses actively transmit information, while invalid clauses are suppressed and silenced." This can enhance the solver's ability to identify the differences in the distribution of clauses with different satisfaction states and improve the modeling accuracy of the semantic structure of CNF formulas.

[0123] S313: Structure-aware graph message propagation: Variable (signal) updates are driven only by valid clauses.

[0124] After completing the clause feature vector degradation process, the adjacency matrix of clauses and literals is used. The graph message propagation from execution clauses to literals achieves semantic aggregation under structural constraints. The specific calculation method is as follows:

[0125] ;

[0126] in, This represents the message vector of the text node. Since the unsatisfied negative sample clauses have been disturbed by noise, their feature vector features have lost effective semantic information. Therefore, it is almost impossible to transmit valuable structural signals to the text layer during the above propagation process, thus ensuring that only the currently satisfyable (effective) logical constraints will affect the variable feature vector.

[0127] Preferably, after propagation is complete, the solver determines the polarity of the text. The message is split into its main text. Negative text messages This process preserves the semantic symmetry of positive and negative text in the SAT problem structure while achieving fine-grained structural alignment. This allows the feature updates of the two polarities of variables to receive structural constraint feedback from CNF formulas from different directions, improving the accuracy of variable feature vectors in modeling the semantics of formulas.

[0128] S314: Query Vector Construction: Connecting variable features with clause loss to support bidirectional updates.

[0129] The current feature vector of the variable (chip signal) Text message Negative text messages The vector is concatenated and simultaneously injected with 4D random noise to form the query input vector. .Will Input a dedicated variable lookup network, output a variable lookup vector. This variable query vector not only encodes the association information between the variable and the text, but also serves as the backpropagation of the gradient, and provides error feedback for subsequent clause feature updates.

[0130] In one example, step S32 involves dynamically updating the feature vector of the chip logic constraint clause based on the current chip logic constraint clause loss and the gradient information contained in the variable query vector, including:

[0131] S321: Clause Feature Integration and Gradient Injection: If message passing is enabled, the current feature vector of the clause will be... Clause loss Variable query vector Messages from text to clause Concatenation, by querying variable lookup vectors containing chip signal variable characteristics. Incorporate gradient information; if message passing is not enabled, only concatenate the current feature vector. With clause loss Forming the clause feature vector .

[0132] S322: Feature Transformation and Residual Update: Transforming the Clause Feature Vector The input is processed by a multilayer perceptron to obtain the variable (signal) loss signal. new state of clause The clause feature vector is updated through residual connections, calculated using the following formula:

[0133] ;

[0134] in, Indicates the first The clause feature vector is updated in the next iteration; This indicates pairwise normalization, ensuring the numerical stability of clause features. Furthermore, the update process incorporates gradient feedback from the query vector, adjusting constraint features to reduce logical conflicts and align with the verification progress. By directly superimposing the original input features and the transformed features in the feature update path, the solver retains previously valid feature information, avoiding feature oscillations caused by increased network depth or excessive update amplitude, thus ensuring the stability and continuity of feature updates.

[0135] In step S32, the feature vector of the chip signal variables is dynamically updated using the sampling reward and main path reward generated in the previous iteration, including:

[0136] S323: The sampling reward is concatenated with the current feature vector of the chip signal variable and input into the first update gating network to generate a feature offset. The current feature vector of the chip signal variable is initially updated based on the first feature offset to obtain the initial updated feature vector.

[0137] Specifically, first, reward the original samples of each variable from the previous round. Normalization is performed, through The function maps it to The interval is used to eliminate the interference of reward value fluctuations on the update, resulting in a normalized sampled reward. This normalized reward is then compared with the current variable state. Multiplication achieves a deep binding between the reward signal and variable features, and finally, it is combined with the original variable feature vector. Concatenate the vectors to construct a reward enhancement vector. The calculation formula is:

[0138] .

[0139] Joint enhancement vector Input reward-gated network, adaptively generate feature offsets (Filtering valid rewards and tier information), combined with dynamic decay coefficients (To achieve "early exploration and later reuse") and with a fixed scaling factor such as 0.05, the first update of the variable feature vector is completed, and the calculation formula is as follows:

[0140] ;

[0141] in, Indicates the first The variable feature vector after the first update (initial update of feature vector); Indicates the first The feature vector of the variables before the update.

[0142] S324: The initial updated feature vector, positive and negative text loss signals, and gradient information related to chip signal variables are concatenated and input into the update gate network for feature transformation to generate directional feature vectors and perform normalization to obtain the directional vector; the directional adjustment amount is generated by combining the main path reward generated in the previous iteration with the dynamic decay coefficient; the directional adjustment amount is used to optimize the directional feature vector along the directional vector, and the optimization result is combined with the current feature vector through residual connection to obtain the final updated chip signal variable feature vector.

[0143] Specifically, to further enhance the structural semantics of variable features, the first updated... loss of text Negative text loss (Loss from global variables) (obtained by splitting) gradient information of variables The concatenation process involves inputting the data into an updated gating network for feature transformation, generating a preliminary optimized variable feature vector from the sampling phase. :

[0144] ;

[0145] in, This indicates an update to the gating network. This step, by fusing gradient and structural loss signals, optimizes the variable feature vectors in a direction that reduces the degree of clause dissatisfaction, laying the foundation for the subsequent main path stage.

[0146] Then, to Do Normalization eliminates the interference of feature magnitude differences on the update, retaining only feature direction information to obtain the direction vector. The calculation formula is:

[0147]

[0148] in Norm, after normalization This ensures that subsequent rewards only adjust the feature direction and do not change the feature scale.

[0149] The previous round of main path rewards through Function mapping to The interval, combined with the dynamic attenuation coefficient, generates a direction adjustment amount along the range. The direction is correct Optimization is performed to complete the second update of the variable feature vector. The calculation formula is as follows:

[0150] ;

[0151] in, Indicates the first The variable feature vector after the second update. The positive reward signal drives the variable feature vector along... Positive shifts reinforce the optimal strategy, while negative reward signals drive feature shifts in the opposite direction, suppressing ineffective strategies.

[0152] Finally, normalization and residual update (final state) are performed. To ensure the numerical stability of the variable eigenvectors, the following steps are taken: Paired normalization is performed, and residual connections are used to preserve the feature vectors from the previous round. Effective information is used to avoid excessively large update amplitudes that could lead to feature oscillations, ultimately resulting in a new variable feature vector. Represented as:

[0153] ;

[0154] in, This indicates pairwise normalization.

[0155] In step S32 of this example, using the "query vector gradient" as the error criterion and the "reinforcement learning reward" as the optimization direction, the rewards from the previous sampling phase and main path utilization phase are used to enable bidirectional dynamic updates of the variable feature vector and clause feature vector under the feedback of chip logic constraints. This mechanism enables the features to continuously evolve in the direction of improving the logical consistency of the RISC-V data path (i.e., clause satisfaction).

[0156] In one example, step S33 executes the sampling exploration path and the main path utilization path in parallel, including the following sub-steps:

[0157] S331: Sampling and exploration phase: Randomly select actions to avoid local optima.

[0158] S3311: Policy Distribution Construction: A policy distribution is constructed based on signal (variable) query vectors. Hardware signal levels are randomly sampled and assigned values ​​to explore the chip logic solution space (simulating signal configuration scenarios under different operating conditions). Specifically, the variable query vectors... Input Multilayer Perceptron (MLP) generates assigned scores (8 represents the candidate values ​​for the assigned values, corresponding to the possible values ​​of the variables). The multilayer perceptron is a policy network within a reinforcement learning framework, further processed through... The function constructs a strategy distribution to ensure the randomness of the assignment selection. The specific formula is as follows:

[0159] ;

[0160] in, This indicates the action of assigning values ​​to chip signal variables; Represents variable query vector This indicates the current status of chip verification.

[0161] S3312: Assignment Sampling and State Judgment: Randomly select sampled signal levels from the policy distribution for assignment. Call the processor logic constraint satisfaction determination function and assign values ​​based on the samples. Determine the satisfaction state of each clause and generate a chip logic constraint clause satisfaction state vector. And count the number of clauses that did not meet the constraints in this round. , Indicates the first The satisfied state components of each sentence under the sampling assignment.

[0162] S3313: Reward generation and strategy loss calculation.

[0163] S33131: Determine the satisfaction status of chip logic constraint clauses based on sampled assignments, and generate sampling exploration rewards, including:

[0164] The difference between the number of chip logic constraint clauses that are not satisfied before and after the sampling assignment is calculated to obtain the graph-level reward of the sampling path; the chip signal variables whose values ​​change during the sampling assignment are identified; the graph-level reward of the sampling path is allocated to the chip signal variables whose values ​​change during the sampling assignment to generate the sampling exploration reward.

[0165] Specifically, compare the signal level assignment with the previous round. In conjunction with this round of sampling and assignment, identify hardware signals that have undergone value changes, and generate an assignment change mask. Then, the atlas reward is calculated based on the change in the number of unmet constraints. (Reflecting the degree of improvement in chip logic conflicts), the calculation formula is:

[0166] ;

[0167] in This represents the number of unsatisfied clauses from the previous round. The atlas rewards are then distributed to individual variables via a variable relationship graph, using masks in this process. To ensure that the reward applies only to signals with varying levels (the excitation solver explores and adjusts the configuration of valid signals), the sampling phase reward for each signal is obtained. :

[0168] ;

[0169] S33131: Calculate the policy loss during the sampling phase The solver is guided to explore the effective solution space, and the calculation formula is as follows:

[0170] ;

[0171] in, Representation Strategy Expectations; Indicates the state Next, select an action. The logarithmic probability.

[0172] S332: Main path utilization stage: Optimize known strategies to improve solution efficiency.

[0173] Based on the signal query vector, the optimal hardware signal level is assigned, and the learned effective strategies are used to improve chip verification efficiency (focusing on conflict-free signal configuration). The specific steps are as follows:

[0174] S3321: Optimal Assignment Selection and Clause Loss Calculation: Based on Variable Query Vector The generated assignment score Calculate the loss for each clause (measuring the degree of constraint violation, i.e., the severity of chip logic conflict), and then apply the clause loss to the clause relationship graph. Aggregate the results into graph-level losses (reflecting the overall logical conflict level of the processor), and assign values ​​to variables that minimize these graph-level losses. As the optimal action.

[0175] S3322: Reward and Strategy Loss Calculation: The same calculation method as the sampling exploration phase is used, only replacing the sampling-related variables with the main path-related variables to generate clauses that satisfy the state in the main path phase. Assignment change mask With main path rewards And calculate the strategy loss for the main path phase. .

[0176] In step S33, by constructing a two-stage reinforcement learning framework of "sampling exploration + main path utilization", the problems of reward sparsity, insufficient exploration or excessive greed leading to local optima that are common in traditional policy optimization are solved. This enables the RISC-V processor hardware signal level assignment strategy to accurately fit the CNF structure and chip logic constraint characteristics, thereby significantly improving the chip verification solution efficiency and logic constraint satisfiability optimization effect.

[0177] In one example, step S34 optimizes the solver parameters through multi-loss co-optimization and determines whether the CNF formula (corresponding to hardware logic constraints) of the RISC-V processor data path can be satisfied based on the iteration results. If satisfied, the iteration is terminated early to improve verification efficiency. This includes the following sub-steps:

[0178] S341: Construction of multiple loss functions: taking into account both the minimization of logical error and the optimization of signal assignment strategy.

[0179] A multi-objective optimization system consisting of principal loss and policy loss is constructed to ensure that the solver simultaneously reduces constraint violation (clause loss) and optimizes the hardware signal level assignment strategy.

[0180] S3411: Calculate the main loss Based on multi-round graph-level loss aggregation, the solver is guided to reduce the number of unsatisfied clauses, thereby reducing chip logic conflicts. The calculation formula is as follows:

[0181]

[0182] in, The graph-level loss for the k-th round; principal loss It reflects the average solution error across multiple iterations and is directly related to the average degree of satisfaction of chip logic constraints.

[0183] S3411: Calculate the total policy loss Loss based on the sampling phase strategy and the loss of the main path phase strategy The total policy loss is obtained. Specifically, the policy loss during the sampling phase is... Main path phase strategy loss The total strategy loss is calculated by integrating components according to preset weights, ensuring that the solver simultaneously considers both traversal of the chip logic solution space and optimization of the optimal verification strategy. The calculation formula is as follows:

[0184] ;

[0185] in, , These are the weighting coefficients for the policy loss during the sampling phase and the policy loss during the main path phase, respectively. The processor hardware signal level assignment strategy is optimized through the total policy loss to ensure that the solver selects a signal configuration scheme that is more likely to improve chip verification progress (reduce logic conflicts and shorten verification cycles).

[0186] S342: Dynamic gradient merging and parameter update: Balancing optimization directions to ensure convergence.

[0187] The gradient weights of the main loss and policy loss are dynamically adjusted according to the training progress, and then merged to update all parameters of the solver. This ensures that the parameter optimization not only meets the requirements for reducing chip logic errors but also adapts to the direction of signal assignment policy optimization. The specific steps are as follows:

[0188] S3421: Gradient Calculation: In the training step function, the main loss gradient is calculated separately using the two gradient bands. (The direction of parameter adjustment corresponding to the reduction of chip logic error) and the gradient of policy loss (The direction of parameter adjustment for corresponding signal assignment strategy optimization), where These are all trainable parameters for the solver.

[0189] S3422: Dynamic gradient weight adjustment: based on training progress Calculate the weights of the loss gradient for dynamically adjusting the strategy. .

[0190] S3423: Gradient merging and parameter update: The main loss gradient is merged with the weighted policy loss gradient, and the solver parameters are updated using the Adam optimizer. The calculation formula is as follows:

[0191] ;

[0192] ;

[0193] in, This represents the sum of the main loss gradient and the weighted policy loss gradient; the left side of the equation... Indicates the updated solver parameters; right side of the equation This indicates the solver parameters before the update; The learning rate (default value) This ensures that the solver parameters are updated stably along the lines of "reducing chip logic conflicts and optimizing signal assignment strategies," thus avoiding gradient oscillations during the verification process.

[0194] S343: SAT satisfies the judgment and early termination mechanism.

[0195] To optimize chip verification efficiency, after each iteration, a SAT (Standard Availability) determination is performed based on the current processor logic constraints (clauses), triggering an early termination mechanism. The specific logic is as follows:

[0196] S3431: Constraint Satisfaction Status Determination: After iteration, calculate the processor logic constraint satisfaction status of the current round and set parameters. This is used to determine whether all constraints are satisfied. If all constraints are satisfied, then... This indicates that a valid signal level assignment scheme for the processor data path has been found (corresponding to compliant and conflict-free chip logic functions), and further iteration is unnecessary; if A value of 0 indicates that there are still unmet constraints (conflicts exist in the chip logic), and the chip needs to enter the next iteration for further optimization.

[0197] S3432: Early termination trigger: When detected If the condition is met, immediately terminate the subsequent iteration process, skip the remaining rounds of calculation, and directly return the hardware signal level assignment result of the current round (corresponding to the signal configuration scheme for normal chip operation) and solver parameters, which greatly shortens the chip verification cycle; otherwise, use the updated parameters for the next round of iteration to continuously optimize the signal assignment strategy to eliminate logical conflicts.

[0198] Combining the above examples yields preferred examples of the present invention, such as... Figure 2 As shown, the method includes the following steps:

[0199] S100: Obtain the original conjunctive normal form formula for the chip design verification task, construct a literal-clause bipartite graph, a clause relation graph, and a variable relation graph based on the conjunctive normal form formula, and normalize the three graph structures.

[0200] S200: Calculate the polarity score and consistency score of each chip signal variable based on the adjacency matrix, standardize the polarity score and consistency score to obtain polarity features and consistency features, generate the initial feature vector of the chip signal variable based on the polarity features and consistency features, and generate the initial feature vector of the chip logic constraint clause based on the initial feature vector of the chip signal variable and the adjacency matrix.

[0201] S300: Performs multiple rounds of iterative solution, determines whether the maximum number of iterations has been reached, and if so, outputs the variable assignment results and solver parameters; if not, initializes the iteration-related parameters, and determines the satisfaction status of each chip logic constraint clause based on the current chip signal variable assignment. Noise perturbation is injected into the feature vector of chip logic constraint clauses that are not satisfied. Then, the feature vector of the clause is aggregated to the text node through the text-clause bipartite graph to obtain the message aggregated to the text node. The message aggregated to the text node, the current feature vector of the chip signal variable, and random noise are fused to generate a query input vector. The query input vector is then input into the variable query network to generate a variable query vector.

[0202] S400: Based on the current chip logic constraint clause loss and the gradient information contained in the variable query vector, dynamically update the feature vector of the chip logic constraint clause; utilize the sampling reward and main path reward generated in the previous iteration to dynamically update the feature vector of the chip signal variable;

[0203] S500: Parallel execution of sampling exploration path and main path utilization path: In the sampling exploration path, a policy distribution is constructed based on the query vector, and sampling assignments are obtained by random sampling according to the policy distribution. The satisfaction status of chip logic constraint clauses is judged based on the sampling assignments, and sampling exploration rewards are generated. The policy loss of the sampling exploration path stage is calculated based on the selection probability of the sampling assignments in the policy distribution and the sampling exploration rewards. In the main path utilization path, the estimated loss caused by the assignment of different chip signal variables is evaluated based on the query vector, and the assignment with the minimum loss is selected as the main path assignment. The satisfaction status of chip logic constraint clauses is judged based on the main path assignments, and main path utilization rewards are generated. The policy loss of the main path stage is calculated based on the selection probability of the main path assignments in the policy distribution and the main path utilization rewards. The total policy loss is obtained based on the policy loss of the sampling exploration path stage and the policy loss of the main path stage.

[0204] S600: Calculate the main loss based on the constraint loss evaluated in the history of multiple iterations, and optimize the parameters of the solver together with the main loss and the total policy loss. When it is detected that all chip logic constraint clauses are satisfied, the iteration is terminated, and the chip signal variable assignment results, chip logic constraint satisfiability judgment conclusions, and corresponding solver parameters are output.

[0205] This invention provides a structure-aware, two-stage reinforcement learning method for solving the SAT (Solution of Design Assurance) problem in chip design verification. This method addresses issues in chip design verification such as the heavy reliance of traditional solvers on heuristic rules, the limited ability of existing neural solvers to model the CNF (Conventional Functional Reasoning) formula structure of chip logic constraints, the sparse and unbalanced exploration and utilization of reinforcement learning rewards, and the insufficient interpretability of variable feature evolution. By constructing a text-clause bipartite graph (LCG) to explicitly represent the chip's CNF formula structure, combining a two-stage reinforcement learning mechanism to balance solution space exploration and optimal strategy utilization, and introducing a reward-guided feature update strategy and a multi-loss collaborative optimization scheme with dynamic gradient merging, this method achieves efficient end-to-end solution to the SAT problem in chip design verification.

[0206] The present invention also provides a computer program product, comprising a computer program that, when executed by a processor, implements the steps of the structure-aware two-stage reinforcement learning SAT solving method formed by any or a combination of the above examples. The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.

[0207] The present invention also provides a storage medium having the same inventive concept as the structure-aware two-stage reinforcement learning SAT solving method formed by any or more of the above examples, wherein computer instructions are stored thereon, and the computer instructions, when executed, perform the steps of the structure-aware two-stage reinforcement learning SAT solving method formed by any or more of the above examples.

[0208] Based on this understanding, the technical solution of this embodiment, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0209] This invention also provides a terminal that shares the same inventive concept as any or a combination of examples corresponding to the aforementioned structure-aware two-stage reinforcement learning SAT solving method, including a memory and a processor. The memory stores computer instructions executable on the processor, and the processor executes the steps of the aforementioned structure-aware two-stage reinforcement learning SAT solving method when executing the computer instructions. The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement this invention.

[0210] In one example, the terminal, i.e., the electronic device, is represented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit (processor) mentioned above, at least one storage unit mentioned above, and a bus connecting different system components (including storage units and processing units).

[0211] The storage unit stores program code that can be executed by the processing unit, causing the processing unit to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit can execute the above-described structure-aware two-stage reinforcement learning SAT solving method.

[0212] The storage unit may include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 3201 and / or a cache storage unit, and may further include a read-only memory (ROM).

[0213] The storage unit may also include a program / utility having a set (at least one) of program modules, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0214] A bus can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus that uses any of the various bus structures.

[0215] The electronic device can also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output (I / O) interfaces. Furthermore, the electronic device can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. It should be understood that other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0216] Through the above description, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to this exemplary embodiment can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method of the exemplary embodiment of this application.

[0217] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.

Claims

1. A structure-aware, two-stage reinforcement learning method for solving the SAT, characterized in that, Includes the following steps: Obtain the conjunctive normal form formula for the chip design verification task, and construct a literal-clause bipartite graph based on the conjunctive normal form formula; the Boolean variables of the conjunctive normal form formula correspond to chip signal variables, and the clauses correspond to chip logic constraint clauses; In the literal-clause bipartite diagram, each chip signal variable is split into positive literal nodes and negative literal nodes. Each chip logic constraint clause is treated as an independent node. Based on whether the literal appears in the chip logic constraint clause, a connection relationship is established between the corresponding literal node and the clause node to form an adjacency matrix. The polarity score and consistency score of each chip signal variable are calculated based on the adjacency matrix. An initial feature vector of the chip signal variable is generated based on the polarity score and consistency score. An initial feature vector of the chip logic constraint clause is generated based on the initial feature vector of the chip signal variable and the adjacency matrix. The polarity score is used to indicate whether the chip signal variable tends to be high or low level in the logic constraint, and the consistency score is used to indicate the degree of tendency to be high or low level. Perform multiple rounds of iterative solution, each round of iteration including the following sub-steps: Based on the current chip signal variable assignments, the satisfaction status of each chip logic constraint clause is determined. For chip logic constraint clauses that are not satisfied, noise perturbation is injected into their feature vectors. Then, the feature vectors of the clauses are aggregated to literal nodes using a literal-clause bipartite graph, yielding the aggregated message. The aggregated message, the current feature vector of the chip signal variable, and random noise are fused to generate a query input vector. This query input vector is then input into a variable query network to generate a variable query vector. Based on the current chip logic constraint clause loss and the gradient information contained in the variable query vector, the feature vector of the chip logic constraint clause is dynamically updated; the feature vector of the chip signal variable is dynamically updated using the sampling reward and main path reward generated in the previous iteration. Parallel execution of the sampling exploration path and the main path utilization path: In the sampling exploration path, a policy distribution is constructed based on the query vector, and sampling assignments are obtained by random sampling according to the policy distribution. The satisfaction status of chip logic constraint clauses is determined based on the sampling assignments, and a sampling exploration reward is generated. The policy loss of the sampling exploration path stage is calculated based on the selection probability of the sampling assignments in the policy distribution and the sampling exploration reward. In the main path utilization path, the estimated loss caused by the assignment of different chip signal variables is evaluated based on the query vector, and the assignment with the minimum loss is selected as the main path assignment. The satisfaction status of chip logic constraint clauses is determined based on the main path assignments, and a main path utilization reward is generated. The policy loss of the main path stage is calculated based on the selection probability of the main path assignments in the policy distribution and the main path utilization reward. The main loss is calculated based on the constraint loss evaluated in the history of multiple iterations. The parameters of the solver are optimized in conjunction with the main loss, the sampling exploration path stage strategy loss, and the main path stage strategy loss. The iteration is terminated when it is detected that all chip logic constraint clauses are satisfied. After the iteration is terminated, the output chip signal variable assignment results and the chip logic constraint satisfaction determination conclusion are presented.

2. The structure-aware two-stage reinforcement learning SAT solution method according to claim 1, characterized in that, After constructing the text-clause bipartite graph, it also includes: Based on the connection relationship between text nodes and clause nodes, a clause relationship graph and a variable relationship graph are constructed; the edges of the clause relationship graph connect two chip logic constraint clause nodes that share at least one text, and the edges of the variable relationship graph connect two chip signal variable nodes that appear in at least one identical chip logic constraint clause.

3. The structure-aware two-stage reinforcement learning SAT solution method according to claim 1, characterized in that, The process of generating the initial feature vector of the chip logic constraint clause based on the initial feature vector of the chip signal variables and the adjacency matrix includes: The initial feature vector of the chip signal variable is expanded into positive and negative text features corresponding to the chip signal variable. The text features associated with each chip logic constraint clause are aggregated through the adjacency matrix, thereby generating the initial feature vector of the chip logic constraint clause.

4. The structure-aware two-stage reinforcement learning SAT solution method according to claim 1, characterized in that, The process of fusing the messages aggregated to the text nodes, the current feature vector of the chip signal variables, and random noise to generate the query input vector includes: The messages aggregated to the text nodes are split into positive text messages and negative text messages according to the text polarity. The positive text messages, negative text messages, the current feature vector of the chip signal variables, and random noise are concatenated to obtain the query input vector.

5. The structure-aware two-stage reinforcement learning SAT solution method according to claim 1, characterized in that, The process of dynamically updating the feature vector of the chip signal variables using the sampling reward and main path reward generated in the previous iteration includes: The sampling reward is concatenated with the current feature vector of the chip signal variable and input into the first update gating network to generate a feature offset. The current feature vector of the chip signal variable is initially updated based on the first feature offset to obtain the initial updated feature vector. The initial updated feature vector, positive and negative text loss signals, and gradient information related to chip signal variables are concatenated and input into the update gate network for feature transformation to generate directional feature vectors. These directional feature vectors are then normalized to obtain the directional vector. The directional adjustment amount is generated by combining the main path reward generated in the previous iteration with the dynamic decay coefficient. The directional adjustment amount is then used to optimize the directional feature vector along the directional vector. Finally, the optimization result is combined with the current feature vector through residual connections to obtain the updated chip signal variable feature vector.

6. The structure-aware two-stage reinforcement learning SAT solution method according to claim 1, characterized in that, The step of determining the satisfaction status of chip logic constraint clauses based on sampled assignments and generating sampling exploration rewards includes: The difference between the number of unsatisfied chip logic constraint clauses before and after the sampling assignment is calculated to obtain the graph-level reward of the sampling path; Identify chip signal variables whose values ​​change during sampling and assignment; The graph-level reward of the sampling path is allocated to the chip signal variables whose values ​​change during the sampling assignment, thereby generating a sampling exploration reward; Based on the main path assignment, determine the satisfaction status of the chip logic constraint clauses, and generate the main path utilization reward, including: The graph-level reward for the main path is obtained by calculating the difference between the number of chip logic constraint clauses that are not satisfied before and after the main path is assigned. Identify chip signal variables whose values ​​change during the main path assignment; The graph-level reward of the main path is allocated to the chip signal variables whose values ​​change during the main path assignment, thereby generating the main path utilization reward.

7. The structure-aware two-stage reinforcement learning SAT solution method according to claim 1, characterized in that, The parameters of the solver that are jointly optimized based on the main loss, the sampling exploration path stage strategy loss, and the main path stage strategy loss include: The total strategy loss is obtained by combining the strategy loss during the sampling exploration phase and the strategy loss during the main path phase. Based on the current training progress, dynamically adjust the weights of the main loss and the total policy loss in gradient updates; The gradients of the main loss and the total policy loss after weight updates are merged, and the parameters of the solver are updated using the merged gradients.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the structure-aware two-stage reinforcement learning SAT solving method according to any one of claims 1-7.

9. A storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed, they perform the steps of the structure-aware two-stage reinforcement learning SAT solving method as described in any one of claims 1-7.

10. A terminal comprising a memory and a processor, wherein the memory stores computer instructions executable on the processor, characterized in that, When the processor executes the computer instructions, it performs the steps of the structure-aware two-stage reinforcement learning SAT solving method according to any one of claims 1-7.

Images