Runtime ML Model for Formal Verification Property Hardness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Formal verification tools face challenges in estimating the hardness of properties without actually solving them, leading to inefficient scheduling and low prediction accuracy due to limited feature sets and non-linear objective functions in ad-hoc heuristics, and machine learning models trained on generic datasets may not accurately predict hardness for unique designs.
Innovation Solution
A machine learning model is trained incrementally during runtime, refining its predictions based on the results of formal verification, allowing it to learn the specific characteristics of the design and adapt to changes in the verification process, thereby improving the accuracy of hardness predictions and scheduling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ad-hoc heuristics with limited feature sets are used to estimate property hardness, then the estimation process is simple and fast, but the prediction accuracy is low
Solution Approach 1:
The patent transforms the estimation approach by changing from a static, limited feature set to a dynamic, evolving feature set. The system initially uses a small feature set for quick estimates, then progressively incorporates additional features (such as property characteristics, design attributes, and verification results) as the verification process unfolds. This parameter change enables the system to maintain simplicity early on while achieving high accuracy as more information becomes available.
Solution Approach 2:
The patent implements a dynamic estimation system where the feature set and prediction model evolve during the verification process. Rather than using a fixed heuristic, the system adapts by incorporating results from previously solved properties and updating its understanding of the design characteristics. This dynamic approach allows the system to improve prediction accuracy over time while managing complexity through staged feature incorporation.
2Measurement precision
If machine learning models are trained on generic training sets from other designs, then the model can be prepared in advance, but the prediction accuracy for unique designs is low
Solution Approach 1:
The patent applies preliminary action by training the machine learning model on a generic training set from other designs before the actual verification begins. This initial training provides a baseline model that can make preliminary predictions. The system then refines this pre-trained model during the verification process using actual design data, combining the benefits of advance preparation with adaptive learning.
Solution Approach 2:
The patent implements feedback by using the results of solved properties during verification to update and refine the machine learning model. As properties are solved, their characteristics and outcomes are fed back into the training set, allowing the model to adapt to the specific design being verified. This feedback loop enables the system to improve prediction accuracy for the current design while maintaining the initial advantage of pre-training.
3Productivity
If properties are solved in arbitrary order, then the verification process is simple to implement, but the throughput is low due to inefficient resource utilization
Solution Approach 1:
The patent applies preliminary action by performing preliminary hardness estimation for all properties before the verification process begins. Using the machine learning model, the system predicts the relative hardness of each property in advance and creates an initial scheduling order. This preliminary ordering allows the verification engine to prioritize easier properties first, improving throughput without requiring complex real-time scheduling decisions during execution.
Solution Approach 2:
The patent implements feedback by continuously updating the property ordering based on actual verification results. As properties are solved, the system learns from the outcomes and adjusts the hardness estimates for remaining unsolved properties. This feedback-driven reordering ensures that the scheduling remains optimal throughout the verification process, maintaining high throughput while adapting to the actual difficulty of properties as they are encountered.
Data Source
AI summary
A machine learning model predicts the hardness of unsolved properties. For example, the machine learning model may predict the relative hardness of pairs of properties—i.e., which property in the pair is harder to solve. These hardness predictions may then be used to formulate a priority order for a formal verification process to attempt to solve the unsolved properties. As the formal verification process progresses, it generates results. For example, certain properties may be solved. These results are used to update a training set, which is used to further train the machine learning model. The machine learning model is trained at runtime with incremental fine-tuning as the formal verification process progresses.