Engineering Tool Recommender With Calibrated Graph Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing recommender systems for engineering tools struggle with encoding irregular, non-Euclidean graph topologies of complex systems, leading to unreliable recommendations when partially configured projects do not resemble training data, especially for less experienced engineers.
Innovation Solution
A calibrated graph neural network-based recommender system that uses graph neural networks to encode complex systems as graphs, providing a ranked list of candidate items with confidence scores, and employs an automated procedure to query user preferences when necessary to filter out unwanted items.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If graph neural networks are used to encode complex systems for recommendation, then the system can handle irregular non-Euclidean topologies, but the recommendations become unreliable when partially configured projects do not resemble training data
Solution Approach 1:
The patent introduces a calibration mechanism as an intermediary layer between the GNN encoder and the recommendation output. This calibration module adjusts the confidence scores produced by the GNN to reflect actual reliability, using techniques like temperature scaling or isotonic regression. This allows the system to maintain its ability to handle irregular graph topologies while correcting the overconfidence issue when projects differ from training data.
Solution Approach 2:
The system implements feedback loops where recommendation outcomes are monitored and used to adjust calibration parameters. When recommendations are made on projects that differ significantly from training data, the feedback mechanism detects lower accuracy and adjusts confidence scores downward, ensuring reliable uncertainty estimation while maintaining the GNN's structural encoding capabilities.
2Adaptability or versatility
If a recommender system provides many candidate items, then the coverage of possible components increases, but the user experiences confusion and errors increase
Solution Approach 1:
The patent applies local quality by differentiating the presentation of candidate items based on their individual confidence scores. High-confidence recommendations are presented with full detail and prominence, while lower-confidence items are either downgraded in visibility or presented with additional verification prompts. This allows comprehensive coverage to be maintained while reducing user confusion by highlighting only the most reliable options.
Solution Approach 2:
The system implements partial action by selectively presenting only a subset of candidate items to the user at any given time. Instead of displaying all possible candidates, the system shows a curated list based on confidence thresholds, providing sufficient coverage for most cases while avoiding overwhelming the user. Additional items can be revealed on demand with appropriate disclaimers about their lower confidence levels.
3Productivity
If the recommender system makes automated recommendations, then engineering productivity increases, but less experienced engineers may be over-reliant and make errors
Solution Approach 1:
The system provides feedback to users about the confidence level of each recommendation, enabling less experienced engineers to understand when to trust the automated suggestion and when to exercise additional caution. High-confidence recommendations encourage efficient automated workflow, while low-confidence alerts prompt users to seek additional validation or expertise, thereby maintaining productivity while reducing error rates.
Solution Approach 2:
The recommender system empowers users to self-regulate their reliance on recommendations by providing transparent confidence metrics. Users can independently assess whether to follow a recommendation based on the displayed confidence level, their own expertise, and the criticality of the decision. This self-service approach maintains productivity through automated suggestions while enabling users to independently manage error risks.
Data Source
Figure 1(a)~1(d)
Figure 2
Figure 3
AI summary
The invention relates to providing a recommender system to be used in the context of an engineering tool. By using the recommender, a list of items is provided in the engineering tool which are likely to be connected in a next step to an engineering project designed in the engineering tool.