Rule-Supplemented Machine Learning for Untrained Event Characterization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
ML models in computer-automated processes are vulnerable to untrained events, leading to incorrect characterizations and requiring manual intervention, which increases resource use and reduces service level agreement compliance.
Innovation Solution
Supplement ML models with rule sets to detect and characterize untrained events, using proactive, reactive, and complimentary rules to generate accurate characterizations and retrain the ML models with new data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If ML models are used to automate event characterization, then productivity increases and manual intervention decreases, but reliability deteriorates due to vulnerability to untrained events
Solution Approach 1:
The system performs preliminary actions by proactively detecting potential untrained events before they cause incorrect characterizations. The proactive rules identify events that may fall outside the ML model's training domain, allowing the system to prepare appropriate rule-based characterizations in advance, thus preventing reliability issues while maintaining automation.
Solution Approach 2:
Rule sets serve as an intermediary between the ML model and the event characterization output. When the ML model encounters untrained events, the rule sets mediate by providing alternative characterization paths, ensuring that automation continues without compromising reliability. This intermediary layer captures and routes untrained events for appropriate handling.
2Reliability
If rule sets are added to supplement ML models, then reliability improves through detection of untrained events, but device complexity increases
Solution Approach 1:
The rule sets are segmented into three distinct functional categories: proactive rules for preventing incorrect characterizations, reactive rules for handling detected untrained events, and complimentary rules for enhancing ML model performance. This segmentation allows each rule type to be independently managed, developed, and optimized, reducing the perceived complexity while maintaining comprehensive coverage.
Solution Approach 2:
The rule set framework is designed as a universal supplement that can work with any ML model across different domains. The same architectural pattern of proactive, reactive, and complimentary rules applies regardless of the specific ML model or application context, reducing complexity through reusability and standardization.
3Reliability
If manual intervention is required for untrained events, then reliability improves through human review, but productivity decreases due to increased resource use
Solution Approach 1:
The system implements self-service by enabling automatic rule-based characterization of untrained events without requiring manual intervention. The reactive rules automatically detect and characterize untrained events, while the system self-learning mechanism uses captured untrained events to retrain the ML model, reducing the need for human resources while maintaining or improving reliability over time.
Solution Approach 2:
The system implements feedback loops where untrained events captured by the rule sets are used to retrain the ML model. This feedback mechanism continuously improves the ML model's ability to handle previously untrained events, reducing the long-term need for manual intervention or complex rule sets while maintaining high reliability.
Data Source
AI summary
Various examples are directed to systems and methods for executing a computer-automated process using trained machine learning (ML) models. A computing system may access first event data describing a first event. The computing system may execute a first ML model to determine an ML characterization of the first event using the first event data. The computing system may also apply a first rule set to the first event data to generate a rule characterization of the first event. The computing system may determine an output characterization of the first event based at least in part on the rule characterization of the first event and determine to deactivate the first rule set based at least in part on the ML characterization of the first event.