Selective Feedback Control for Automation Setting Preferences
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automation systems face challenges in understanding and adapting to exceptions in user preferences, leading to inefficiencies, such as energy savings, due to their inability to differentiate between signal noise and contextual changes.
Innovation Solution
A feedback mechanism is implemented to solicit explicit user feedback when the impact score exceeds a threshold, focusing on contextual factors that significantly impact automation settings, ensuring that feedback is provided only when it can improve the machine-learning system's performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the automation system continuously solicits user feedback to improve preference understanding, then the accuracy of automation settings improves, but user burden and system complexity increase
Solution Approach 1:
The system implements a selective feedback mechanism that calculates an impact score to determine when feedback solicitation is worthwhile. Feedback is only requested when the impact score exceeds a threshold, meaning the potential improvement in preference understanding justifies the user burden. This resolves the contradiction by making feedback solicitation conditional rather than continuous.
Solution Approach 2:
The system changes the parameter of feedback frequency from continuous to selective based on impact score thresholds. By dynamically adjusting when feedback is solicited based on calculated impact scores, the system optimizes the balance between improving preference understanding and minimizing user burden.
2Measurement precision
If the system requests feedback frequently to improve machine-learning performance, then the automation setting accuracy improves, but user time and willingness to provide feedback decrease
Solution Approach 1:
The system uses impact score calculation to determine selective moments for feedback solicitation. By evaluating whether the potential accuracy improvement justifies the time investment, the system minimizes unnecessary feedback requests while maintaining learning opportunities that provide substantial value.
Solution Approach 2:
Instead of continuously requesting feedback, the system applies partial action by selectively soliciting feedback only when the impact score indicates significant potential improvement. This avoids excessive feedback requests that would consume user time while still obtaining sufficient data for accurate automation settings.
3Quantity of substance
If the automation system asks for feedback on every setting adjustment, then the learning data quantity increases, but the signal-to-noise ratio decreases
Solution Approach 1:
The impact score calculation identifies feedback opportunities with high information value. By selectively soliciting feedback based on impact thresholds, the system captures high-signal moments (when user preferences are likely to differ from predictions) while filtering out low-signal situations, thus improving the overall signal-to-noise ratio of collected data.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The technology described herein solicits user feedback in order to improve the processing of contextual signal data to identify automation setting preferences. Users have limited availability or willingness to provide explicit feedback. The technology calculates an impact score that measures a possible improvement to the automation system that could result from receiving feedback. Feedback is solicited when the impact score exceeds a threshold. Other rules can be provided in conjunction with the impact score to determine when feedback is solicited, such as a daily cap on feedback solicitations.