ML Model Retraining via Location-Specific Sensor Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Low-cost physical sensors used in IoT devices often provide unreliable data, making it challenging to detect events effectively, especially in environments where the sensor's performance metrics deviate from expected parameters due to intermittent changes, and obtaining a new, effective ML model is difficult.
Innovation Solution
A system and method that utilize a first and second physical sensor, where the second sensor is different from the first, to generate location-specific training data, allowing for the reconfiguration of an ML model to improve event detection accuracy by automatically determining performance metrics and retraining the model when they fall outside expected parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If low-cost physical sensors are used in IoT devices, then device cost and power consumption are reduced, but sensor data reliability and event detection accuracy deteriorate
Solution Approach 1:
The system automatically detects when ML model performance degrades and initiates self-retraining using a second sensor as a temporary reference. This self-service mechanism allows the low-cost sensor system to maintain reliability without continuous human intervention or replacement with expensive sensors.
Solution Approach 2:
A second physical sensor (different from the first) is introduced as an intermediary to generate reference training data when the first sensor's reliability is questioned. This intermediary sensor helps bridge the gap between low-cost operation and reliable data collection by providing ground truth for model retraining.
2Device complexity
If a fixed ML model is deployed in changing environments, then device complexity is minimized, but model effectiveness and event detection accuracy deteriorate over time
Solution Approach 1:
The system transitions from a static ML model deployment to a dynamic one where the model can be automatically retrained when performance degradation is detected. This dynamic approach allows the model to adapt to environmental changes while maintaining relatively simple device architecture through automated processes.
Solution Approach 2:
The system continuously monitors ML model performance metrics and uses this feedback to automatically trigger retraining when effectiveness falls below expected parameters. This closed-loop feedback mechanism ensures model effectiveness is maintained without requiring complex manual intervention systems.
3Reliability
If continuous monitoring and automated retraining is implemented, then event detection reliability is improved, but device complexity and computational requirements increase
Solution Approach 1:
Instead of continuous monitoring and retraining, the system performs periodic performance checks and only initiates retraining when performance metrics fall below thresholds. This periodic approach maintains reliability while significantly reducing the computational burden and system complexity compared to continuous operations.
Solution Approach 2:
The automated detection and retraining system operates autonomously without requiring complex external management infrastructure. The device self-monitors its own performance and self-corrects by initiating retraining when needed, reducing the complexity of external control systems.
4Adaptability or versatility
If location-specific training data is generated using a second sensor, then model adaptability to changing environments is improved, but the need for multiple sensors and retraining infrastructure increases device complexity
Solution Approach 1:
The system uses the second sensor to generate training data that copies or replicates the environmental conditions being monitored by the first sensor. This copying approach allows the model to learn from reference data without requiring the second sensor to be a permanent, complex addition to the device.
Solution Approach 2:
The system collects training data from the second sensor in advance (when performance degradation is detected) before the actual retraining occurs. This preliminary data collection separates the data gathering phase from the model updating phase, allowing for more flexible and less complex system architecture.
Data Source
AI summary
Techniques for generating a machine learning model to detect event instances from physical sensor data, including applying a first machine learning model to first sensor data from a first physical sensor at a location to detect an event instance, determining that a performance metric for use of the first machine learning model is not within an expected parameter, obtaining second sensor data from a second physical sensor during a period of time at the same location as the first physical sensor, obtaining third sensor data from the first physical sensor during the period of time, generating location-specific training data by selecting portions of the third sensor data based on training event instances detected using the second sensor data, training a second ML model using the location-specific training data, and applying the second ML model instead of the first ML model for detecting event instances.