Weakly Supervised Learning for Multimodal Sensing Context Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing context detection systems in mobile devices rely on high-power sensors that consume significant power and require constant activation, limiting their accuracy and battery life, while also failing to generalize well across diverse user populations due to lack of personalized training.
Innovation Solution
A method that combines low-power always-on sensors with opportunistically accessed high-power sensors to continuously update a machine learning model for context detection, using noisy automated machine annotations within a weakly supervised learning framework, allowing for local training and personalized adaptation without user input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-power sensors are constantly activated to improve context detection accuracy, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The sensing system is segmented into two distinct groups: low-power sensing sources (accelerometer, gyroscope, magnetometer, light sensor, proximity sensor) and high-power sensing sources (high-resolution camera, GPS, cellular system, Wi-Fi system, Bluetooth system). This segmentation allows the device to use low-power sensors continuously while activating high-power sensors only when needed, resolving the contradiction between continuous accurate detection and energy consumption.
Solution Approach 2:
The high-power sensing sources are activated periodically or opportunistically rather than continuously. The system uses low-power sensors to monitor context continuously and triggers high-power sensors only when specific conditions are met or when additional contextual information is needed, reducing overall power consumption while maintaining detection accuracy.
2Measurement precision
If high-power sensors are used to improve context detection accuracy, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The sensing system is divided into modular groups with distinct functions and power characteristics. Low-power sensors handle continuous monitoring while high-power sensors provide supplemental information. This modular segmentation simplifies system management and reduces operational complexity compared to a monolithic high-power sensor system.
Solution Approach 2:
Low-power sensors act as intermediaries that continuously monitor the environment and trigger high-power sensors only when necessary. This intermediary layer reduces the direct control complexity of high-power sensors while maintaining their utility for improved context detection accuracy.
3Adaptability or versatility
If traditional sensing systems are used without personalized training, then device complexity is reduced, but adaptability decreases
Solution Approach 1:
The machine learning model performs self-service by automatically training and personalizing using data from the multimodal sensing platform. The system collects measurements from both low-power and high-power sensors, processes this data locally, and continuously improves its context detection accuracy without requiring external intervention or complex external training infrastructure.
Solution Approach 2:
The system performs preliminary data collection and model training using low-power sensors before activating high-power sensors. This preliminary action allows the model to adapt to individual user patterns using energy-efficient sensors, reducing the need for complex post-deployment training while improving adaptability.
Data Source
AI summary
A machine learning model is trained for user activity detection and context detection on a mobile device. The machine learning model is configured to learn a statistical relationship between an always-on sensing modality of the mobile device and actual user context. Rather than user annotations, the machine learning model is enhanced and personalized for the always-on sensing modality by automated annotations obtained from non-always-on sensing modalities. The non-always-on sensing modality opportunistically provides an imperfect label of user context, where the imperfect label has a known associated probability of error.