Multimodal Relationship Monitoring System Using Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting and monitoring interpersonal relationships and emotional states in real-life settings are limited by high variability in data and confounding factors, such as background speech, which decreases the accuracy of identification systems, and have not effectively utilized machine learning to model complex interpersonal processes.
Innovation Solution
A system using multimodal data from smartphones, wearable devices, and smart home devices applies machine learning algorithms to detect relationship-relevant events and states, providing feedback and monitoring to improve relationship functioning, including classification and quantification of interpersonal relationships through signal-derived features and machine learning techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning algorithms are used to detect relationship-relevant events and states in real-life settings, then the system can provide detailed feedback and monitoring to improve relationship functioning, but the high variability in data and confounding factors such as background speech decrease the accuracy of identification systems
Solution Approach 1:
The patent combines multiple data sources including audio recordings, text messages, social media posts, and survey responses into an integrated machine learning model. This multimodal approach allows the system to cross-validate signals and distinguish relationship-relevant events from confounding factors like background speech, thereby improving measurement precision while adapting to data variability.
Solution Approach 2:
The patent introduces trained researchers or clinicians as intermediaries to validate and annotate data, particularly for complex emotional states and interpersonal processes. This human-in-the-loop approach helps calibrate the machine learning algorithms to handle data variability while maintaining high accuracy in identifying relationship-relevant events.
2Loss of information
If the system monitors and collects data from multiple sources to model complex interpersonal processes, then it can provide comprehensive feedback, but the complexity of the system increases
Solution Approach 1:
The patent segments the monitoring system into distinct modular components: data collection modules (audio, text, social media), data processing modules (machine learning algorithms), and feedback modules (surveys, reports). Each module handles specific tasks independently, allowing comprehensive monitoring while managing system complexity through modular architecture.
Solution Approach 2:
The patent develops a universal machine learning framework that can process multiple types of data (audio, text, social media) and apply to various relationship types (romantic, familial, friendships). This multi-functional approach provides comprehensive feedback across different contexts without requiring separate specialized systems for each relationship type.
3Productivity
If the system uses wearable technologies and mobile devices to detect emotional states in daily life, then it can capture real-world data, but confounding factors like background speech and environmental noise reduce detection accuracy
Solution Approach 1:
The patent implements feedback loops where the system continuously monitors data quality and adjusts its detection algorithms based on validated ground truth from surveys and researcher annotations. This feedback mechanism allows the system to learn from confounding factors like background speech and improve emotional state detection accuracy over time while maintaining real-world data capture.
Solution Approach 2:
The patent dynamically adjusts detection parameters and thresholds based on contextual information from multiple data sources. When confounding factors are detected (e.g., background speech identified through audio analysis), the system modifies its sensitivity parameters and relies more heavily on complementary data sources like text messages or survey responses to maintain accurate emotional state detection.
Data Source
AI summary
A method for monitoring and understanding interpersonal relationships includes a step of monitoring interpersonal relations of a couple or group of interpersonally connected users with a plurality of smart devices by collecting data streams from the smart devices. Representations of interpersonal relationships are formed for increasing knowledge about relationship functioning and detecting interpersonally-relevant mood states and events. Feedback and/or goals are provided to one or more users to increase awareness about relationship functioning.