Rapport Analysis via Wearable ML Classifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for assessing and improving interaction skills lack objective, real-time feedback, making it difficult to enhance rapport and behavioral skills during interactions, which are crucial for various social and professional settings.
Innovation Solution
A computer system utilizing machine learning classifiers and temporal filters to analyze facial expressions, head poses, and gestures from wearable devices like Google Glass, providing objective measures of rapport and sentiment analysis in real-time, enabling immediate feedback and training improvements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If objective real-time feedback on interaction quality is implemented, then interaction skills and rapport can be improved, but system complexity and cost increase
Solution Approach 1:
The system segments interaction quality assessment into multiple independent components: facial expression analysis, head pose analysis, gesture analysis, and audio analysis. Each component is processed separately by dedicated machine learning models, allowing the complex overall assessment to be broken down into manageable, modular parts that can be developed and maintained independently.
Solution Approach 2:
The patent introduces an intermediary computer system that acts as a mediator between the interaction participants and the feedback mechanism. This intermediary automatically captures data from wearables and cameras, processes it through machine learning models, and generates feedback without requiring direct human intervention in the analysis process, thus reducing system complexity while maintaining measurement precision.
2Loss of time
If real-time feedback is provided during interactions, then immediate skill improvement is enabled, but processing speed and computational resources must be sufficient
Solution Approach 1:
The system performs preliminary actions by pre-training machine learning models offline before deployment. The models are trained on extensive datasets in advance, so that during actual interactions, only inference needs to be performed rather than full training, significantly reducing real-time computational requirements while maintaining high measurement precision.
Solution Approach 2:
The system implements periodic action by analyzing interaction data at strategically selected time points rather than continuously processing every frame. Temporal filtering and sampling techniques are used to identify key moments in the interaction that require analysis, reducing computational load while ensuring timely feedback is provided at critical instances.
3Loss of information
If multiple data streams from wearables and cameras are processed, then comprehensive rapport measurement is achieved, but data processing complexity increases
Solution Approach 1:
The patent merges multiple data streams from different sources (wearable sensors, cameras, microphones) into a unified interaction quality assessment. The machine learning models integrate facial expressions, head poses, gestures, and audio data into a single comprehensive rapport measurement, achieving complete information capture while managing processing complexity through unified model architecture.
Solution Approach 2:
The system implements universality by designing machine learning models that can process multiple types of input data (visual, auditory, sensor data) through a common processing framework. The same core models handle different data modalities, reducing overall system complexity while maintaining comprehensive data analysis capabilities across all interaction aspects.
Data Source
AI summary
In selected embodiments, one or more wearable mobile devices provide videos and other sensor data of one or more participants in an interaction, such as a customer service or a sales interaction between a company employee and a customer. A computerized system uses machine learning expression classifiers, temporal filters, and a machine learning function approximator to estimate the quality of the interaction. The computerized system may include a recommendation selector configured to select suggestions for improving the current interaction and/or future interactions, based on the quality estimates and the weights of the machine learning approximator.