Video Conference Engagement Scoring With Parallel Sensor Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems lack the ability to accurately and efficiently determine user engagement in real-time interactions, such as video conferences, using sensor data, leading to latency and increased computational costs.
Innovation Solution
A system utilizing a neural network to process video and audio data from multiple engagement categories in parallel, generating sub-scores that are combined to provide real-time engagement scores with reduced latency and computational overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used to determine user engagement, then measurement capability is achieved, but latency increases and computational costs increase
Solution Approach 1:
The patent segments the engagement determination process into multiple independent engagement categories (e.g., facial expressions, head movements, gestures, verbal responses). Each category is processed separately through dedicated sensor inputs and machine learning models, enabling parallel computation that reduces overall latency while maintaining comprehensive engagement measurement accuracy.
Solution Approach 2:
The patent transitions from traditional single-dimension engagement measurement to multi-dimensional assessment by incorporating diverse sensor modalities (cameras, microphones, touch sensors) and analyzing multiple engagement categories simultaneously. This dimensional expansion enables more accurate engagement detection through complementary signals while distributing computational load across different processing streams.
2Measurement precision
If comprehensive sensor data processing is implemented to improve engagement measurement accuracy, then measurement precision improves, but computational costs increase
Solution Approach 1:
The patent divides comprehensive sensor data processing into separate engagement category streams (facial expressions, head movements, gestures, verbal responses). Each stream processes only relevant sensor data through specialized machine learning models, avoiding the computational overhead of analyzing all sensor data for all engagement categories, thus reducing overall computational cost while maintaining measurement precision.
Solution Approach 2:
The patent implements selective processing where machine learning models analyze only the engagement categories and sensor data streams that are most relevant to determining user engagement in specific contexts. This partial action approach avoids unnecessary computational expenditure on less relevant data while maintaining sufficient measurement precision for engagement determination.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting engagement. One of the methods is a method of determining an engagement score for each of a plurality of participants in a video conference call, and includes, for each of the plurality of participants: receiving an encoded video stream of the participant, the encoded video stream representing an encoding of a sequence of one or more video frames; decoding the encoded video stream to determine the sequence of video frames; for each of a plurality of engagement categories representing respective different modes by which the participant may be engaged in the conference call, processing one or more respective video frames from the sequence of video frames to generate a sub-score for the engagement category; and combining the sub-scores for the plurality of engagement categories to generate the engagement score for the participant.