Personalized Gaze Training With Digital Twins for Emotional Skills
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current therapy training for neurodivergent individuals lacks personalization, fails to analyze their own facial expressions, provides imprecise feedback, and is not scalable, while AI/ML systems struggle to process real-time images and provide personalized methods for improving reciprocal eye engagement and facial emotional skills.
Innovation Solution
A personalized training system using computer vision and multimodal models, incorporating digital twins, to engage neurodivergent individuals with visual cues, track gaze and dwell time, and provide precise real-time feedback through machine learning algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional therapy training is used for neurodivergent individuals, then training can be provided, but the training is not personalized and cannot analyze their own facial expressions
Solution Approach 1:
The system creates a digital twin copy of the user's face that replicates their facial expressions and features. This digital twin is then used to provide personalized feedback and training, allowing the system to analyze and compare the user's expressions against ideal models without requiring complex real-time analysis of the user's actual facial movements.
Solution Approach 2:
The digital twin serves as an intermediary between the user and the training system. Instead of directly analyzing the user's complex facial expressions in real-time, the system uses the digital twin as a mediator to represent the user's expressions and provide feedback, simplifying the overall system architecture while maintaining personalization.
2Measurement precision
If traditional therapy training is used, then training can be provided, but feedback is imprecise and not timely
Solution Approach 1:
The system continuously tracks the user's eye engagement and facial expressions using computer vision, comparing real-time data against the digital twin model. This provides immediate feedback on whether the user is maintaining appropriate eye contact and expression, eliminating delays inherent in traditional human observation methods.
Solution Approach 2:
The system replaces human observation and manual feedback mechanisms with automated computer vision algorithms and digital twin modeling. This substitution enables precise, real-time measurement of eye engagement and facial expressions without the time delays and subjectivity of human raters.
3Productivity
If traditional 1:1 therapy is used, then personalized training can be provided, but it is not scalable and resource intensive
Solution Approach 1:
The digital twin system can serve multiple users simultaneously through standardized training protocols, making the therapy scalable. The same digital twin framework and computer vision system can be applied across different users and settings, reducing the need for specialized therapist resources while maintaining personalized feedback through individual digital twin models.
Data Source
AI summary
Provided herein are methods, devices, and systems for method for training a subject diagnosed with a socio-emotional skills deficit using engagement training to enhance dyadic behavior, comprising: selecting a first visual cue that will engage a visual attention of the subject; exposing the subject to the first visual cue for one or more visual cue cycles; recording at least one of: a focus location on screen, one or more timestamps, tracking one or more directions of gaze, one or more blinks, or a reciprocal gaze; processing the recorded tracking one or more directions of gaze of the one or more eyes to determine a dwell time of the one or more directions of gaze and repeating the step of exposing the subject, wherein an increase of the dwell time, reciprocal gaze, or both, are indicative that the subject diagnosed the socio-emotional skills deficit has increased focus and attention.