AR Intent Prediction via Facial Action Unit Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems are inadequate for reliably visualizing items in real time and predicting user behavior, which limits their ability to provide relevant targeted content and efficiently use computing resources.
Innovation Solution
An augmented reality system that analyzes video frames to determine action units on a user's face, calculates intensity scores, and uses a predictive model to indicate user intent, allowing for real-time visualization of items and personalized content recommendations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If augmented reality systems analyze video frames to predict user behavior, then the relevance of targeted content is improved, but the complexity of the system increases
Solution Approach 1:
The system segments user behavior analysis into distinct action units (AU) corresponding to specific facial muscles. Each AU is independently detected and scored, allowing the complex task of behavior prediction to be divided into manageable components that can be processed separately and then integrated.
Solution Approach 2:
The patent introduces action units as intermediary elements between raw video frames and high-level user intent prediction. The AU detection and scoring system serves as a mediator that transforms complex facial expressions into standardized, quantifiable metrics that can be more easily interpreted by predictive models.
2Measurement precision
If real-time video analysis is performed to detect action units, then user intent prediction accuracy is improved, but the processing time increases
Solution Approach 1:
The system performs preliminary detection and scoring of action units from video frames before feeding this information into the predictive model. By pre-processing the video data to extract and quantify facial muscle movements in advance, the system prepares structured input data that accelerates the subsequent intent prediction process.
Solution Approach 2:
The system focuses on detecting specific, predefined action units rather than analyzing all possible facial features. This selective approach to partial action detection reduces computational overhead while maintaining sufficient accuracy for predicting user intent in augmented reality contexts.
3Adaptability or versatility
If facial action units are detected from video frames, then the ability to provide personalized content is improved, but the difficulty of detecting and measuring increases
Solution Approach 1:
The system transforms facial expressions into standardized numerical scores for each action unit. By converting qualitative facial features into quantitative parameters with defined intensity levels, the system creates measurable data that can be reliably processed by machine learning models to generate personalized content recommendations.
Data Source
AI summary
Systems and methods are disclosed herein for determining user behavior in an augmented reality environment. An augmented reality application executing on a computing system receives a video depicting a face of a person. The video includes a video frame. The augmented reality application augments the video frame with an image of an item selected via input from a user device associated with a user. The augmented reality application determines, from the video frame, a score representing an action unit. The action unit represents a muscle on the face of the person depicted by the video frame and the score represents an intensity of the action unit. The augmented reality application calculates, from a predictive model and based on the score, an indicator of intent of the person depicted by the video.