Virtual Agent Widget Interaction Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual agents in dialogue systems initiate conversations based on pre-programmed questions or statements and lack the ability to dynamically respond to user difficulties with webpage widgets, failing to provide effective assistance when users encounter issues.
Innovation Solution
A computer-implemented method that determines user interactions with webpage widgets, extracts relevant content when difficulties are detected, and maps this content to virtual agent content to initiate a conversation that provides advice and guidance, using eye tracking and facial expression analysis to identify user struggles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual agents use pre-programmed questions or statements to initiate conversations, then the system complexity is low and ease of operation is high, but the adaptability to user difficulties and effectiveness of assistance is poor
Solution Approach 1:
The system pre-programs multiple possible conversation initiation statements covering various user difficulties (e.g., 'Are you having trouble with this widget?', 'Do you need help with something?'). These preliminary statements are prepared in advance and stored in the virtual agent's knowledge base, allowing the agent to select appropriate pre-programmed statements based on detected user states without requiring complex real-time analysis algorithms.
Solution Approach 2:
The system changes the parameter of conversation initiation from static (single fixed script) to dynamic (multiple selectable statements). By maintaining a set of pre-programmed statements with different tones and approaches, the virtual agent can adapt its initiation strategy based on the detected user state, widget type, and conversation context, thereby improving adaptability while keeping the underlying system relatively simple.
2Productivity
If virtual agents initiate conversations based on static pre-programmed content, then the implementation is simple and development time is short, but the ability to dynamically respond to specific user situations is poor
Solution Approach 1:
The system implements feedback loops where the virtual agent continuously monitors user interactions with the widget, detects signs of difficulty (such as prolonged inactivity, repeated actions, or error states), and uses this feedback to dynamically select and initiate appropriate pre-programmed conversation statements. This feedback mechanism enables the agent to respond effectively to specific user situations without requiring complex generative AI capabilities.
Solution Approach 2:
The conversation initiation system transitions from static to dynamic by allowing the virtual agent to select different pre-programmed statements based on real-time user state detection. The system dynamically adjusts which pre-programmed content to use based on detected user difficulties, widget context, and conversation history, thereby improving assistance effectiveness while maintaining implementation simplicity through reuse of pre-prepared content.
3Loss of information
If virtual agents use pre-programmed conversation scripts, then the reliability of conversation flow is high, but the loss of information about specific user widget interactions is significant
Solution Approach 1:
The system segments the conversation initiation process into distinct components: (1) user state detection (monitoring widget interactions), (2) difficulty detection (analyzing interaction patterns), (3) context extraction (identifying specific widget elements), and (4) statement selection (choosing appropriate pre-programmed content). This segmentation allows the virtual agent to capture detailed information about user widget interactions and map it to specific conversation statements, reducing information loss while maintaining adaptability.
Solution Approach 2:
The system introduces an intermediary layer between user widget interactions and conversation initiation. This intermediary component analyzes user interactions with the widget, extracts relevant context information (such as which widget element the user is struggling with), and maps this information to appropriate pre-programmed conversation statements. This intermediary process preserves detailed information about user situations while enabling effective response through structured pre-programmed content.
Data Source
AI summary
A computer-implemented method, system and computer program product for initiating a conversation by a virtual agent. The boundary of a widget of a webpage the user is utilizing is determined. The user's interactions or lack of interactions with the widget within the determined boundary of the widget of the webpage is then determined, such as based on tracking the eye gaze or focus of the user. If it is determined that the user is experiencing difficulty in utilizing the widget, then the content associated with the widget is extracted. The extracted widget content is then mapped to the virtual agent content. The virtual agent is instructed to initiate a conversation based on the extracted widget content. In this manner, the virtual agent will now be able to dynamically initiate a conversation with the user to assist the user in addressing a problem the user is currently experiencing with the widget.