AI Video Emotion Transcript Routing for Contact Centers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In contact centers, detecting micro-expressions and other non-verbal cues is challenging due to cultural differences, framing rates, compression ratios, and bandwidth issues, leading to increased customer dissatisfaction, longer call times, and reduced efficiency.
Innovation Solution
A system processes real-time video streams from customer-contact center agent interactions to generate emotion transcripts, tracking micro-expressions and macro-expressions, and compares these to previous calls to manage the video call effectively, potentially rerouting it to a different agent or supervisor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If video processing is used to detect micro-expressions, then emotional state detection accuracy is improved, but system complexity increases
Solution Approach 1:
An AI processor acts as an intermediary between the video stream and the contact center system. The AI processor receives video frames, detects micro-expressions and facial emotions, and generates emotion transcripts that are sent to the contact center system. This intermediary handles the complex video analysis tasks, keeping the main system architecture relatively simple while achieving high detection accuracy.
Solution Approach 2:
The patent replaces manual observation and interpretation of facial expressions with automated AI-based computer vision technology. Instead of relying on human agents or supervisors to visually assess customer emotions, the system uses machine learning models to automatically detect micro-expressions and generate emotion transcripts, substituting mechanical human perception with automated digital processing.
2Reliability
If real-time video processing is implemented, then customer dissatisfaction is reduced, but processing speed requirements increase system resource consumption
Solution Approach 1:
The system performs preliminary processing by generating emotion transcripts in real-time during the video call, rather than analyzing all video data after the call ends. The AI processor continuously analyzes video frames and generates emotion transcripts that are immediately available to the contact center system, allowing for real-time intervention while distributing processing load over time.
Solution Approach 2:
The patent extracts only the essential emotional information from video data by generating condensed emotion transcripts. Instead of processing and storing entire video streams for analysis, the system extracts key emotional states and expressions into structured transcript data, significantly reducing the amount of data that needs to be processed and stored while maintaining the ability to detect customer dissatisfaction.
3Productivity
If emotion transcripts are compared to previous calls for routing decisions, then call handling efficiency is improved, but data processing time increases
Solution Approach 1:
The system performs preliminary comparison of emotion transcripts with historical call data during or immediately after the video call. By comparing emotion patterns, detected expressions, and transcript data with stored historical calls in real-time or near-real-time, the system prepares routing recommendations before the call officially ends, reducing the time needed for post-call processing and enabling faster overall handling.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A video stream of a video call between a communication endpoint of customer and a communication endpoint of a contact center agent is received. The video stream of the video call is processed in real-time to generate a real-time emotion transcript. The real-time emotion transcript tracks a plurality of separate emotions based on non-verbal expressions (e.g. facial expressions) that occur in the video stream of the video call. For example, different emotions of both the customer and the contact center agent may tracked in the real-time emotion transcript. The real-time emotion transcript is compared to an emotion transcript of at least one previous video call to determine if the video call should be handled differently in the contact center. In response to determining that video call should be handled differently in the contact center, an action determined to change how the video call is managed in the contact center.