Two-way real-time emotion perception system

Abstract

This system utilizes a two-way, real-time emotion perception system to instantly obtain emotional indicators from both customers and customer service representatives. This information serves as a guide for customer service representatives in responding to customer issues and helps prevent the risk of customer complaints. Furthermore, when it is determined that the current customer service representative is unable to handle a customer's situation, a seamless connection can be established with another representative to provide the customer with a more comprehensive solution.

Claims

1. A two-way, real-time emotion perception system, comprising: Storage, storing multiple modules; The processor, coupled to the storage, is configured to: acquire real-time voice call data of the customer and customer service representatives; execute a channel separation module to segment the real-time voice call data into customer channel data and customer service channel data; execute a sentiment analysis module to acquire a customer sentiment index based on the customer channel data and a customer service sentiment index based on the customer service channel data; and predict a complaint risk value based on the customer sentiment index and the customer service sentiment index.

2. The two-way real-time emotion perception system as described in claim 1, wherein the processor is further configured to: execute a speech feature extraction module to extract customer prosodic features from the customer's voice channel data, and extract customer service prosodic features from the customer service voice channel data, wherein the customer prosodic features include customer fundamental frequency, customer energy, customer speech rate, and customer silence rate, wherein the customer service prosodic features include customer service fundamental frequency, customer service energy, customer service speech rate, and customer service silence rate; determine the customer's emotion index based on the customer prosodic features; and determine the customer service's emotion index based on the customer service prosodic features.

3. The two-way real-time emotion perception system as described in claim 2, wherein the processor is further configured to: classify the customer's emotions based on the customer emotion index, and classify the customer service representative's emotions based on the customer service representative's emotion index, wherein the emotion classification includes one of anger, dissatisfaction, anxiety, neutrality, calmness, and positivity.

4. The two-way real-time emotion sensing system as described in claim 3, wherein the processor is further configured to: push a script suggestion in response to the customer emotion index being greater than the customer emotion threshold or the customer service emotion index being greater than the customer service emotion threshold, wherein the script suggestion is relevant to the real-time voice call data.

5. The two-way real-time emotion perception system as described in claim 3, wherein the processor is further configured to: output customer emotion labels and customer emotion intensity based on the customer emotion classification; and output customer service emotion labels and customer service emotion intensity based on the customer service emotion classification.

6. The two-way real-time emotion sensing system as described in claim 5, wherein the processor is further configured to: calculate the rate of change of customer emotion based on the change of customer emotion intensity relative to time, and calculate the rate of change of customer service emotion based on the change of customer service emotion intensity relative to time.

7. The two-way real-time emotion perception system as described in claim 6, wherein the processor is further configured to: calculate an interaction tension index based on the customer's emotion intensity, the customer's emotion change rate, the customer service representative's emotion intensity, and the customer service representative's emotion change rate, wherein the interaction tension index is calculated using the following formula: ITS(t) = α×E_c(t) + β×E_a(t) + γ×max(ΔE_c(t), ΔE_a(t)), where t is time (seconds), ITS is the interaction tension index, E_c is the customer's emotion intensity, E_a is the customer service representative's emotion intensity, ΔE_c is the customer's emotion change rate, ΔE_a is the customer service representative's emotion change rate, max() is to take the maximum value, and α, β, and γ are weighting coefficients.

8. The bidirectional real-time emotion perception system as described in claim 2, wherein the processor is further configured to: execute the speech feature extraction module to calculate the customer prosodic feature and the customer service prosodic feature using a 2-second sliding window, wherein the customer prosodic feature and the customer service prosodic feature are updated every 500 milliseconds.

9. The two-way real-time emotion perception system as described in claim 1, wherein the delay time for predicting the appeal risk value does not exceed 1 second.

10. The two-way real-time emotion perception system as claimed in claim 1, wherein the processor is further configured to: acquire multiple historical call records and corresponding multiple claim records; label the historical call records with tension levels based on the claim records; train the emotion analysis module based on the historical call records and the claim records; and train the emotion analysis module with the goal of reducing the error between multiple interaction tension indices of the historical call records and whether a conflict actually occurred in the tension levels.

Images