Dialogue processing method, device, terminal and storage medium based on emotion recognition

Abstract

The embodiment of the present application discloses a conversation processing method, device, terminal and storage medium based on emotion recognition. In the process of communication between an agent and a customer based on a business scenario, this solution collects the customer's first voice information and the agent's second voice information, identifies the customer's emotion based on the first voice information, and if the customer's emotion is negative, generates a conversation voice based on the first voice information and the second voice information according to the time point when the customer's negative emotion appears, then performs voice analysis on the conversation voice to obtain a voice analysis result, and performs text analysis on the conversation voice to obtain a text analysis result, and generates customer soothing strategy information based on the voice analysis result and the text analysis result, so that the agent can soothe the customer's emotion according to the customer soothing strategy information, thereby improving the customer's business experience.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a conversation processing method, device, terminal and storage medium based on emotion recognition. Background Art

[0002] With the rapid development of financial technology and the social economy, people's expectations for bank services are becoming increasingly high. Banks have specifically implemented measures to monitor the service quality of telephone agents. Real-time monitoring and alerting of emotional interactions between agents and customers during calls is one of the most effective and direct ways to improve customer experience and service levels.

[0003] In related technologies, during a call between a customer and an agent, when it is identified that the customer is emotionally unstable, the reasons affecting the customer's emotions cannot be promptly understood, and the customer cannot be comforted in time, resulting in the customer's service experience after the call cannot be effectively guaranteed. Summary of the Invention

[0004] The embodiments of the present application provide a conversation processing method, device, terminal and storage medium based on emotion recognition, which can improve the business experience of customer personnel.

[0005] This embodiment of the present application provides a method for processing a conversation based on emotion recognition, including:

[0006] Get the first voice of the customer and the second voice of the agent in the current business scenario;

[0007] extracting a first text feature and a first voice feature from the first speech, and identifying whether the customer has a negative emotion based on the first text feature and the first voice feature;

[0008] If the negative emotion occurs, obtaining a target time point at which the negative emotion occurs, and generating a conversation voice between the customer service representative and the agent according to the first voice and the second voice;

[0009] Extracting a second speech feature from the speech segment corresponding to the target time point in the conversation speech, identifying tone information of the agent and the customer representative during each round of conversation in the speech segment based on the second speech feature, and organizing the tone information into a tone information sequence based on the order of each round of conversation in the speech segment;

[0010] Obtaining a text segment corresponding to the speech segment from the conversation text corresponding to the conversation speech, performing conversation anomaly recognition on each round of conversation in the text segment, and obtaining a conversation anomaly recognition result corresponding to each round of conversation;

[0011] A dialogue adjustment strategy is generated based on the tone information sequence and the dialogue anomaly recognition result, and the dialogue adjustment strategy is presented to the agent.

[0012] Accordingly, an embodiment of the present application further provides a conversation processing device based on emotion recognition, comprising:

[0013] A first acquisition unit is used to acquire a first voice of a customer and a second voice of an agent in a current business scenario;

[0014] a first extraction unit, configured to extract a first text feature and a first voice feature from the first speech, and identify whether the customer has a negative emotion based on the first text feature and the first voice feature;

[0015] A second acquisition unit is configured to acquire a target time point at which the negative emotion occurs, and generate a conversation voice between the customer agent and the agent based on the first voice and the second voice;

[0016] a second extraction unit configured to extract a second speech feature from the speech segment corresponding to the target time point in the conversation speech, identify tone information of the agent and the customer representative during each round of conversation in the speech segment based on the second speech feature, and organize the tone information into a tone information sequence based on the order of each round of conversation in the speech segment;

[0017] a third acquisition unit, configured to acquire a text segment corresponding to the speech segment from the conversation text corresponding to the conversation speech, perform conversation anomaly recognition on each round of conversation in the text segment, and obtain a conversation anomaly recognition result corresponding to each round of conversation;

[0018] A generating unit is used to generate a dialogue adjustment strategy based on the tone information sequence and the dialogue anomaly recognition result, and present the dialogue adjustment strategy to the seat staff.

[0019] In some embodiments, the first extraction unit includes:

[0020] a first processing subunit, configured to perform recognition processing on the first text feature and the first voice feature using an emotion recognition model to obtain emotion scores corresponding to the first text feature and the first voice feature;

[0021] A first determining subunit is configured to determine that the customer has the negative emotion if the emotion score is less than a preset emotion score;

[0022] The second determining subunit is configured to determine that the customer representative does not have the negative emotion if the emotion score is not less than the preset emotion score.

[0023] In some embodiments, the second acquiring unit includes:

[0024] a first acquiring subunit, configured to acquire first time information corresponding to the first voice and second time information corresponding to the second voice;

[0025] The synthesis subunit is used to synthesize the first voice and the second voice based on the first time information and the second time information to obtain the conversation voice between the customer staff and the seat staff.

[0026] In some embodiments, the second extraction unit comprises:

[0027] a third determining subunit, configured to determine a target time range based on the target time point and a time point adjacent to the target time point in the conversational speech;

[0028] The extraction subunit is used to extract features from the speech segment corresponding to the target time range in the conversation speech to obtain the second speech feature.

[0029] In some embodiments, the second extraction unit comprises:

[0030] a second acquisition subunit, configured to acquire time points corresponding to each round of dialogue in the speech segment, and determine a dialogue order of each round of dialogue based on the time points corresponding to each round of dialogue;

[0031] The sorting subunit is used to sort the tone information corresponding to each round of dialogue based on the dialogue sequence to obtain the tone information sequence.

[0032] In some embodiments, the third acquiring unit includes:

[0033] The second processing sub-unit is used to perform text semantic recognition processing on each round of dialogue to obtain semantic information of each round of dialogue;

[0034] a fourth determining subunit, configured to determine that the conversation is an abnormal conversation if the semantic information meets a preset abnormal conversation condition;

[0035] The fifth determining subunit is configured to determine that the conversation is a normal conversation if the semantic information does not meet a preset abnormal conversation condition.

[0036] In some embodiments, the generating unit includes:

[0037] A sixth determining subunit is configured to determine target tone information corresponding to a negative tone of the customer from the tone information sequence;

[0038] a seventh determining subunit, configured to determine a target turn of dialogue corresponding to the target tone information from the dialogue segment;

[0039] A third acquisition subunit is used to obtain a dialogue anomaly recognition result corresponding to the target round dialogue;

[0040] A generating subunit is configured to generate customer soothing information based on the content of the target round dialogue if the dialogue anomaly identification result indicates an abnormal dialogue, and obtain the dialogue adjustment strategy.

[0041] Correspondingly, an embodiment of the present application also provides a terminal, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes any emotion recognition-based dialogue processing method provided in any embodiment of the present application.

[0042] Correspondingly, an embodiment of the present application further provides a storage medium storing a plurality of instructions, which are suitable for loading by a processor to execute the above-mentioned emotion recognition-based dialogue processing method.

[0043] The embodiment of the present application collects first voice information of the customer and second voice information of the agent during the communication between the agent and the customer based on the business scenario, identifies the customer's emotion based on the first voice information, and if the customer's emotion is negative, generates a conversational voice based on the first voice information and the second voice information according to the time point when the customer has negative emotions, then performs voice analysis on the conversational voice to obtain a voice analysis result, and performs text analysis on the conversational voice to obtain a text analysis result, and generates customer soothing strategy information based on the voice analysis result and the text analysis result, so that the agent can soothe the customer's emotion according to the customer soothing strategy information, thereby improving the customer's business experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 A flowchart of a conversation processing method based on emotion recognition provided in an embodiment of the present application.

[0046] Figure 2 A flowchart of another method for processing conversations based on emotion recognition provided in an embodiment of the present application.

[0047] Figure 3 This is a structural block diagram of a conversation processing device based on emotion recognition provided in an embodiment of the present application.

[0048] Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of this application.

[0050] The embodiments of the present application provide a method, device, storage medium, and terminal for processing a conversation based on emotion recognition. Specifically, the method for processing a conversation based on emotion recognition in the embodiments of the present application can be executed by a terminal, wherein the terminal can be a device such as a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0051] For example, the terminal can be a server, which can obtain the first voice of a customer and the second voice of an agent in the current business scenario; extract the first text feature and the first voice feature of the first voice, and identify whether the customer has negative emotions based on the first text feature and the first voice feature; if negative emotions appear, obtain the target time point when the negative emotions appear, and generate a conversation voice between the customer and the agent based on the first voice and the second voice; extract the second voice feature of the voice segment corresponding to the target time point in the conversation voice, identify the tone information of the agent and the customer in each round of conversation in the voice segment based on the second voice feature, and organize the tone information into a tone information sequence based on the order of each round of conversation in the voice segment; obtain the text segment corresponding to the voice segment from the conversation text corresponding to the conversation voice, perform conversation anomaly recognition on each round of conversation in the text segment, and obtain the conversation anomaly recognition result corresponding to each round of conversation; generate a conversation adjustment strategy based on the tone information sequence and the conversation anomaly recognition result, and display the conversation adjustment strategy to the agent.

[0052] Based on the above problems, the embodiments of the present application provide a conversation processing method, device, terminal and storage medium based on emotion recognition, which can improve the business experience of customer personnel.

[0053] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0054] An embodiment of the present application provides a conversation processing method based on emotion recognition, which can be executed by a terminal or a server. The embodiment of the present application takes the conversation processing method based on emotion recognition executed by a server as an example for explanation.

[0055] See also Figure 1 , Figure 1 This is a flow chart of a method for processing a conversation based on emotion recognition provided by an embodiment of the present application. The specific process of the method for processing a conversation based on emotion recognition can be as follows:

[0056] 101. Obtain a first voice of a customer service representative and a second voice of an agent in a current business scenario.

[0057] In the embodiments of this application, a business scenario refers to the interaction between a service provider and a customer. For example, a business scenario may include: business processing, business consultation, business recommendation, etc. The service provider refers to the party that can provide business services to users, such as a bank. The agent refers to the staff of the service provider, such as a salesperson or customer service representative.

[0058] The first voice refers to the audio voice of the customer service representative, and the second voice refers to the audio voice of the agent. Specifically, the first voice of the customer service representative and the second voice of the agent are collected by a voice collection device.

[0059] In some embodiments, in order to accurately distinguish the voices of the customer staff and the agent staff, the voices of the customer staff and the agent staff can be collected respectively by different voice collection devices.

[0060] For example, when an agent communicates with a customer based on the current business scenario, the first voice acquisition device collects the customer's voice to obtain a first voice; and the second voice acquisition device collects the agent's voice to obtain a second voice.

[0061] 102. Extract a first text feature and a first voice feature from the first speech, and identify whether the customer has negative emotions based on the first text feature and the first voice feature.

[0062] Specifically, the voice information of the customer person is obtained from the first voice, and then the voice information is converted into text information.

[0063] Among them, voice information can be converted into text information through ASR (Automatic Speech Recognition) technology, which is a technology that converts human speech into text.

[0064] For example, the acquired voice information is converted into text using ASR technology to obtain text information corresponding to the voice information.

[0065] In an embodiment of the present application, feature extraction is performed on text information using a text feature extraction model, and feature extraction is performed on voice information using a voice feature extraction model.

[0066] For example, the text information is input into the text feature extraction model, and the text feature extraction model is used to extract features of the text information to obtain the first text feature of the text information; the voice information is input into the voice feature extraction model, and the voice feature extraction model is used to extract features of the voice information to obtain the first voice feature of the voice information.

[0067] The speech feature extraction model is based on Meta's open-source speech pre-training model, XLSR-53. XLSR-53 is a multilingual speech pre-training model developed by Meta using the self-supervised speech pre-training model wav2vec2 (an open-source speech pre-training model developed by Meta). The model was trained on 56,000 hours of data in 53 languages.

[0068] Training the text feature extraction model involves using the publicly available Chinese text pre-trained model, Chinese-BERT-wwm (the open-source Chinese BERT model), as the initial model. The Chinese-BERT-wwm model is trained for 60 epochs using only the text information from the sentiment training data. The Adam optimizer is used with an exponential decay learning rate of 0.00001. The model's loss function remains the same as that of the initial Chinese-BERT-wwm model.

[0069] In some embodiments, to improve the accuracy of emotion recognition, the step of “identifying whether the customer has a negative emotion based on the first text feature and the first voice feature” may include the following operations:

[0070] Recognize the first text feature and the first voice feature using an emotion recognition model to obtain emotion scores corresponding to the first text feature and the first voice feature;

[0071] If the emotion score is less than the preset emotion score, it is determined that the customer has negative emotions;

[0072] If the emotion score is not less than the preset emotion score, it is determined that the customer service representative has no negative emotions.

[0073] Among them, negative sentiment indicates that the customer staff is in a bad mood.

[0074] In an embodiment of the present application, emotion recognition processing is performed on the first text feature and the first speech feature through an emotion recognition model.

[0075] Specifically, the first text feature and the first speech feature are input into the emotion recognition model. The emotion recognition model then performs feature fusion processing on the first text feature and the first speech feature to obtain fused feature information of the first text feature and the first speech feature. The emotion recognition model then calculates the emotion score corresponding to the fused feature information to obtain the emotion score of the current customer. For example, the emotion score can range from 0 to 100, with a higher emotion score indicating a better emotion.

[0076] The preset emotion score can be used to determine whether the customer service representative has negative emotions. For example, if the customer service representative's emotion score is less than the preset emotion score, it can be determined that the customer service representative has negative emotions; if the customer service representative's emotion score is greater than or equal to the preset emotion score, it can be determined that the customer service representative has no negative emotions.

[0077] For example, the emotion score of a customer identified by the emotion recognition model may be 40 points, and the preset emotion score may be 50 points. By comparing the emotion score with the preset emotion score, if the emotion score is less than the preset emotion score, it can be determined that the customer has a negative emotion.

[0078] For another example, the emotion score of the customer identified by the emotion recognition model may be 80 points, and the preset emotion score may be 50 points. By comparing the emotion score with the preset emotion score, if the emotion score is greater than the preset emotion score, it can be determined that the customer does not have a negative emotion.

[0079] 103. If negative emotions occur, a target time point at which the negative emotions occur is obtained, and a conversation voice between the customer service representative and the agent is generated based on the first voice and the second voice.

[0080] The target time point refers to the time point corresponding to the voice information and text information on the time axis of the first voice when the customer is identified to have negative emotions based on the voice information and text information of the first voice.

[0081] For example, the start time of the first speech can be: 12:08:09, and the end time can be: 12:08:39. The duration of the first speech is 30 seconds, and the time axis corresponding to the first speech is: 12:08:09-12:08:39. Emotion recognition is performed on the voice information of the first speech and the text information corresponding to the voice information. If a negative emotion is recognized, the time point in the first speech corresponding to the text information of the voice information at which the negative emotion appears is obtained. For example, it can be: 12:08:29. The target time point can be determined as: 12:08:29.

[0082] In some embodiments, the step of "generating a conversation voice between the customer agent and the agent based on the first voice and the second voice" may include the following operations:

[0083] Obtaining first time information corresponding to the first voice and second time information corresponding to the second voice;

[0084] Based on the first time information and the second time information, the first voice and the second voice are synthesized to obtain the conversation voice between the customer staff and the seat staff.

[0085] The first time information refers to the time axis corresponding to the first voice, and the second voice information refers to the time axis corresponding to the second voice.

[0086] For example, the time axis corresponding to the first voice may be: 12:08:09-12:08:39, and the time axis corresponding to the second voice may be: 12:08:12-12:08:45.

[0087] Furthermore, according to the time axis of the first voice and the time axis of the second voice, the first voice and the second voice are cross-synthesized in chronological order to generate the conversation voice between the customer staff and the seat staff.

[0088] For example, the first speech may include speech segments: a first speech segment, a second speech segment, and a third speech segment, wherein the time point corresponding to the first speech segment on the time axis of the first speech may be: 12:8:09, the time point corresponding to the second speech segment on the time axis of the first speech may be: 12:8:16, and the time point corresponding to the third speech segment on the time axis of the first speech may be: 12:8:30. The second speech may include speech segments: a fourth speech segment, a fifth speech segment, and a sixth speech segment, wherein the time point corresponding to the fourth speech segment on the time axis of the second speech may be: 12:8:12, the time point corresponding to the fifth speech segment on the time axis of the second speech may be: 12:8:20, and the time point corresponding to the fifth speech segment on the time axis of the second speech may be: 12:8:35. The first speech and the second speech are synthesized based on the order of the time points on the time axis, that is, the time periods in the first speech and the second speech are sorted, and the resulting dialogue speech includes: the first speech segment, the fourth speech segment, the second speech segment, the fifth speech segment, the third speech segment, and the sixth speech segment.

[0089] 104. Extract a second speech feature from the speech segment corresponding to the target time point in the conversation speech, identify the tone information of the agent and the customer staff in each round of conversation in the speech segment based on the second speech feature, and organize the tone information into a tone information sequence based on the order of each round of conversation in the speech segment.

[0090] Specifically, a speech segment corresponding to a target time point is obtained from the conversation speech, the speech segment is input into a speech feature extraction model, speech features are extracted from the speech segment by the speech feature extraction model, and a second speech feature corresponding to the speech segment is obtained.

[0091] In some embodiments, to accurately analyze the reasons for the customer's negative emotions, the step of "extracting a second speech feature from the speech segment corresponding to the target time point in the conversation speech" may include the following operations:

[0092] Determining a target time range based on the target time point and time points adjacent to the target time point in the conversational speech;

[0093] Feature extraction is performed on the speech segment corresponding to the target time range in the conversation speech to obtain a second speech feature.

[0094] Specifically, the target time point and adjacent time points of the target time point may be selected from the time axis of the conversational speech to obtain the target time range.

[0095] The time axis of the conversational speech may be a combination of the time axis of the first speech and the time axis of the second speech, that is, the time axis of the conversational speech is the union of the time axis of the first speech and the time axis of the second speech.

[0096] For example, the time axis corresponding to the first voice can be: 12 hours 8 minutes 9 seconds - 12 hours 8 minutes 39 seconds, the time axis corresponding to the second voice can be: 12 hours 8 minutes 12 seconds - 12 hours 8 minutes 45 seconds, and the time axis of the dialogue voice can be: 12 hours 8 minutes 9 seconds - 12 hours 8 minutes 45 seconds.

[0097] Furthermore, the target time range is determined based on the time axis of the conversational speech. For example, the target time point may be 12:08:29. The adjacent time points of the target time point obtained from the time axis of 12:08:09 to 12:08:45 may be 12:08:28 and 12:08:30. Therefore, the target time range may be 12:08:28 to 12:08:30.

[0098] Specifically, a speech segment corresponding to a target time range is obtained from the conversation speech, and then feature extraction is performed on the speech segment corresponding to the target time range to obtain the second speech feature.

[0099] The voice segment may include at least one round of dialogue between the customer service representative and the agent. One round of dialogue includes one sentence of voice from the customer service representative and one sentence of voice from the agent, that is, one dialogue turn.

[0100] Specifically, the voice segment is processed for tone recognition to obtain the tone of the customer and the tone of the agent in each round of conversation in the voice segment, thereby obtaining tone information.

[0101] In the embodiment of the present application, the tone can be represented by a tone type. For example, the tone type may include: poor tone, medium tone, good tone, etc.

[0102] For example, a speech segment includes: the first round of conversation, the second round of conversation, and the third round of conversation. Tone recognition is performed on each round of conversation, and the tone recognition result can be: in the first round of conversation, the tone of the customer service representative is good, and the tone of the agent is good; in the second round of conversation, the tone of the customer service representative is good, and the tone of the agent is good; in the third round of conversation, the tone of the customer service representative is bad, and the tone of the agent is good, thereby obtaining the voice information of the speech segment.

[0103] In some embodiments, to determine the exact reason why the customer exhibits negative emotions, the step of "combining the tone information into a tone information sequence based on the order of each round of dialogue in the voice segment" may include the following operations:

[0104] Obtaining the time points corresponding to each round of dialogue in the speech segment, and determining the dialogue order of each round of dialogue based on the time points corresponding to each round of dialogue;

[0105] The tone information corresponding to each round of dialogue is sorted based on the dialogue order to obtain a tone information sequence.

[0106] For example, the speech segment includes: a first round of dialogue, a second round of dialogue, and a third round of dialogue, wherein the time point corresponding to the first round of dialogue on the time axis of the speech segment may be: 12 hours, 8 minutes, 28 seconds, the time point corresponding to the second round of dialogue on the time axis of the speech segment may be: 12 hours, 8 minutes, 30 seconds, and the time point corresponding to the third round of dialogue on the time axis of the speech segment may be: 12 hours, 8 minutes, 35 seconds. Furthermore, each round of dialogue is sorted according to the sequence of the time points corresponding to each round of dialogue, and the resulting dialogue order may be: first round of dialogue, second round of dialogue, and third round of dialogue.

[0107] Furthermore, the tone information of each round of dialogue is sorted according to the dialogue order of each round of dialogue, and the tone information sequence after sorting is obtained as: the tone information of the first round of dialogue, the tone information of the second round of dialogue, and the tone information of the third round of dialogue.

[0108] 105. Obtain a text segment corresponding to the speech segment from the conversation text corresponding to the conversation speech, perform conversation anomaly recognition on each round of conversation in the text segment, and obtain a conversation anomaly recognition result corresponding to each round of conversation.

[0109] First, convert the conversational speech into text. This can be processed using ASR technology to convert the speech into text. The text segments within the conversation are obtained, i.e., the text corresponding to different points in time. Each segment should contain at least one conversation between the customer representative and the agent.

[0110] In some embodiments, to accurately analyze the causes of negative emotions among customers, the step of "identifying conversation anomalies for each round of conversation in the text segment and obtaining conversation anomaly identification results corresponding to each round of conversation" may include the following operations:

[0111] Perform text semantic recognition processing on each round of dialogue to obtain the semantic information of each round of dialogue;

[0112] If the semantic information meets the preset abnormal dialogue conditions, the dialogue is determined to be an abnormal dialogue;

[0113] If the semantic information does not meet the preset abnormal conversation conditions, the conversation is determined to be a normal conversation.

[0114] Specifically, semantic recognition is performed on the text content of each round of dialogue to obtain the semantic information corresponding to each round of dialogue.

[0115] The preset abnormal conversation conditions are used to determine whether the conversation between the customer service representative and the agent contains any abnormal content. Abnormal content can be preset, indicating content that could cause negative emotions in the customer service representative, such as excessive language. The preset abnormal conversation conditions are not limited to the aforementioned abnormal content and can be set differently based on specific business scenarios, and are not limited here.

[0116] Specifically, the semantic information of each round of conversation is compared with the preset abnormal conversation conditions. If the semantic information meets the preset abnormal conversation conditions, it can be determined that the conversation between the customer staff and the agent is an abnormal conversation; if the semantic information does not meet the preset abnormal conversation conditions, it can be determined that the conversation between the customer staff and the agent is a normal conversation.

[0117] 106. Generate a dialogue adjustment strategy based on the tone information sequence and dialogue anomaly recognition results, and display the dialogue adjustment strategy to the agent.

[0118] In some embodiments, to improve the customer service experience, the step of "generating a dialogue adjustment strategy based on the tone information sequence and the dialogue anomaly recognition result" may include the following operations:

[0119] Determine the target tone information corresponding to the negative tone of the customer personnel from the tone information sequence;

[0120] Determine the target turn of dialogue corresponding to the target tone information from the dialogue segment;

[0121] Obtain the dialogue anomaly recognition results corresponding to the target dialogue round;

[0122] If the conversation anomaly recognition result indicates an abnormal conversation, customer comfort information is generated based on the content of the target round of conversation to obtain a conversation adjustment strategy.

[0123] Among them, negative tone refers to the tone type: poor tone.

[0124] For example, a tone information sequence includes the tone information of the first round of conversation, the tone information of the second round of conversation, and the tone information of the third round of conversation. In the first round of conversation, the customer service representative's tone was good, and the agent's tone was good; in the second round of conversation, the customer service representative's tone was good, and the agent's tone was good; in the third round of conversation, the customer service representative's tone was bad, and the agent's tone was good. Based on the speech information of the speech segment, the target tone information with a negative tone can be determined to be the tone information of the third round of conversation, and the target round of conversation can be determined to be the third round of conversation.

[0125] Furthermore, the conversation anomaly recognition result of the text content of the third round of conversation is obtained. If the conversation anomaly recognition result indicates an abnormal conversation, it is determined whether the conversation text that meets the preset abnormal conversation conditions in the text content of the third round of conversation is the conversation text corresponding to the agent. If so, corrections are made based on the conversation text, and customer comfort information is generated, thereby obtaining a conversation adjustment strategy so that the agent can understand their own problems in a timely manner and correct them in a timely manner to provide customers with a better business experience.

[0126] In some embodiments, it is first possible to determine from the tone information sequence whether there is tone information showing that the agent has a negative tone. If there is tone information showing that the agent has a negative tone, tone correction prompt information can be generated and displayed to the agent, so that the agent can maintain a good tone during the communication process with the customer.

[0127] In an embodiment of the present application, there may be multiple ways to generate a dialogue adjustment strategy based on a tone information sequence and dialogue anomaly recognition results. Specifically, different dialogue adjustment strategies are generated based on different tone types in a tone information sequence and different dialogue anomaly recognition results. The purpose is to avoid negative emotions in the customer during the conversation, so that the agent can comfort the customer in a timely manner according to the dialogue adjustment strategy, so that the customer can maintain a good mood.

[0128] The present application discloses a method for processing conversations based on emotion recognition, the method comprising: obtaining a first speech of a customer and a second speech of an agent in a current business scenario; extracting a first text feature and a first speech feature from the first speech, and identifying whether the customer exhibits negative emotions based on the first text feature and the first speech feature; if negative emotions are present, obtaining a target time point at which the negative emotions are present, and generating conversational speech between the customer and the agent based on the first speech and the second speech; extracting a second speech feature from a speech segment corresponding to the target time point in the conversational speech, identifying tone information of the agent and the customer during each round of conversation in the speech segment based on the second speech feature, and forming a tone information sequence based on the order of each round of conversation in the speech segment; obtaining a text segment corresponding to the speech segment from the conversational text corresponding to the conversational speech, performing conversation anomaly identification on each round of conversation in the text segment, and obtaining conversation anomaly identification results corresponding to each round of conversation; generating a conversation adjustment strategy based on the tone information sequence and the conversation anomaly identification results, and presenting the conversation adjustment strategy to the agent. In this way, the efficiency of conversation processing based on emotion recognition in the service process can be improved.

[0129] Based on the above introduction, the following examples will further illustrate the conversation processing method based on emotion recognition in this application. Figure 2 , Figure 2 This is a flow chart of another method for processing a conversation based on emotion recognition provided in an embodiment of the present application. Taking the method for processing a conversation based on emotion recognition as an example, the specific process may be as follows:

[0130] 201. During a business scenario conversation between an agent and a customer, the server obtains a first voice of the agent and a second voice of the agent during the conversation.

[0131] In an embodiment of the present application, the business scenario may be a banking scenario. When an agent is communicating with a customer, the server may collect the agent's voice using a first voice collection device to obtain a first voice, and collect the customer's voice using a second voice collection device to obtain a second voice. By using different voice collection devices to separately collect the customer's and agent's voices, it is possible to easily distinguish between the customer's and agent's voices, and thus process the voices of different individuals differently.

[0132] 202. If the server recognizes that the customer has negative emotions based on the voice information and text information of the first voice, the target time point at which the negative emotions appear is obtained.

[0133] First, voice information is acquired from the first voice, and then text conversion is performed on the voice information to obtain text information corresponding to the voice information.

[0134] Furthermore, the text information is input into a text feature extraction model, and the text information is subjected to feature extraction by the text feature extraction model to obtain a first text feature of the text information; the voice information is input into a voice feature extraction model, and the voice information is subjected to feature extraction by the voice feature extraction model to obtain a first voice feature of the voice information, and then the first text feature and the first voice feature are recognized and processed by the emotion recognition model to obtain emotion scores corresponding to the first text feature and the first voice feature. If the emotion score is less than the preset emotion score, it is determined that the customer has negative emotions; if the emotion score is not less than the preset emotion score, it is determined that the customer does not have negative emotions.

[0135] 203. The server generates a conversation voice based on the first voice and the second voice, and obtains a conversation text corresponding to the target time point from the conversation voice.

[0136] Specifically, the first voice of the customer and the second voice of the agent are synthesized into the conversation voice between the customer and the agent according to the time axis of the first voice and the time axis of the second voice. The specific synthesis method can be found in the above embodiment and will not be elaborated here.

[0137] The target time point refers to the time point on the timeline of the conversation voice when the customer shows negative emotions.

[0138] Specifically, a speech segment corresponding to a target time point is obtained from the conversation speech, and speech recognition processing is performed on the speech segment to obtain a text corresponding to the speech segment, that is, a conversation text.

[0139] 204. The server performs semantic recognition processing on the conversation text to obtain semantic information of the conversation text.

[0140] After obtaining the conversation text corresponding to the target time point, semantic recognition may be performed on the conversation text to identify semantic information of the conversation text.

[0141] 205. If the semantic information meets the conversation abnormality condition, the server generates customer soothing information based on the text content of the conversation text and displays the customer soothing information to the agent.

[0142] The abnormal conversation condition may be: abnormal content appears in the conversation content, and the abnormal content may be preset text content that may cause negative emotions in the customer.

[0143] If the semantic information meets the abnormal conversation conditions, that is, if it contains abnormal content, the customer service representative's speech can be determined to contain abnormal content. Furthermore, the conversation text corresponding to the abnormal content is corrected to obtain customer reassurance information. The reassurance information is then displayed on the agent's terminal, allowing the agent to calm the customer based on the reassurance information, ensuring a positive customer service experience.

[0144] An embodiment of the present application discloses a conversation processing method based on emotion recognition, the method comprising: during a business scenario conversation between an agent and a customer, a server respectively obtains a first voice of the agent and a second voice of the agent during the conversation; if the server recognizes that the customer has negative emotions based on the voice information and text information of the first voice, the target time point at which the negative emotions occur is obtained; a conversation voice is generated based on the first voice and the second voice, and a conversation text corresponding to the target time point is obtained from the conversation voice; semantic recognition processing is performed on the conversation text to obtain semantic information of the conversation text; if the semantic information meets the conversation abnormality condition, the server generates customer soothing information based on the text content of the conversation text, and displays the customer soothing information to the agent, which can facilitate the agent to soothe the customer's emotions in real time, thereby improving the customer's business experience.

[0145] To facilitate better implementation of the emotion recognition-based conversation processing method provided in the embodiments of this application, the embodiments of this application also provide an emotion recognition-based conversation processing device based on the emotion recognition-based conversation processing method. The meanings of the terms herein are the same as those in the emotion recognition-based conversation processing method. For specific implementation details, please refer to the description in the method embodiment.

[0146] See also Figure 3 , Figure 3 This is a structural block diagram of a conversation processing device based on emotion recognition provided in an embodiment of the present application, which includes:

[0147] The first acquisition unit 301 is configured to acquire a first voice of a customer and a second voice of an agent in a current business scenario;

[0148] A first extraction unit 302 is configured to extract a first text feature and a first voice feature from the first speech, and identify whether the customer has a negative emotion based on the first text feature and the first voice feature;

[0149] The second acquisition unit 303 is configured to acquire a target time point at which the negative emotion occurs, and generate a conversation voice between the customer agent and the agent based on the first voice and the second voice;

[0150] A second extraction unit 304 is configured to extract a second speech feature from the speech segment corresponding to the target time point in the conversation speech, identify tone information of the agent and the customer representative during each round of conversation in the speech segment based on the second speech feature, and organize the tone information into a tone information sequence based on the order of each round of conversation in the speech segment;

[0151] The third acquisition unit 305 is configured to acquire a text segment corresponding to the speech segment from the conversation text corresponding to the conversation speech, perform conversation anomaly recognition on each round of conversation in the text segment, and obtain a conversation anomaly recognition result corresponding to each round of conversation;

[0152] The generating unit 306 is configured to generate a dialogue adjustment strategy based on the tone information sequence and the dialogue anomaly recognition result, and present the dialogue adjustment strategy to the agent.

[0153] In some embodiments, the first extraction unit 302 may include:

[0154] a first processing subunit, configured to perform recognition processing on the first text feature and the first voice feature using an emotion recognition model to obtain emotion scores corresponding to the first text feature and the first voice feature;

[0155] A first determining subunit is configured to determine that the customer has the negative emotion if the emotion score is less than a preset emotion score;

[0156] The second determining subunit is configured to determine that the customer representative does not have the negative emotion if the emotion score is not less than the preset emotion score.

[0157] In some embodiments, the second obtaining unit 303 may include:

[0158] a first acquiring subunit, configured to acquire first time information corresponding to the first voice and second time information corresponding to the second voice;

[0159] The synthesis subunit is used to synthesize the first voice and the second voice based on the first time information and the second time information to obtain the conversation voice between the customer staff and the seat staff.

[0160] In some embodiments, the second extraction unit 304 may include:

[0161] a third determining subunit, configured to determine a target time range based on the target time point and a time point adjacent to the target time point in the conversational speech;

[0162] The extraction subunit is used to extract features from the speech segment corresponding to the target time range in the conversation speech to obtain the second speech feature.

[0163] In some embodiments, the second extraction unit 304 may include:

[0164] a second acquisition subunit, configured to acquire time points corresponding to each round of dialogue in the speech segment, and determine a dialogue order of each round of dialogue based on the time points corresponding to each round of dialogue;

[0165] The sorting subunit is used to sort the tone information corresponding to each round of dialogue based on the dialogue sequence to obtain the tone information sequence.

[0166] In some embodiments, the third obtaining unit 305 may include:

[0167] The second processing sub-unit is used to perform text semantic recognition processing on each round of dialogue to obtain semantic information of each round of dialogue;

[0168] a fourth determining subunit, configured to determine that the conversation is an abnormal conversation if the semantic information meets a preset abnormal conversation condition;

[0169] The fifth determining subunit is configured to determine that the conversation is a normal conversation if the semantic information does not meet a preset abnormal conversation condition.

[0170] In some embodiments, the generating unit 306 may include:

[0171] A sixth determining subunit is configured to determine target tone information corresponding to a negative tone of the customer from the tone information sequence;

[0172] a seventh determining subunit, configured to determine a target turn of dialogue corresponding to the target tone information from the dialogue segment;

[0173] A third acquisition subunit is used to obtain a dialogue anomaly recognition result corresponding to the target round dialogue;

[0174] A generating subunit is configured to generate customer soothing information based on the content of the target round dialogue if the dialogue anomaly identification result indicates an abnormal dialogue, and obtain the dialogue adjustment strategy.

[0175] The embodiment of the present application discloses a dialogue processing device based on emotion recognition, wherein a first acquisition unit 301 acquires a first voice of a customer in a current business scenario and a second voice of an agent, a first extraction unit 302 is used to extract a first text feature and a first voice feature from the first voice, and identify whether the customer has a negative emotion based on the first text feature and the first voice feature, a second acquisition unit 303 acquires a target time point for the occurrence of the negative emotion if the negative emotion occurs, and generates a dialogue voice between the customer and the agent based on the first voice and the second voice, and a second extraction unit 304 extracts a first text feature and a first voice feature from the dialogue voice. A second speech feature is extracted from the speech segment corresponding to the target time point. The tone information of each conversation between the agent and the customer representative in the speech segment is identified based on the second speech feature. The tone information is then organized into a tone information sequence based on the order of each conversation in the speech segment. A third acquisition unit 305 acquires a text segment corresponding to the speech segment from the conversation text corresponding to the speech, identifies conversation anomalies for each conversation in the text segment, and obtains conversation anomaly identification results corresponding to each conversation. A generation unit 306 generates a conversation adjustment strategy based on the tone information sequence and the conversation anomaly identification results, and presents the conversation adjustment strategy to the agent. This improves the customer service experience.

[0176] Correspondingly, the embodiment of the present application also provides a terminal, which may be a server. Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal 400 includes a processor 401 having one or more processing cores, a memory 402 having one or more computer-readable storage media, and a computer program stored in the memory 402 and executable on the processor. The processor 401 is electrically connected to the memory 402. Those skilled in the art will appreciate that the terminal structure shown in the figure does not constitute a limitation of the terminal, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0177] The processor 401 is the control center of the terminal 400. It uses various interfaces and lines to connect the various parts of the entire terminal 400. By running or loading software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, it executes various functions of the terminal 400 and processes data, thereby monitoring the terminal 400 as a whole.

[0178] In the embodiment of the present application, the processor 401 in the terminal 400 loads instructions corresponding to one or more application processes into the memory 402 according to the following steps, and the processor 401 runs the application stored in the memory 402 to implement various functions:

[0179] Get the first voice of the customer and the second voice of the agent in the current business scenario;

[0180] extracting a first text feature and a first voice feature from the first speech, and identifying whether the customer has a negative emotion based on the first text feature and the first voice feature;

[0181] If negative emotions occur, the target time point at which the negative emotions occur is obtained, and the conversation voice between the customer service representative and the agent is generated based on the first voice and the second voice;

[0182] Extracting a second speech feature from the speech segment corresponding to the target time point in the conversational speech, identifying the tone information of the agent and the customer representative during each round of conversation in the speech segment based on the second speech feature, and organizing the tone information into a tone information sequence based on the order of each round of conversation in the speech segment;

[0183] Obtaining a text segment corresponding to the speech segment from the conversation text corresponding to the conversation speech, performing conversation anomaly recognition on each round of conversation in the text segment, and obtaining a conversation anomaly recognition result corresponding to each round of conversation;

[0184] Generate a dialogue adjustment strategy based on the tone information sequence and dialogue anomaly recognition results, and display the dialogue adjustment strategy to the agent.

[0185] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0186] Optional, such as Figure 4 As shown, the terminal 400 further includes: a touch screen 403, a radio frequency circuit 404, an audio circuit 405, an input unit 406, and a power supply 407. Among them, the processor 401 is electrically connected to the touch screen 403, the radio frequency circuit 404, the audio circuit 405, the input unit 406, and the power supply 407 respectively. Those skilled in the art will understand that Figure 4 The terminal structure shown in the figure does not constitute a limitation to the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0187] The touch screen display 403 can be used to display a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. The touch screen display 403 may include a display panel and a touch panel. The display panel can be used to display messages input by the user or messages provided to the user, as well as various graphical user interfaces of the terminal. These graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. The touch panel can be used to collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any other suitable object or accessory on or near the touch panel), generate corresponding operation instructions, and the operation instructions execute corresponding programs. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch direction, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch message from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 401, and can receive the command sent by the processor 401 and execute it. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 401 to determine the type of touch event, and then the processor 401 provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 403 to realize input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize input and output functions. That is, the touch display screen 403 can also be used as part of the input unit 406 to realize the input function.

[0188] In this embodiment of the present application, processor 401 executes a game application to generate a graphical user interface (GUI) on touchscreen display 403. The virtual scene on the GUI includes at least one skill control area, which includes at least one skill control. Touchscreen display 403 is used to present the GUI and receive user commands generated by operating the GUI.

[0189] The radio frequency circuit 404 may be used to transmit and receive radio frequency signals, so as to establish wireless communication with a network device or other terminals through wireless communication, and to transmit and receive signals with the network device or other terminals.

[0190] Audio circuit 405 can be used to provide an audio interface between the user and the terminal through a speaker and microphone. Audio circuit 405 can convert received audio data into electrical signals and transmit them to the speaker, which then converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuit 405 and converted into audio data. The audio data is then output to processor 401 for processing, then transmitted via RF circuit 404 to, for example, another terminal, or to memory 402 for further processing. Audio circuit 405 may also include an earphone jack to allow communication between an external headset and the terminal.

[0191] The input unit 406 may be configured to receive input digital, character messages, or user feature messages (such as fingerprint, iris, or facial messages), and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0192] Power supply 407 is used to supply power to various components of terminal 400. Optionally, power supply 407 can be logically connected to processor 401 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. Power supply 407 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0193] although Figure 4 Not shown in the figure, the terminal 400 may further include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0194] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0195] As can be seen from the above, the terminal provided in this embodiment can obtain the first voice of the customer and the second voice of the agent in the current business scenario; extract the first text feature and the first voice feature of the first voice, and identify whether the customer has negative emotions based on the first text feature and the first voice feature; if negative emotions occur, obtain the target time point when the negative emotions occur, and generate the conversation voice between the customer and the agent based on the first voice and the second voice; extract the second voice feature of the voice segment corresponding to the target time point in the conversation voice, identify the tone information of the agent and the customer in each round of conversation in the voice segment based on the second voice feature, and form a tone information sequence based on the order of each round of conversation in the voice segment; obtain the text segment corresponding to the voice segment from the conversation text corresponding to the conversation voice, perform conversation anomaly recognition on each round of conversation in the text segment, and obtain the conversation anomaly recognition result corresponding to each round of conversation; generate a conversation adjustment strategy based on the tone information sequence and the conversation anomaly recognition result, and display the conversation adjustment strategy to the agent.

[0196] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0197] To this end, an embodiment of the present application provides a computer-readable storage medium storing a plurality of computer programs, which can be loaded by a processor to execute the steps of any of the emotion recognition-based conversation processing methods provided in the embodiments of the present application. For example, the computer program can execute the following steps:

[0198] Get the first voice of the customer and the second voice of the agent in the current business scenario;

[0199] Extracting a first text feature and a first voice feature from the first speech, and identifying whether the customer has a negative emotion based on the first text feature and the first voice feature;

[0200] If negative emotions occur, the target time point at which the negative emotions occur is obtained, and a conversation voice between the customer service representative and the agent is generated based on the first voice and the second voice;

[0201] Extracting a second speech feature from the speech segment corresponding to the target time point in the conversational speech, identifying the tone information of the agent and the customer representative during each round of conversation in the speech segment based on the second speech feature, and organizing the tone information into a tone information sequence based on the order of each round of conversation in the speech segment;

[0202] Obtaining a text segment corresponding to the speech segment from the conversation text corresponding to the conversation speech, performing conversation anomaly recognition on each round of conversation in the text segment, and obtaining a conversation anomaly recognition result corresponding to each round of conversation;

[0203] Generate a dialogue adjustment strategy based on the tone information sequence and dialogue anomaly recognition results, and display the dialogue adjustment strategy to the agent.

[0204] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0205] The storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0206] Since the computer program stored in the storage medium can execute the steps of any one of the dialogue processing methods based on emotion recognition provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the dialogue processing methods based on emotion recognition provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0207] The above is a detailed introduction to the conversation processing method, device, storage medium and terminal based on emotion recognition provided by the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0208] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the relevant information or data such as "face messages" involved in this disclosure are all obtained with full authorization.

Claims

1. A conversation processing method based on emotion recognition, characterized in that: The method comprises: Get the first voice of the customer and the second voice of the agent in the current business scenario; extracting a first text feature and a first voice feature from the first speech, and identifying whether the customer has a negative emotion based on the first text feature and the first voice feature; If the negative emotion occurs, obtaining a target time point at which the negative emotion occurs, and generating a conversation voice between the customer service representative and the agent according to the first voice and the second voice; Extracting a second speech feature from the speech segment corresponding to the target time point in the conversation speech, identifying tone information of the agent and the customer representative during each round of conversation in the speech segment based on the second speech feature, and organizing the tone information into a tone information sequence based on the order of each round of conversation in the speech segment; Obtaining a text segment corresponding to the speech segment from the conversation text corresponding to the conversation speech, performing conversation anomaly recognition on each round of conversation in the text segment, and obtaining a conversation anomaly recognition result corresponding to each round of conversation; generating a dialogue adjustment strategy based on the tone information sequence and the dialogue anomaly recognition result, and presenting the dialogue adjustment strategy to the agent; Among them, the generating of the dialogue adjustment strategy based on the tone information sequence and the dialogue anomaly recognition result includes: determining the target tone information corresponding to the negative tone of the customer person from the tone information sequence; determining the target round dialogue corresponding to the target tone information from the dialogue segment; obtaining the dialogue anomaly recognition result corresponding to the target round dialogue; if the dialogue anomaly recognition result indicates an abnormal dialogue, generating customer soothing information based on the content of the target round dialogue to obtain the dialogue adjustment strategy.

2. The method according to claim 1, characterized in that The identifying whether the customer has a negative emotion based on the first text feature and the first voice feature includes: Performing recognition processing on the first text feature and the first voice feature using an emotion recognition model to obtain emotion scores corresponding to the first text feature and the first voice feature; If the emotion score is less than the preset emotion score, it is determined that the customer has the negative emotion; If the emotion score is not less than the preset emotion score, it is determined that the customer person does not have the negative emotion.

3. The method according to claim 1, characterized in that Generating the conversation voice between the customer agent and the agent based on the first voice and the second voice includes: Obtaining first time information corresponding to the first voice and second time information corresponding to the second voice; Based on the first time information and the second time information, the first voice and the second voice are synthesized to obtain the conversation voice between the customer staff and the agent.

4. The method according to claim 1, wherein The extracting a second speech feature from the speech segment corresponding to the target time point in the conversation speech includes: Determining a target time range based on the target time point and time points adjacent to the target time point in the conversational speech; Feature extraction is performed on the speech segment corresponding to the target time range in the conversation speech to obtain the second speech feature.

5. The method according to claim 1, characterized in that The forming of a tone information sequence based on the order of each round of dialogue in the speech segment comprises: Obtaining time points corresponding to each round of dialogue in the speech segment, and determining a dialogue order of each round of dialogue based on the time points corresponding to each round of dialogue; The tone information corresponding to each round of dialogue is sorted based on the dialogue sequence to obtain the tone information sequence.

6. The method according to claim 1, characterized in that The performing dialogue anomaly recognition on each round of dialogue in the text segment to obtain a dialogue anomaly recognition result corresponding to each round of dialogue includes: Perform text semantic recognition processing on each round of dialogue to obtain the semantic information of each round of dialogue; If the semantic information meets the preset abnormal dialogue condition, determining the corresponding dialogue as an abnormal dialogue; If the semantic information does not meet the preset abnormal dialogue condition, the corresponding dialogue is determined to be a normal dialogue.

7. A conversation processing device based on emotion recognition, characterized in that: The device comprises: A first acquisition unit is used to acquire a first voice of a customer and a second voice of an agent in a current business scenario; a first extraction unit, configured to extract a first text feature and a first voice feature from the first speech, and identify whether the customer has a negative emotion based on the first text feature and the first voice feature; A second acquisition unit is configured to acquire a target time point at which the negative emotion occurs, and generate a conversation voice between the customer agent and the agent based on the first voice and the second voice; a second extraction unit configured to extract a second speech feature from the speech segment corresponding to the target time point in the conversation speech, identify tone information of the agent and the customer representative during each round of conversation in the speech segment based on the second speech feature, and organize the tone information into a tone information sequence based on the order of each round of conversation in the speech segment; a third acquisition unit, configured to acquire a text segment corresponding to the speech segment from the conversation text corresponding to the conversation speech, perform conversation anomaly recognition on each round of conversation in the text segment, and obtain a conversation anomaly recognition result corresponding to each round of conversation; a generating unit, configured to generate a dialogue adjustment strategy based on the tone information sequence and the dialogue anomaly recognition result, and present the dialogue adjustment strategy to the agent; Among them, the generating of the dialogue adjustment strategy based on the tone information sequence and the dialogue anomaly recognition result includes: determining the target tone information corresponding to the negative tone of the customer person from the tone information sequence; determining the target round dialogue corresponding to the target tone information from the dialogue segment; obtaining the dialogue anomaly recognition result corresponding to the target round dialogue; if the dialogue anomaly recognition result indicates an abnormal dialogue, generating customer soothing information based on the content of the target round dialogue to obtain the dialogue adjustment strategy.

8. A terminal, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the method for processing dialogues based on emotion recognition according to any one of claims 1 to 6 is implemented.

9. A storage medium, characterized in that: The storage medium stores a plurality of instructions, and the plurality of instructions are suitable for being loaded by a processor to execute the conversation processing method based on emotion recognition according to any one of claims 1 to 6.

Images