A random switching method between human and AI customer service in question-answering dialogue based on emotion recognition

By real-time emotional recognition of voice tone changes in conversations between customers and customers, efficient switching and consistent processing between manual and AI customer service is achieved, the problems of poor switching experience and high switching triggering mechanism in the existing technology are solved, and customers are enhanced Experience.

CN117350739BActive Publication Date: 2024-11-26GUANGZHOU EAPHONETECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311398078.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-26
Publication Date
2024-11-26
Estimated Expiration
2043-10-26

AI Technical Summary

Technical Problem

After the existing customer service landing system switches between manual and AI customer service, customers need to re-explain their needs. The voice difference between before and after the switch is large, resulting in poor customer switching experience and high threshold of the switch trigger mechanism, making it impossible to achieve efficient switching.

Method used

By real-time emotional recognition based on the voice tone change characteristics in the conversation between customers and customer service, the switching trigger mechanism is judged, and the takeover and consistency processing between manual and AI customer service is realized, and the voice difference before and after the switching is narrowed.

Benefits of technology

It realizes efficient switching between manual and AI customer service at any time, improves the experience of customers when switching channels, and reduces voice differences.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117350739B_ABST
    Figure CN117350739B_ABST
Patent Text Reader

Abstract

The present invention provides a method for random switching between manual and AI customer service in question-and-answer dialogues based on emotion recognition, which belongs to the field of process management technology, and includes: based on the voice and intonation change characteristics in the dialogue between the customer object and the AI ​​customer service or manual customer service, real-time emotion recognition is performed on the customer object to obtain a real-time emotion recognition result; when the real-time emotion recognition result satisfies the switching trigger mechanism, the customer object is taken over and switched to obtain a takeover and switching result; when the takeover and switching is successful, consistency processing is performed on the switched customer service channel based on the customer service voice characteristics before the switching to obtain a switching consistency processing result; the emotion recognition result of the customer object is obtained based on the voice and intonation change characteristics of the customer object, and when it is judged based on the emotion recognition result that it satisfies the switching trigger mechanism, the customer service channel to which it is located is taken over and switched and consistency processing is performed, so as to realize random switching between manual and AI customers, and improve the experience of the customer object when the channel is switched.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of process management technology, and in particular to a method for randomly switching between human and AI customer service in question-and-answer dialogues based on emotion recognition. Background Art

[0002] At present, in addition to solving high-frequency business problems encountered by customers, AI customer service will also provide customers with multi-dimensional assistants, shopping guides, voice chats, entertainment and other services with human-like capabilities to improve the service experience of customers. Existing customer service implementation systems mostly adopt a combination of AI customer service and manual customer service. When AI customer service cannot meet the needs of customers, manual customer service will be used to serve them, so as to save labor costs while meeting the needs of customer service objects within a certain range.

[0003] However, in the existing customer service implementation system, the customer is required to restate their needs after switching between manual and AI customer service, and the voice difference before and after the switch is large, resulting in the customer's switching experience needs to be improved. In addition, the existing customer service implementation system often switches to manual customer service when AI customer service cannot meet the needs of the customer, resulting in a high trigger threshold for its switching trigger mechanism, which makes it impossible to achieve efficient switching at any time.

[0004] Therefore, the present invention proposes a method for randomly switching between human and AI customer service in question-and-answer dialogues based on emotion recognition. Summary of the invention

[0005] The present invention provides a method for random switching between manual and AI customer service in question-and-answer dialogues based on emotion recognition, which is used to obtain the emotion recognition result of a customer object based on the voice and intonation change characteristics when the customer object talks with the customer service, and when it is judged based on the emotion recognition result that the customer service channel to which it is located meets the switching trigger mechanism, takeover switching and consistency processing are performed to achieve efficient switching between manual and AI customers at any time, reduce the voice difference before and after switching, and improve the customer experience when the channel is switched.

[0006] The present invention provides a method for randomly switching between human and AI customer service in a question-and-answer dialogue based on emotion recognition, comprising:

[0007] S1: Based on the voice and intonation change characteristics of the customer object in the conversation with the AI ​​customer service or manual customer service, the customer object is recognized in real time to obtain the real-time emotion recognition result;

[0008] S2: When the real-time emotion recognition result satisfies the switching trigger mechanism, the customer object is switched over to obtain the switching result;

[0009] S3: When the takeover switch is successful, consistency processing is performed on the customer service channel after the switch based on the customer service voice characteristics before the switch to obtain a switch consistency processing result.

[0010] Preferably, S1: based on the voice and intonation change characteristics in the conversation between the customer object and the AI ​​customer service or the manual customer service, real-time emotion recognition is performed on the customer object to obtain a real-time emotion recognition result, including:

[0011] S101: AI customer service channels and manual customer service channels that can be randomly switched to each other are aggregated as a switchable channel group, and customer objects that are currently in conversation with all customer service channels in the switchable channel group are aggregated as a switchable object group;

[0012] S102: Acquire the voice of each customer object in the switchable object group when communicating with the customer service channel, and perform real-time emotion recognition on the customer object based on the voice intonation change characteristics to obtain a real-time emotion recognition result.

[0013] Preferably, S102: acquiring the voice of each customer object in the switchable object group when communicating with the customer service channel, and performing real-time emotion recognition on the customer object based on the voice intonation change characteristics to obtain the real-time emotion recognition result, including:

[0014] Get the voice of each customer object in the switchable object group when they are talking with the customer service channel;

[0015] Determine the physiological characteristic deviation between the physiological characteristics of the client object of each gender and the reference physiological characteristics of the corresponding gender;

[0016] Based on the physiological feature deviation, the voice intonation variation characteristics of the customer object and the reference intonation variation characteristics of the corresponding gender, the real-time dissatisfaction of the customer object is analyzed as the real-time emotion recognition result.

[0017] Preferably, based on the physiological feature deviation and the voice intonation variation characteristics of the customer object and the reference intonation variation characteristics of the corresponding gender, the real-time dissatisfaction of the customer object is analyzed as the real-time emotion recognition result, including:

[0018] Based on the list of deviation item-speech and intonation feature dimension-influence dimension-influence coefficient, determine the target speech and intonation feature dimension affected by each target deviation item in the physiological feature deviation, the target influence dimension affected by the target speech and intonation feature dimension, and the target influence coefficient on the target influence dimension;

[0019] Based on the specific deviation value of the target deviation item in the physiological characteristic deviation and the corresponding target influence coefficient and the reference characteristic value range of the target voice and intonation characteristic dimension in the target influence dimension in the reference intonation change characteristics of the corresponding gender, the standard characteristic value range of the target voice and intonation characteristic dimension in the target influence dimension of the client object is calculated;

[0020] In the voice and intonation variation characteristics of the client object, determining the current feature value of the target voice and intonation feature dimension in the target impact dimension;

[0021] Based on all current feature values ​​and standard feature value ranges of the customer object, the real-time dissatisfaction of the customer object is analyzed as a real-time emotion recognition result.

[0022] Preferably, based on all current feature values ​​and standard feature value ranges of the customer object, analyzing the real-time dissatisfaction of the customer object as the real-time emotion recognition result includes:

[0023] Determine whether the current feature value does not exceed the corresponding standard feature value range. If so, determine that the single item dissatisfaction of the target speech and intonation feature dimension in the target impact dimension is 1;

[0024] Otherwise, determine the target exceeding direction of the current eigenvalue relative to the corresponding standard eigenvalue range, and calculate the single item dissatisfaction of the target speech and intonation feature dimension in the target impact dimension based on the preset dissatisfaction calculation method corresponding to the target exceeding direction and the current eigenvalue and the corresponding standard eigenvalue range;

[0025] Based on the individual dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, the real-time dissatisfaction of the customer object is calculated.

[0026] Preferably, the real-time dissatisfaction of the customer object is calculated based on the individual dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, including:

[0027] Calculate the numerical span of the single item dissatisfaction of all target voice and intonation feature dimensions of the client object in all target influence dimensions;

[0028] When the value span does not exceed the span threshold, the average of the single dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions is regarded as the real-time dissatisfaction of the customer object;

[0029] When the numerical span exceeds the span threshold, the sum of the minimum value of the single item dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions and the span threshold is used as the screening threshold;

[0030] Based on the individual dissatisfaction and screening thresholds of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, the real-time dissatisfaction of the customer object is calculated.

[0031] Preferably, the real-time dissatisfaction of the customer object is calculated based on the individual dissatisfaction and screening thresholds of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, including:

[0032] The average of the single dissatisfaction of all target voice and intonation feature dimensions of the customer object that exceeds the screening threshold among the single dissatisfaction of all target impact dimensions is regarded as the real-time dissatisfaction of the customer object.

[0033] Preferably, S2: when the real-time emotion recognition result satisfies the switching trigger mechanism, the customer object is switched to obtain the switching result, including:

[0034] S201: Determine the total number of all switchable AI customer service channels and the total number of all manual customer service channels;

[0035] S202: sorting the real-time dissatisfaction in the real-time emotion recognition results of all customer objects according to the real-time dissatisfaction from large to small, to obtain a real-time dissatisfaction sequence;

[0036] S203: setting the target customer service channel of the customer object corresponding to the total number of real-time dissatisfactions of the previous manual customer service channels in the real-time dissatisfaction sequence as the manual customer service channel, and setting the target customer service channel of the remaining customer objects except the customer object corresponding to the total number of real-time dissatisfactions of the previous manual customer service channels in the real-time dissatisfaction sequence as the AI ​​customer service channel;

[0037] S204: Determine whether the current service channel of the customer object is inconsistent with the target customer service channel. If so, determine that the real-time emotion recognition result of the customer object meets the switching trigger mechanism, and take over the customer object to obtain the takeover switching result. Otherwise, determine that the real-time emotion recognition result of the customer object does not meet the switching mechanism, and retain the real-time dissatisfaction of the customer object.

[0038] Preferably, S3: when the takeover switch is successful, consistency processing is performed on the customer service channel after the switch based on the customer service voice characteristics before the switch to obtain a switch consistency processing result, including:

[0039] Determine the voice content from the customer service voice features before the switch, generate prompt content based on the voice content, and transmit the prompt content to the customer service channel after the switch to obtain the content processing result;

[0040] Performing real-time voice change processing on the customer service channel after switching based on the sound features in the customer service voice features before switching, and obtaining a voice change processing result;

[0041] The content processing result and the voice change processing result are regarded as the switching consistency processing result.

[0042] Preferably, generating prompt content based on voice content includes:

[0043] The speech content is input into the connection content generation model to obtain the prompt content.

[0044] The beneficial effects of the present invention compared with the prior art are: obtaining the emotion recognition result of the customer object based on the voice and intonation change characteristics when the customer object talks with the customer service, and taking over and switching the customer service channel to which it is located and performing consistency processing when it is judged based on the emotion recognition result that it meets the switching trigger mechanism, so as to realize efficient switching between manual and AI customers at any time, reduce the voice difference before and after switching, and improve the customer experience when the channel is switched.

[0045] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0046] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0048] Figure 1 This is a flow chart of a method for randomly switching between human and AI customer service in a question-and-answer dialogue based on emotion recognition in an embodiment of the present invention;

[0049] Figure 2 This is a flow chart of another method for randomly switching between human and AI customer service for question-and-answer dialogue based on emotion recognition in an embodiment of the present invention;

[0050] Figure 3 This is a flow chart of a method for randomly switching between human and AI customer service in a question-and-answer dialogue based on emotion recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0052] Embodiment 1:

[0053] The present invention provides a method for randomly switching between human and AI customer service in question-and-answer dialogue based on emotion recognition, referring to Figure 1 ,include:

[0054] S1: Based on the voice and intonation change characteristics (such as the highest decibel value, stress position, sentence form, etc. (i.e., interrogative sentence, rhetorical question, interrogative sentence, declarative sentence, etc.)) of the customer object (i.e., the customer served by the manual customer service channel and the AI ​​customer service channel) in the conversation with the AI ​​customer service or the manual customer service, real-time emotion recognition is performed on the customer object to obtain the real-time emotion recognition result (i.e., the result obtained after real-time emotion recognition of the customer object);

[0055] S2: When the real-time emotion recognition result satisfies the switching trigger mechanism (i.e., the trigger mechanism for switching the customer service channel, and the switching of the customer service channel includes switching from the manual customer service channel to the AI ​​customer service channel and switching from the AI ​​customer service channel to the manual customer service channel), the customer object is taken over and switched (the takeover switching is switching the customer service channel category of the customer object, and the customer service channel category includes obtaining the emotion recognition result of the customer object based on the voice and intonation change characteristics when the customer object talks with the customer service, and judging based on the emotion recognition result that it satisfies the switching trigger mechanism, the customer service channel to which it belongs is taken over and switched and consistency processed, so as to realize the efficient switching of manual and AI customers at any time, and reduce the voice difference before and after the switching, and improve the experience of the customer object when the channel is switched and the AI ​​customer service channel), and obtain the takeover switching result (i.e., the result obtained after the takeover switching of the customer object);

[0056] S3: When the takeover switch is successful, the customer service channel after switching is processed for consistency based on the customer service voice features before switching (the consistency processing includes content consistency processing and voice feature consistency processing, wherein the content consistency processing is to generate prompt content that can be connected with the conversation content before switching for the customer service after switching to refer to connect the conversation before switching, and the voice feature consistency processing is to process the voice of the customer service channel after switching so that the customer object can feel that the difference in voice features before and after switching is small), and obtain the switching consistency processing result (that is, the result obtained after consistency processing of the customer service channel after switching based on the customer service voice features before switching).

[0057] The emotion recognition results of the customer objects are obtained based on the characteristics of voice and tone changes when the customer objects talk with customer service. When it is judged based on the emotion recognition results that the customer object meets the switching trigger mechanism, the customer service channel where the customer object is located is taken over and switched and consistency processing is performed to achieve efficient switching between manual and AI customers at any time, narrow the voice difference before and after switching, and improve the customer object's experience when the channel is switched.

[0058] Embodiment 2:

[0059] Based on Example 1, S1: Based on the voice and intonation change characteristics of the customer object in the conversation with the AI ​​customer service or manual customer service, real-time emotion recognition is performed on the customer object to obtain real-time emotion recognition results, referring to Figure 2 ,include:

[0060] S101: The AI ​​customer service channels and manual customer service channels that can be randomly switched to each other are aggregated as a switchable channel group (i.e., a cluster including all AI customer service channels and all manual customer service channels that can be randomly switched to each other), and the customer objects that are currently in conversation with all customer service channels in the switchable channel group are aggregated as a switchable object group (i.e., a cluster including customer objects that are currently in conversation with all customer service channels in the switchable channel group);

[0061] S102: Acquire the voice of each customer object in the switchable object group when communicating with the customer service channel, and perform real-time emotion recognition on the customer object based on the voice intonation change characteristics to obtain a real-time emotion recognition result.

[0062] The above process realizes the determination of the switchable channel group and the switchable object group and the real-time emotion recognition of the customer object based on the voice intonation change characteristics.

[0063] Embodiment 3:

[0064] Based on Example 2, S102: obtaining the voice of each customer object in the switchable object group when talking with the customer service channel, and performing real-time emotion recognition on the customer object based on the voice intonation change characteristics to obtain the real-time emotion recognition result, including:

[0065] Get the voice of each customer object in the switchable object group when they are talking with the customer service channel;

[0066] Determine the physiological characteristic deviation (i.e., the information difference or data difference between the physiological characteristics of the customer object and the standard physiological characteristics of the same gender, such as the physiological age difference) between the physiological characteristics of the customer object of each gender (the physiological characteristics here are the physiological characteristics that cause differences in the voice characteristics of different objects, such as age) and the reference physiological characteristics of the corresponding gender (i.e., the preset physiological characteristics of the corresponding gender, such as: physiological age 25 years old);

[0067] Based on the physiological characteristic deviation, the voice intonation change characteristics of the customer object and the reference intonation change characteristics of the corresponding gender (that is, the preset intonation change characteristics of the corresponding gender, for example: the reference intonation change characteristics of people between 20 and 30 years old (such as the maximum decibel value), etc.), the real-time dissatisfaction of the customer object is analyzed as the real-time emotion recognition result.

[0068] Through the physiological characteristic deviation between the physiological characteristics of customer objects of the same gender and the reference physiological characteristics and the reference intonation change characteristics of the corresponding gender, real-time emotion recognition of customer objects is achieved from the perspective of voice intonation change characteristics.

[0069] Embodiment 4:

[0070] On the basis of Example 3, based on the physiological feature deviation and the voice intonation variation characteristics of the customer object and the reference intonation variation characteristics of the corresponding gender, the real-time dissatisfaction of the customer object is analyzed as the real-time emotion recognition result, including:

[0071] Based on the deviation item (i.e., the information item in the physiological feature deviation: for example, age, weight, etc.) - speech and intonation feature dimension (i.e., the speech intonation feature dimension directly affected by the deviation item, such as decibel value, decibel change rate, etc.) - influence dimension (i.e., the dimension directly affected by the deviation item on the speech and intonation feature dimension, such as extreme value, maximum value, numerical fluctuation trend, etc.) - influence coefficient (i.e., the multiple of the influence of the deviation item information on the numerical value of the speech and intonation feature dimension in the influence dimension, represented by the ratio of the numerical value of the deviation item to the numerical value of the speech and intonation feature dimension in the influence dimension) list (i.e., the speech and intonation feature dimension directly affected by the preset deviation item, as well as the influence dimension directly affected by the speech and intonation feature dimension, and the influence dimension on the speech and intonation feature dimension). The target speech and intonation feature dimensions affected by each target deviation item in the physiological feature deviation (i.e., the deviation item included in the physiological feature deviation of the client object) are determined, and the target influence dimension of the target speech and intonation feature dimension (i.e., the influence dimension of each target deviation item directly affected by the target speech and intonation feature dimension determined based on the above list) and the target influence coefficient on the target influence dimension (i.e., the influence coefficient of each target deviation item determined based on the above list on the target speech and intonation feature dimension’s numerical value);

[0072] Based on the specific deviation value of the target deviation item in the physiological characteristic deviation (i.e., the specific numerical value corresponding to the target deviation item in the physiological characteristic deviation, such as age difference, weight difference, etc.) and the corresponding target influence coefficient and the reference characteristic value range of the target voice and intonation feature dimension in the target influence dimension in the reference intonation change feature of the corresponding gender (i.e., the value range of the preset reference characteristic value of the target voice and intonation feature dimension in the target influence dimension in the reference intonation change feature in the state of no emotional dissatisfaction, such as the allowable value range of the maximum decibel value), calculate the standard characteristic value range of the target voice and intonation feature dimension in the target influence dimension of the customer object (i.e., the value range of the characteristic value of the target voice and intonation feature dimension in the target influence dimension when the customer object is in a state of no emotional dissatisfaction, such as the allowable value range of the maximum decibel value when the customer object is in a state of no emotional dissatisfaction, and the calculation method is: take the product of the upper limit value of the reference characteristic value range and the influence coefficient as the upper limit value of the standard characteristic value range, and take the product of the lower limit value of the reference characteristic value range and the influence coefficient as the lower limit value of the standard characteristic value range);

[0073] In the voice and intonation variation characteristics of the client object, determine the current feature value of the target voice and intonation feature dimension in the target impact dimension (that is, the corresponding value of the target voice and intonation feature dimension in the voice and intonation variation characteristics in the target impact dimension, for example, the decibel value is 70);

[0074] Based on all current feature values ​​and standard feature value ranges of the customer object, the real-time dissatisfaction of the customer object (ie, a numerical value representing the degree of emotional dissatisfaction expressed by the customer object during the current call) is analyzed as a real-time emotion recognition result.

[0075] Based on the list of deviation items-speech and intonation feature dimensions-influence dimensions-influence coefficients, the target speech and intonation feature dimensions and the corresponding target influence dimensions and the target impression coefficients of each target deviation item in the physiological feature deviation are determined, and based on the corresponding reference feature value range and the specific deviation value of the deviation item, the standard feature value range of the target speech and intonation feature dimension of the customer object in the target influence dimension is calculated, and further based on the current feature value of the target speech and intonation feature dimension in the target influence dimension, the real-time dissatisfaction of the customer object is analyzed to represent the emotion recognition result of the customer object, so as to realize accurate analysis of the emotional dissatisfaction of the customer object from the influence dimension of different feature dimensions of the customer object's speech and intonation.

[0076] Embodiment 5:

[0077] On the basis of Example 4, based on all current feature values ​​and standard feature value ranges of the customer object, the real-time dissatisfaction of the customer object is analyzed as the real-time emotion recognition result, including:

[0078] Determine whether the current feature value does not exceed the corresponding standard feature value range. If so, determine that the single item dissatisfaction of the target speech and intonation feature dimension in the target impact dimension is 1;

[0079] Otherwise, determine the target exceeding direction of the current eigenvalue relative to the corresponding standard eigenvalue range (there are two target exceeding directions, one is that the current eigenvalue is less than the lower limit of the standard eigenvalue range, and the other is that the current eigenvalue is greater than the upper limit of the standard eigenvalue range), based on the preset dissatisfaction calculation method corresponding to the target exceeding direction (for example: when the current eigenvalue A is less than the lower limit of the standard eigenvalue range AMIN, the preset dissatisfaction Q calculation method corresponding to the target exceeding direction is: Q = (AMIN-A) × α1; when the current eigenvalue A is greater than the upper limit of the standard eigenvalue range AMAX, the preset dissatisfaction Q calculation method corresponding to the target exceeding direction is: Q = (A-AMAX) × α2, where Q is a single item dissatisfaction) and the current eigenvalue and the corresponding standard eigenvalue range, calculate the single item dissatisfaction of the target voice and intonation feature dimension in the target influence dimension (that is, the current dissatisfaction of the customer object evaluated from the perspective of the target influence dimension of the target voice and intonation feature dimension);

[0080] Based on the individual dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, the real-time dissatisfaction of the customer object is calculated.

[0081] The above-mentioned two preset non-linear calculation methods based on the relative size of the current eigenvalue and the standard eigenvalue range and the target exceeding direction of the current eigenvalue relative to the corresponding standard eigenvalue range when the current eigenvalue exceeds the corresponding standard eigenvalue range can accurately calculate the target exceeding direction of the current eigenvalue of the customer object relative to the corresponding standard eigenvalue range, and further calculate the real-time dissatisfaction of the customer object, thereby realizing the accurate calculation of the real-time dissatisfaction of the customer object.

[0082] Embodiment 6:

[0083] On the basis of Example 5, the real-time dissatisfaction of the customer object is calculated based on the individual dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, including:

[0084] Calculate the numerical span of the single dissatisfaction of all target voice and intonation feature dimensions of the client object in all target impact dimensions (that is, the maximum difference between the differences of any two single dissatisfactions of all target voice and intonation feature dimensions of the client object in all target impact dimensions);

[0085] When the numerical span does not exceed the span threshold (i.e., the preset comparison threshold of the numerical span used to compare with the numerical span when distinguishing the calculation method of calculating the real-time dissatisfaction of the customer object based on the single dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions), the average of the single dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions is taken as the real-time dissatisfaction of the customer object;

[0086] When the numerical span exceeds the span threshold, the sum of the minimum value of the single dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions and the span threshold is used as the screening threshold (that is, when the numerical span exceeds the span threshold, it is used to screen out the single dissatisfaction of all target voice and intonation feature dimensions of all customer objects in all target impact dimensions to finally determine (calculate) the real-time dissatisfaction of the customer object).

[0087] Based on the individual dissatisfaction and screening thresholds of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, the real-time dissatisfaction of the customer object is calculated.

[0088] The above process analyzes the numerical span of single dissatisfaction of all target voice and intonation feature dimensions of the customer objects in all target influence dimensions, and divides the calculation method of the real-time dissatisfaction of the customer objects into two cases: the numerical span of all single dissatisfaction items is large and the numerical span is not large. When the numerical span is large, some single dissatisfaction items with large values ​​are used to calculate the real-time dissatisfaction of the customer objects, avoiding missing the excessive dissatisfaction of the customer objects.

[0089] Embodiment 7:

[0090] On the basis of Example 6, based on the individual dissatisfaction and screening thresholds of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, the real-time dissatisfaction of the customer object is calculated, including:

[0091] The average of the single dissatisfaction of all target voice and intonation feature dimensions of the customer object that exceeds the screening threshold among the single dissatisfaction of all target impact dimensions is regarded as the real-time dissatisfaction of the customer object.

[0092] The average of some single dissatisfaction items with larger values ​​is used as the real-time dissatisfaction of the customer object, avoiding the omission of excessive dissatisfaction of the customer object.

[0093] Embodiment 8:

[0094] On the basis of Example 1, S2: when the real-time emotion recognition result satisfies the switching trigger mechanism, the customer object is switched to obtain the switching result, referring to Figure 3 ,include:

[0095] S201: Determine the total number of all switchable AI customer service channels (i.e., the total number of AI customer service channels included in the AI ​​customer service channels and manual customer service channels that can be switched to each other) and the total number of all manual customer service channels (i.e., the total number of manual customer service channels included in the AI ​​customer service channels and manual customer service channels that can be switched to each other);

[0096] S202: Sort the real-time dissatisfaction in the real-time emotion recognition results of all customer objects according to the real-time dissatisfaction from large to small, and obtain a real-time dissatisfaction sequence (i.e., a sequence of real-time dissatisfaction of customer objects currently taken over by all switchable AI customer service channels and manual customer service channels sorted from large to small);

[0097] S203: setting the target customer service channel (i.e., the type of customer service channel that should be taken over by the customer object determined based on the real-time dissatisfaction of the customer object, wherein the customer service channel types include manual customer service channels and AI customer service channels) of the customer object corresponding to the first total number of manual customer service channels (for example, the total number of manual customer service channels is n) of the real-time dissatisfaction in the real-time dissatisfaction sequence as the manual customer service channel (then setting the target customer service channel of the customer object corresponding to the first n real-time dissatisfaction in the real-time dissatisfaction sequence as the manual customer service channel), and setting the target customer service channel of the remaining customer objects except the customer objects corresponding to the first total number of manual customer service channels of the real-time dissatisfaction sequence as the AI ​​customer service channel;

[0098] S204: Determine whether the current service channel of the customer object is inconsistent with the target customer service channel. If so, determine that the real-time emotion recognition result of the customer object meets the switching trigger mechanism, and take over the customer object to obtain the takeover switching result. Otherwise, determine that the real-time emotion recognition result of the customer object does not meet the switching mechanism, and retain the real-time dissatisfaction of the customer object.

[0099] The above process sorts the real-time dissatisfaction of the customer objects currently taken over by the AI ​​customer service channel and the manual customer service channel that can be switched between each other, determines the target customer service channel of the customer object based on the sorting results, and further determines whether the customer object needs to be taken over and switched, thereby realizing whether the switching trigger mechanism is met based on the real-time emotion recognition results, and realizing the takeover and switching of the customer object.

[0100] Embodiment 9:

[0101] On the basis of Example 1, S3: when the takeover switch is successful, consistency processing is performed on the customer service channel after the switch based on the customer service voice characteristics before the switch to obtain a switch consistency processing result, including:

[0102] Determine the voice content from the customer service voice features before the switch (i.e., the content spoken by the customer object in the call record before the switch), generate prompt content based on the voice content (i.e., used to prompt the manual customer service or AI customer service in the customer service channel after the switch), and transmit the prompt content to the customer service channel after the switch to obtain the content processing result (i.e., the result obtained after performing content consistency processing on the customer service channel after the switch);

[0103] Performing real-time voice change processing on the customer service channel after switching based on the sound features in the customer service voice features before switching, and obtaining a voice change processing result (i.e., a result obtained after performing voice feature consistency processing on the customer service channel after switching);

[0104] The content processing result and the voice change processing result are regarded as the switching consistency processing result.

[0105] In this embodiment, the prompt content is transmitted to the switched customer service channel, and then:

[0106] When the switched customer service channel is a manual customer service channel, the prompt content is used to prompt the manual customer service staff of the switched manual customer service channel to continue communication according to the prompt content;

[0107] When the switched customer service channel is an AI customer service channel, the prompt content is used to prompt the AI ​​response model of the AI ​​customer service channel so that it can continue to communicate according to the prompt content.

[0108] The above process realizes the prompt generation and voice change processing of the conversation content of the switched customer service channel, that is, the customer object does not need to repeat the first half of the conversation content after being switched, and the voice change processing makes the customer object feel that the voice sound difference from before the switch is smaller, thereby improving the customer object's service experience.

[0109] Embodiment 10:

[0110] On the basis of Example 9, the prompt content is generated based on the voice content, including:

[0111] The speech content is input into the connection content generation model to obtain prompt content (ie, part of the text content obtained after the speech content is input into the connection content generation model).

[0112] In this embodiment, the connection content generation model is pre-trained using a large number of complete customer service texts, and can generate the second half of the customer service conversation text that is partially connected to the first half of the customer service conversation text based on the input first half of the customer service conversation text.

[0113] The above process realizes the generation of prompt content based on the pre-prepared connection content generation model.

[0114] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for randomly switching between human and AI customer service in question-and-answer dialogue based on emotion recognition, characterized in that: include: S1: Based on the voice and intonation change characteristics of the customer object in the conversation with the AI ​​customer service or manual customer service, the customer object is recognized in real time to obtain the real-time emotion recognition result; S2: When the real-time emotion recognition result satisfies the switching trigger mechanism, the customer object is switched over to obtain the switching result; S3: When the takeover switch is successful, consistency processing is performed on the customer service channel after the switch based on the customer service voice characteristics before the switch to obtain a switch consistency processing result; S1: Based on the voice and intonation change characteristics of the customer object in the conversation with the AI ​​customer service or manual customer service, the customer object is recognized in real time to obtain real-time emotion recognition results, including: S101: AI customer service channels and manual customer service channels that can be randomly switched to each other are aggregated as a switchable channel group, and customer objects that are currently in conversation with all customer service channels in the switchable channel group are aggregated as a switchable object group; S102: Acquire the voice of each customer object in the switchable object group when talking with the customer service channel, and perform real-time emotion recognition on the customer object based on the voice intonation change characteristics to obtain a real-time emotion recognition result; S102: Acquire the voice of each customer object in the switchable object group when talking with the customer service channel, and perform real-time emotion recognition on the customer object based on the voice intonation change characteristics to obtain real-time emotion recognition results, including: Get the voice of each customer object in the switchable object group when they are talking with the customer service channel; Determine the physiological characteristic deviation between the physiological characteristics of the client object of each gender and the reference physiological characteristics of the corresponding gender; Based on the physiological feature deviation and the voice intonation variation characteristics of the customer object and the reference intonation variation characteristics of the corresponding gender, the real-time dissatisfaction of the customer object is analyzed as the real-time emotion recognition result; Based on the physiological feature deviation, the voice tone change characteristics of the customer object, and the reference tone change characteristics of the corresponding gender, the real-time dissatisfaction of the customer object is analyzed as the real-time emotion recognition result, including: Based on the list of deviation item-speech and intonation feature dimension-influence dimension-influence coefficient, determine the target speech and intonation feature dimension affected by each target deviation item in the physiological feature deviation, the target influence dimension affected by the target speech and intonation feature dimension, and the target influence coefficient on the target influence dimension; Based on the specific deviation value of the target deviation item in the physiological characteristic deviation and the corresponding target influence coefficient and the reference characteristic value range of the target voice and intonation characteristic dimension in the target influence dimension in the reference intonation change characteristics of the corresponding gender, the standard characteristic value range of the target voice and intonation characteristic dimension in the target influence dimension of the client object is calculated; In the voice and intonation variation characteristics of the client object, determining the current feature value of the target voice and intonation feature dimension in the target impact dimension; Based on all current feature values ​​and standard feature value ranges of the customer object, the real-time dissatisfaction of the customer object is analyzed as a real-time emotion recognition result.

2. The method for random switching between human and AI customer service in question-and-answer dialogue based on emotion recognition according to claim 1, characterized in that: Based on all current feature values ​​and standard feature value ranges of the customer object, the real-time dissatisfaction of the customer object is analyzed as the real-time emotion recognition result, including: Determine whether the current feature value does not exceed the corresponding standard feature value range. If so, determine that the single item dissatisfaction of the target speech and intonation feature dimension in the target impact dimension is 1; Otherwise, determine the target exceeding direction of the current eigenvalue relative to the corresponding standard eigenvalue range, and calculate the single item dissatisfaction of the target speech and intonation feature dimension in the target impact dimension based on the preset dissatisfaction calculation method corresponding to the target exceeding direction and the current eigenvalue and the corresponding standard eigenvalue range; Based on the individual dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, the real-time dissatisfaction of the customer object is calculated.

3. The method for random switching between human and AI customer service in question-and-answer dialogue based on emotion recognition according to claim 2, characterized in that: Based on the individual dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, the real-time dissatisfaction of the customer object is calculated, including: Calculate the numerical span of the single item dissatisfaction of all target voice and intonation feature dimensions of the client object in all target influence dimensions; When the value span does not exceed the span threshold, the average of the single dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions is regarded as the real-time dissatisfaction of the customer object; When the numerical span exceeds the span threshold, the sum of the minimum value of the single item dissatisfaction of all target voice and intonation feature dimensions of the customer object in all target impact dimensions and the span threshold is used as the screening threshold; Based on the individual dissatisfaction and screening thresholds of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, the real-time dissatisfaction of the customer object is calculated.

4. The method for random switching between human and AI customer service in question-and-answer dialogue based on emotion recognition according to claim 3, characterized in that: Based on the individual dissatisfaction and screening thresholds of all target voice and intonation feature dimensions of the customer object in all target impact dimensions, the real-time dissatisfaction of the customer object is calculated, including: The average of the single dissatisfaction of all target voice and intonation feature dimensions of the customer object that exceeds the screening threshold among the single dissatisfaction of all target impact dimensions is regarded as the real-time dissatisfaction of the customer object.

5. The method for random switching between human and AI customer service in question-and-answer dialogue based on emotion recognition according to claim 1, characterized in that: S2: When the real-time emotion recognition result meets the switching trigger mechanism, the customer object is switched to obtain the switching result, including: S201: Determine the total number of all switchable AI customer service channels and the total number of all manual customer service channels; S202: sorting the real-time dissatisfaction in the real-time emotion recognition results of all customer objects according to the real-time dissatisfaction from large to small, to obtain a real-time dissatisfaction sequence; S203: setting the target customer service channel of the customer object corresponding to the total number of real-time dissatisfactions of the previous manual customer service channels in the real-time dissatisfaction sequence as the manual customer service channel, and setting the target customer service channel of the remaining customer objects except the customer object corresponding to the total number of real-time dissatisfactions of the previous manual customer service channels in the real-time dissatisfaction sequence as the AI ​​customer service channel; S204: Determine whether the current service channel of the customer object is inconsistent with the target customer service channel. If so, determine that the real-time emotion recognition result of the customer object meets the switching trigger mechanism, and take over the customer object to obtain the takeover switching result. Otherwise, determine that the real-time emotion recognition result of the customer object does not meet the switching mechanism, and retain the real-time dissatisfaction of the customer object.

6. The method for random switching between human and AI customer service in question-and-answer dialogue based on emotion recognition according to claim 1, characterized in that: S3: When the takeover switch is successful, consistency processing is performed on the customer service channel after the switch based on the customer service voice characteristics before the switch to obtain a switch consistency processing result, including: Determine the voice content from the customer service voice features before the switch, generate prompt content based on the voice content, and transmit the prompt content to the customer service channel after the switch to obtain the content processing result; Performing real-time voice change processing on the customer service channel after switching based on the sound features in the customer service voice features before switching, and obtaining a voice change processing result; The content processing result and the voice change processing result are regarded as the switching consistency processing result.

7. The method for random switching between human and AI customer service in question-and-answer dialogue based on emotion recognition according to claim 6, characterized in that: Generate prompt content based on voice content, including: The speech content is input into the connection content generation model to obtain the prompt content.