Intelligent outbound method and device based on emotion recognition, equipment and storage medium

Abstract

The application relates to artificial intelligence and digital financial technology, and discloses an intelligent outbound call method based on emotion recognition applied to a telephone marketing system in the financial field, which comprises the following steps: extracting acoustic features of audio data; performing speech recognition on the audio data to obtain a speech recognition result and extracting text features of the speech recognition result; classifying the acoustic features from an emotion fluctuation dimension by using a preset classification model to obtain a first-dimension emotion category; classifying the text features from an emotion positivity dimension by using the preset classification model to obtain a second-dimension emotion category; analyzing the first-dimension and second-dimension emotion categories by using a two-dimensional emotion model to obtain an emotion state of a customer; and selecting a corresponding reply text to reply to the customer according to the emotion state. The application further provides an intelligent outbound call device based on emotion recognition, an electronic device and a storage medium. The application can improve the emotion recognition accuracy of a customer in the intelligent outbound call of the telephone marketing system.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence technology and digital financial technology, and in particular to an intelligent outbound calling method, device, electronic device and computer-readable storage medium based on emotion recognition applied to telephone marketing systems in the financial field. Background Technology

[0002] With the booming development of the telemarketing industry, problems such as high operating costs of manual sales, significant impact of work status on emotions, and easy loss of data statistics have become increasingly prominent. Intelligent outbound calling has emerged to address these issues. Intelligent outbound calling integrates multiple functions such as automatic dialing, multi-round voice interaction, intelligent customer intent classification, and customizable outbound calling tasks. It is widely used in scenarios such as telemarketing and telephone follow-up in the financial and insurance industries.

[0003] Currently, in the process of intelligent outbound calling, natural language processing and text mining technologies are used to analyze, process, and extract subjective text with emotional connotations. However, text cannot accurately and objectively describe a person's emotional state. For example, the same word expressed with different tones can lead to different understandings. Furthermore, the generation of emotions often involves a process, requiring incubation before eruption. It has both continuity and instantaneity, and conventional text classifiers are unable to capture these characteristics. As a result, it is difficult to accurately identify customer emotions in the process of intelligent outbound calling in the financial and insurance sectors. Summary of the Invention

[0004] This invention provides an intelligent outbound calling method, device, and computer-readable storage medium based on emotion recognition. Its main purpose is to solve the problem of inaccurate customer emotion recognition when making intelligent outbound calls in financial and insurance telemarketing systems.

[0005] To achieve the above objectives, the present invention provides an intelligent outbound calling method based on emotion recognition, comprising:

[0006] Acquire customer audio data and extract the acoustic features of the audio data;

[0007] The audio data is subjected to speech recognition to obtain speech recognition results, and the text features of the speech recognition results are extracted.

[0008] The acoustic features are classified from the dimension of emotional fluctuation using a preset classification model to obtain the first dimension of emotional category;

[0009] The text features are classified from the perspective of emotional positivity using the preset classification model to obtain the second dimension of emotion category;

[0010] The customer's emotional state is obtained by analyzing the first dimension of the emotion category and the second dimension of the emotion category using a preset two-dimensional emotion model.

[0011] Select the corresponding response text to reply to the customer based on their emotional state.

[0012] Optionally, the step of classifying the acoustic features using a preset classification model from the perspective of emotional fluctuation to obtain a first-dimensional emotional category includes:

[0013] The acoustic features are obtained by using a recurrent neural network in a preset classification model to acquire contextual feature information, and the acoustic features are updated using the contextual feature information to obtain the updated acoustic features.

[0014] The updated acoustic features are weighted and adjusted using the attention mechanism module in the preset classification model to obtain attention acoustic features.

[0015] The attention acoustic features are standardized, convolved, and pooled. The softmax function is used to calculate the probability of the first-dimensional emotion category on the pooled result. The category with the highest probability is taken as the first-dimensional emotion category.

[0016] Optionally, the step of adjusting the weights of the updated acoustic features using the attention mechanism module in the preset classification model to obtain attention acoustic features includes:

[0017] The updated acoustic features are subjected to average pooling to obtain one-dimensional acoustic features;

[0018] A one-dimensional convolution operation is performed on the one-dimensional acoustic features, and the channel attention weights are obtained by calculating the result of the one-dimensional convolution operation using the sigmoid function.

[0019] The updated acoustic features are adjusted using the channel attention weights to obtain the channel attention acoustic features;

[0020] The updated acoustic features are convolved using two preset matrices to obtain a first convolutional feature map and a second convolutional feature map.

[0021] The first operation result is obtained by performing dimensionality transformation and row and column transpose operations on the first convolutional feature map.

[0022] The second operation result is obtained by performing a dimension transformation operation on the second convolutional feature map;

[0023] The first operation result and the second operation result are multiplied by a matrix, and the result of the matrix multiplication is calculated using the softmax function to obtain the spatial attention weights.

[0024] The updated acoustic features are adjusted using the spatial attention weights to obtain spatial attention acoustic features;

[0025] The attention acoustic features are obtained by averaging the channel attention acoustic features and the spatial attention acoustic features and then fusing them.

[0026] Optionally, the step of analyzing the first-dimensional emotion category and the second-dimensional emotion category using a preset two-dimensional emotion model to obtain the customer's emotional state includes:

[0027] Using a pre-defined two-dimensional emotion model, an emotion coordinate system is constructed with the emotion fluctuation dimension and the emotion positivity dimension as the horizontal axis and the vertical axis, respectively.

[0028] Define the correspondence between emotion categories and emotion states in the four quadrants of the emotion coordinate system;

[0029] Based on the correspondence between emotion categories and emotion states in the four quadrants of the emotion coordinate system, the emotion states corresponding to the first-dimensional emotion category and the second-dimensional emotion category are identified.

[0030] Optionally, selecting the corresponding response text to reply to the customer based on the emotional state includes:

[0031] Select the corresponding response text based on the emotional state described;

[0032] The response text is converted into voice data;

[0033] The customer is responded to using the voice data.

[0034] Optionally, the step of performing speech recognition on the audio data to obtain a speech recognition result includes:

[0035] The audio data is subjected to noise reduction processing;

[0036] The audio data after noise reduction is segmented and windowed to obtain multiple speech frames.

[0037] Extract speech features from each speech frame one by one;

[0038] Using a preset speech recognizer, multiple candidate words with different scores are generated for each text word in the audio data based on the speech features of each speech frame;

[0039] All candidate words are combined to form the speech recognition result.

[0040] Optionally, the extraction of acoustic features from the audio data includes:

[0041] The audio data is pre-emphasized using a high-pass filter;

[0042] The pre-emphasized audio data is subjected to frame-by-frame windowing processing to obtain windowed audio frames;

[0043] Perform a Fast Fourier Transform on the windowed audio frame to obtain the corresponding energy spectrum;

[0044] The energy spectrum is smoothed and harmonics are eliminated using a triangular bandpass filter to obtain an optimized energy spectrum.

[0045] The logarithmic energy spectrum of the optimized energy spectrum is calculated, and the discrete cosine transform is performed on the logarithmic energy spectrum to obtain speech feature parameters. The first-order and second-order dynamic difference parameters of the speech feature parameters are extracted as speech feature vectors.

[0046] To address the aforementioned problems, the present invention also provides an intelligent outbound calling device based on emotion recognition, the device comprising:

[0047] The feature extraction module is used to acquire customer audio data, extract the acoustic features of the audio data, perform speech recognition on the audio data, obtain speech recognition results, and extract the text features of the speech recognition results.

[0048] The emotion classification module is used to classify the acoustic features from the dimension of emotion fluctuation using a preset classification model to obtain a first-dimensional emotion category; and to classify the text features from the dimension of emotion positivity using the preset classification model to obtain a second-dimensional emotion category.

[0049] The emotion fusion module is used to analyze the first dimension emotion category and the second dimension emotion category using a preset two-dimensional emotion model to obtain the customer's emotional state;

[0050] The select response module is used to select the corresponding response text to reply to the customer based on the emotional state.

[0051] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0052] At least one processor; and,

[0053] A memory communicatively connected to the at least one processor; wherein,

[0054] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the above-described intelligent outbound calling method based on emotion recognition.

[0055] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the aforementioned intelligent outbound calling method based on emotion recognition.

[0056] This invention extracts acoustic features from the audio data; performs speech recognition on the audio data to obtain speech recognition results, and extracts text features from the speech recognition results; classifies the acoustic features from the dimension of emotional fluctuation using a preset classification model to obtain a first-dimensional emotional category, capturing the continuity of the acoustic features and better reflecting the emotional fluctuation of the customer's audio data; classifies the text features from the dimension of emotional positivity using the preset classification model to obtain a second-dimensional emotional category, capturing the instantaneity of the text features and better reflecting emotional positivity; analyzes the first-dimensional emotional category and the second-dimensional emotional category using a preset two-dimensional emotional model to obtain the customer's emotional state. This combines the different features of both acoustic and text features, improving the limitations of traditional emotion analysis using single-feature data, more accurately grasping a person's emotional state, and selecting the corresponding response text to reply to the customer based on the emotional state, thereby improving the user experience and the effectiveness of the intelligent outbound calling process in financial and insurance telemarketing systems. Therefore, the intelligent outbound calling method, device, electronic device, and computer-readable storage medium based on emotion recognition proposed in this invention can solve the problem of inaccurate customer emotion recognition when making intelligent outbound calls in financial and insurance telemarketing systems. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating an embodiment of the intelligent outbound calling method based on emotion recognition provided by the present invention.

[0058] Figure 2 for Figure 1 The diagram shows a detailed implementation flow of one step in the emotion recognition-based intelligent outbound calling method.

[0059] Figure 3 for Figure 1 The diagram shows a detailed implementation flow of another step in the emotion recognition-based intelligent outbound calling method.

[0060] Figure 4 This is a functional block diagram of an intelligent outbound calling device based on emotion recognition provided in an embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the intelligent outbound calling method based on emotion recognition, according to an embodiment of the present invention.

[0062] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0063] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0064] This application provides an intelligent outbound calling method based on emotion recognition. The executing entity of the emotion-recognition-based intelligent outbound calling method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application embodiment: a server, a terminal, etc. In other words, the emotion-recognition-based intelligent outbound calling method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0065] Reference Figure 1 The diagram shown is a flowchart illustrating an intelligent outbound calling method based on emotion recognition according to an embodiment of the present invention. In this embodiment, the intelligent outbound calling method based on emotion recognition includes:

[0066] S1. Obtain the customer's audio data and extract the acoustic features of the audio data.

[0067] In one embodiment of the present invention, the audio data is customer audio data obtained during the intelligent outbound calling process in a financial and insurance-related telephone marketing system.

[0068] In this embodiment of the invention, Mel-scale Frequency Cepstral Coefficients (MFCCs) can be used to extract the acoustic features of audio data. Mel frequency is proposed based on the characteristics of human hearing and has a non-linear relationship with frequency. Mel-scale frequency cepstral coefficients (MFCCs) are used to calculate spectral features using this relationship.

[0069] Specifically, the extraction of acoustic features from the audio data in S1 includes:

[0070] The audio data is pre-emphasized using a high-pass filter;

[0071] The pre-emphasized audio data is subjected to frame-by-frame windowing processing to obtain windowed audio frames;

[0072] Perform a Fast Fourier Transform on the windowed audio frame to obtain the corresponding energy spectrum;

[0073] The energy spectrum is smoothed and harmonics are eliminated using a triangular bandpass filter to obtain an optimized energy spectrum.

[0074] The logarithmic energy spectrum of the optimized energy spectrum is calculated, and the discrete cosine transform is performed on the logarithmic energy spectrum to obtain speech feature parameters. The first-order and second-order dynamic difference parameters of the speech feature parameters are extracted as speech feature vectors.

[0075] S2. Perform speech recognition on the audio data to obtain the speech recognition result, and extract the text features of the speech recognition result.

[0076] In detail, the process of performing speech recognition on the audio data as described in S2 to obtain the speech recognition result includes:

[0077] The audio data is subjected to noise reduction processing;

[0078] The audio data after noise reduction is segmented and windowed to obtain multiple speech frames.

[0079] Extract speech features from each speech frame one by one;

[0080] Using a preset speech recognizer, multiple candidate words with different scores are generated for each text word in the audio data based on the speech features of each speech frame;

[0081] All candidate words are combined to form the speech recognition result.

[0082] In this embodiment of the invention, the preset speech recognizer is a natural language learning model based on a neural network, comprising a speech recognition decoder, an acoustic model, and a language model. Audio data from the customer is collected, and the speech features of the audio data are extracted using the acoustic model and the language model. The speech recognition decoder then identifies the textual words in the audio data based on these speech features, resulting in a speech recognition result.

[0083] In this embodiment of the invention, the preset speech recognizer has a large vocabulary. For audio data with the same speech features, it generates multiple candidate words. For example, if the audio data sent by the customer is "baofeijine", the corresponding candidate words could be "scrap amount" or "premium amount". Each candidate word can exist in the form of a word graph, which is a directed graph representing different recognition results. It typically includes three parts: the generation time of the candidate word, the candidate word itself, and its corresponding score. For example, the candidate words corresponding to the audio data "baofeijin e" are (0.8, 'premium amount', 0.7) and (0.8, 'scrap amount', 0.4), where 0.8 represents the generation time of the audio data "baofeijin e", and 0.7 and 0.4 represent the scores corresponding to the candidate words "premium amount" and "scrap amount", respectively. Because the text term "premium amount" has a high usage frequency, the corresponding candidate word "premium amount" has the highest score. The higher the score of a candidate word, the greater the likelihood that the candidate word is the text term corresponding to the speech information.

[0084] In this embodiment of the invention, machine learning methods such as convolutional neural networks can be used to extract text features from the speech recognition results.

[0085] S3. Use a preset classification model to classify the acoustic features from the dimension of emotional fluctuation to obtain the first dimension of emotional category.

[0086] In this embodiment of the invention, the preset classification model can be a model composed of a recurrent neural network and an attention mechanism module, wherein the attention mechanism module includes a channel attention module and a spatial attention module.

[0087] In this embodiment of the invention, before classifying the acoustic features from the dimension of emotional fluctuation using a preset classification model to obtain the first dimension of emotional category, the invention may further include training the preset classification model using a training set.

[0088] For details, please refer to Figure 2 As shown, S3 includes:

[0089] S31. Obtain the contextual feature information of the acoustic feature using the recurrent neural network in the preset classification model, and update the acoustic feature using the contextual feature information to obtain the updated acoustic feature.

[0090] S32. The updated acoustic features are weighted and adjusted using the attention mechanism module in the preset classification model to obtain attention acoustic features.

[0091] S33. Standardize, convolution, and pool the attention acoustic features, and use the softmax function to calculate the probability of the first-dimensional emotion category on the pooled result to obtain the probability of the first-dimensional emotion category. The category with the highest probability is taken as the first-dimensional emotion category.

[0092] In this embodiment of the invention, in financial and insurance-related telemarketing, the customer's emotional state is related not only to the text content but also to the tone of voice. Text content is better able to express the positivity of emotions, while audio data is better able to express the volatility of emotions.

[0093] In this embodiment of the invention, the first dimension of emotion category is a category related to the dimension of emotion fluctuation, such as the emotional state of customers such as relaxation or tension that occurs during the conversation in telemarketing for financial insurance.

[0094] Further, S32 includes:

[0095] The updated acoustic features are subjected to average pooling to obtain one-dimensional acoustic features;

[0096] A one-dimensional convolution operation is performed on the one-dimensional acoustic features, and the channel attention weights are obtained by calculating the result of the one-dimensional convolution operation using the sigmoid function.

[0097] The updated acoustic features are adjusted using the channel attention weights to obtain the channel attention acoustic features;

[0098] The updated acoustic features are convolved using two preset matrices to obtain a first convolutional feature map and a second convolutional feature map.

[0099] The first operation result is obtained by performing dimensionality transformation and row and column transpose operations on the first convolutional feature map.

[0100] The second operation result is obtained by performing a dimension transformation operation on the second convolutional feature map;

[0101] The first operation result and the second operation result are multiplied by a matrix, and the result of the matrix multiplication is calculated using the softmax function to obtain the spatial attention weights.

[0102] The updated acoustic features are adjusted using the spatial attention weights to obtain spatial attention acoustic features;

[0103] The attention acoustic features are obtained by averaging the channel attention acoustic features and the spatial attention acoustic features and then fusing them.

[0104] In one embodiment of the present invention, a global average pooling operation is performed on the updated acoustic features to obtain a C×1×1 average pooled feature map. The average pooled feature map is then convolved with a C / r×1×1 convolution to obtain a first result with a dimension of C / r×1×1. The first result is then passed through a ReLU activation function to output a second result with the same dimension as the first result. The second result is then convolved with a C×1×1 convolution to obtain a third result with a dimension of C×1×1. The third result is then passed through a sigmoid function to obtain channel attention weights, where C represents the number of channels and r represents the compression ratio. The channel attention weights are then dot-producted with the updated acoustic features to obtain dot-product acoustic features. Finally, the dot-product acoustic features are added to the updated acoustic features to obtain the channel attention acoustic features.

[0105] In this embodiment of the invention, the attention mechanism module is used to optimize the weights of the acoustic features. This allows data that makes a significant contribution to emotion recognition to be given greater weights, making the features more accurate and thus improving the accuracy of customer emotion category classification in financial and insurance telemarketing.

[0106] In this embodiment of the invention, acoustic features can better reflect the emotional fluctuations of customers. By using a preset classification model to classify the acoustic features from the perspective of emotional fluctuation, a first-dimensional emotion category is obtained. Classifying acoustic features from the perspective of emotional fluctuation can better reflect the emotional fluctuations of customer audio data, which is beneficial for subsequent emotion recognition of audio data.

[0107] S4. Using the preset classification model, classify the text features from the dimension of emotional positivity to obtain the second dimension of emotion category.

[0108] In this embodiment of the invention, the second dimension of emotion category is a category related to the dimension of positive emotion, such as positive emotions such as excitement, love, and calmness, and negative emotions such as shame, boredom, and annoyance that customers experience during the conversation in financial insurance telemarketing.

[0109] In this embodiment of the invention, the method of classifying the text features from the dimension of emotional positivity using the preset classification model to obtain the second dimension of emotional category is the same as the method of classifying the acoustic features from the dimension of emotional volatility using the preset classification model to obtain the first dimension of emotional category, and will not be described in detail here.

[0110] In this embodiment of the invention, acoustic features and text features are classified from the dimensions of emotional fluctuation and emotional positivity, respectively, based on the continuity of emotional expression in audio data and the instantaneity of emotional expression in text. This can more accurately capture the emotional changes of customers, improve the accuracy of emotion recognition, and thus enable timely adjustment of response scripts.

[0111] S5. Analyze the first dimension emotion category and the second dimension emotion category using a preset two-dimensional emotion model to obtain the customer's emotional state.

[0112] For details, please refer to Figure 3 As shown, S5 includes:

[0113] S51. Using a preset two-dimensional emotion model, construct an emotion coordinate system with the emotion fluctuation dimension and the emotion positivity dimension as the horizontal axis and vertical axis.

[0114] S52. Define the correspondence between emotion categories and emotion states in the four quadrants of the emotion coordinate system.

[0115] S53. Based on the correspondence between emotion categories and emotion states in the four quadrants of the emotion coordinate system, identify the emotion states corresponding to the first-dimensional emotion category and the second-dimensional emotion category.

[0116] In one embodiment of the present invention, the four quadrants of the emotion coordinate system can be defined as follows: the first quadrant is joy, the second quadrant is anger, the third quadrant is loss, and the fourth quadrant is calm. For example, in financial insurance telemarketing, when the first dimension of the customer's audio data classification is tension and the second dimension of the text feature classification is positive, the customer is located in the first quadrant of the emotion coordinate system, and their emotional state is joy. In financial insurance telemarketing, when the first dimension of the customer's audio data classification is tension and the second dimension of the text feature classification is annoyance (negative emotion), the customer is located in the second quadrant of the emotion coordinate system, and their emotional state is anger. In financial insurance telemarketing, when the first dimension of the customer's audio data classification is relaxation and the second dimension of the text feature classification is annoyance (negative emotion), the customer is located in the third quadrant of the emotion coordinate system, and their emotional state is loss. In financial insurance telemarketing, when the first dimension of the customer's audio data classification is relaxation and the second dimension of the text feature classification is calm (positive emotion), the customer is located in the third quadrant of the emotion coordinate system, and their emotional state is calm.

[0117] In this embodiment of the invention, the first dimension emotion category and the second dimension emotion category are fused to obtain the customer's emotional state, making the customer's emotional state more accurate.

[0118] S6. Select the corresponding response text to reply to the customer based on the emotional state.

[0119] Specifically, S6 includes:

[0120] Select the corresponding response text based on the emotional state described;

[0121] The response text is converted into voice data;

[0122] The customer is responded to using the voice data.

[0123] In another embodiment of the present invention, the intent of the reply text can also be identified. For example, when the intent of the reply text is to transfer to a human agent, the user can select to transfer to a human agent, or the user can hang up the phone after recognizing that the reply text meets the conditions for hanging up the phone.

[0124] This invention extracts acoustic features from audio data; performs speech recognition on the audio data to obtain speech recognition results, and extracts text features from the speech recognition results; classifies the acoustic features from the dimension of emotional fluctuation using a preset classification model to obtain a first-dimensional emotional category, capturing the continuity of acoustic features and better reflecting the emotional fluctuation of customer audio data; classifies the text features from the dimension of emotional positivity using the preset classification model to obtain a second-dimensional emotional category, capturing the instantaneity of text features and better reflecting emotional positivity; analyzes the first-dimensional emotional category and the second-dimensional emotional category using a preset two-dimensional emotion model to obtain the customer's emotional state. This combines the different features of both acoustic and text features, improving the limitations of traditional emotion analysis using single-feature data, more accurately grasping a person's emotional state, and selecting the corresponding response text to reply to the customer based on the emotional state, thereby improving the user experience and the effectiveness of intelligent outbound calling in financial and insurance telemarketing systems. Therefore, the intelligent outbound calling method based on emotion recognition proposed in this invention can solve the problem of inaccurate customer emotion recognition during intelligent outbound calling in financial and insurance telemarketing systems.

[0125] like Figure 4 The diagram shown is a functional block diagram of an intelligent outbound calling device based on emotion recognition provided in an embodiment of the present invention.

[0126] The emotion-recognition-based intelligent outbound calling device 100 of this invention can be installed in an electronic device. Depending on the functions implemented, the emotion-recognition-based intelligent outbound calling device 100 may include a feature extraction module 101, an emotion classification module 102, an emotion fusion module 103, and a response selection module 104. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0127] In this embodiment, the functions of each module / unit are as follows:

[0128] The feature extraction module 101 is used to acquire the customer's audio data, extract the acoustic features of the audio data, perform speech recognition on the audio data, obtain the speech recognition result, and extract the text features of the speech recognition result.

[0129] The emotion classification module 102 is used to classify the acoustic features from the dimension of emotion fluctuation using a preset classification model to obtain a first-dimensional emotion category; and to classify the text features from the dimension of emotion positivity using the preset classification model to obtain a second-dimensional emotion category.

[0130] The emotion fusion module 103 is used to analyze the first dimension emotion category and the second dimension emotion category using a preset two-dimensional emotion model to obtain the customer's emotional state.

[0131] The response selection module 104 is used to select the corresponding response text to reply to the customer based on the emotional state.

[0132] In detail, the modules described in the emotion-recognition-based intelligent outbound calling device 100 in this embodiment of the invention employ the same methods as described above during use. Figures 1 to 3 The method uses the same technical means as the emotion recognition-based intelligent outbound calling method described in the article and can produce the same technical effect, so it will not be repeated here.

[0133] like Figure 5 The diagram shown is a structural schematic of an electronic device that implements an intelligent outbound calling method based on emotion recognition, according to an embodiment of the present invention.

[0134] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as an intelligent outbound calling program based on emotion recognition.

[0135] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing an intelligent outbound calling program based on emotion recognition) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0136] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of an intelligent outbound calling program based on emotion recognition, but also to temporarily store data that has been output or will be output.

[0137] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0138] The communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a client interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The client interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual client interface.

[0139] Figure 5 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 5The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0140] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0141] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0142] The intelligent outbound calling program based on emotion recognition stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0143] Acquire customer audio data and extract the acoustic features of the audio data;

[0144] The audio data is subjected to speech recognition to obtain speech recognition results, and the text features of the speech recognition results are extracted.

[0145] The acoustic features are classified from the dimension of emotional fluctuation using a preset classification model to obtain the first dimension of emotional category;

[0146] The text features are classified from the perspective of emotional positivity using the preset classification model to obtain the second dimension of emotion category;

[0147] The customer's emotional state is obtained by analyzing the first dimension of the emotion category and the second dimension of the emotion category using a preset two-dimensional emotion model.

[0148] Select the corresponding response text to reply to the customer based on their emotional state.

[0149] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.

[0150] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0151] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0152] Acquire customer audio data and extract the acoustic features of the audio data;

[0153] The audio data is subjected to speech recognition to obtain speech recognition results, and the text features of the speech recognition results are extracted.

[0154] The acoustic features are classified from the dimension of emotional fluctuation using a preset classification model to obtain the first dimension of emotional category;

[0155] The text features are classified from the perspective of emotional positivity using the preset classification model to obtain the second dimension of emotion category;

[0156] The customer's emotional state is obtained by analyzing the first dimension of the emotion category and the second dimension of the emotion category using a preset two-dimensional emotion model.

[0157] Select the corresponding response text to reply to the customer based on their emotional state.

[0158] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0159] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0161] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0162] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0163] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0164] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0165] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent outbound calling method based on emotion recognition, characterized in that, The method includes: Acquire customer audio data and extract the acoustic features of the audio data; The audio data is subjected to speech recognition to obtain speech recognition results, and the text features of the speech recognition results are extracted. The acoustic features are classified from the dimension of emotional fluctuation using a preset classification model to obtain the first dimension of emotional category; The text features are classified from the perspective of emotional positivity using the preset classification model to obtain the second dimension of emotion category; The customer's emotional state is obtained by analyzing the first dimension of the emotion category and the second dimension of the emotion category using a preset two-dimensional emotion model. Select the corresponding response text to reply to the customer based on their emotional state; The acoustic features are classified according to the emotional fluctuation dimension using a preset classification model to obtain the first dimension of emotional categories, including: The acoustic features are obtained by using a recurrent neural network in a preset classification model to acquire contextual feature information, and the acoustic features are updated using the contextual feature information to obtain the updated acoustic features. The updated acoustic features are weighted and adjusted using the attention mechanism module in the preset classification model to obtain attention acoustic features. The attention acoustic features are standardized, convolved, and pooled. The softmax function is used to calculate the probability of the first-dimensional emotion category on the pooled result. The category with the highest probability is taken as the first-dimensional emotion category. The step of using the attention mechanism module in the preset classification model to adjust the weights of the updated acoustic features to obtain attention acoustic features includes: The updated acoustic features are subjected to average pooling to obtain one-dimensional acoustic features; A one-dimensional convolution operation is performed on the one-dimensional acoustic features, and the channel attention weights are obtained by calculating the result of the one-dimensional convolution operation using the sigmoid function. The updated acoustic features are adjusted using the channel attention weights to obtain channel attention acoustic features; The updated acoustic features are convolved using two preset matrices to obtain a first convolutional feature map and a second convolutional feature map. The first operation result is obtained by performing dimensionality transformation and row and column transpose operations on the first convolutional feature map. The second operation result is obtained by performing a dimension transformation operation on the second convolutional feature map; The spatial attention weights are obtained by performing matrix multiplication on the result of the first operation and the result of the second operation, and then using the softmax function to calculate the result of the matrix multiplication. The spatial attention acoustic features are obtained by adjusting the updated acoustic features using the spatial attention weights. The attention acoustic features are obtained by averaging the channel attention acoustic features and the spatial attention acoustic features and then fusing them.

2. The intelligent outbound calling method based on emotion recognition as described in claim 1, characterized in that, The step of analyzing the first and second dimension emotion categories using a preset two-dimensional emotion model to obtain the customer's emotional state includes: Using a pre-defined two-dimensional emotion model, an emotion coordinate system is constructed with the emotion fluctuation dimension and the emotion positivity dimension as the horizontal axis and the vertical axis, respectively. Define the correspondence between emotion categories and emotion states in the four quadrants of the emotion coordinate system; Based on the correspondence between emotion categories and emotion states in the four quadrants of the emotion coordinate system, the emotion states corresponding to the first-dimensional emotion category and the second-dimensional emotion category are identified.

3. The intelligent outbound calling method based on emotion recognition as described in claim 1, characterized in that, The step of selecting the corresponding response text to reply to the customer based on the emotional state includes: Select the corresponding response text based on the emotional state described; The response text is converted into voice data; The customer is responded to using the voice data.

4. The intelligent outbound calling method based on emotion recognition as described in claim 1, characterized in that, The step of performing speech recognition on the audio data to obtain the speech recognition result includes: The audio data is subjected to noise reduction processing; The audio data after noise reduction is segmented and windowed to obtain multiple speech frames. Extract speech features from each speech frame one by one; Using a preset speech recognizer, multiple candidate words with different scores are generated for each text word in the audio data based on the speech features of each speech frame; All candidate words are combined to form the speech recognition result.

5. The intelligent outbound calling method based on emotion recognition as described in claim 1, characterized in that, The extraction of acoustic features from the audio data includes: The audio data is pre-emphasized using a high-pass filter; The pre-emphasized audio data is subjected to frame-by-frame windowing processing to obtain windowed audio frames; Perform a Fast Fourier Transform on the windowed audio frame to obtain the corresponding energy spectrum; The energy spectrum is smoothed and harmonics are eliminated using a triangular bandpass filter to obtain an optimized energy spectrum. The logarithmic energy spectrum of the optimized energy spectrum is calculated, and the discrete cosine transform is performed on the logarithmic energy spectrum to obtain speech feature parameters. The first-order and second-order dynamic difference parameters of the speech feature parameters are extracted as speech feature vectors.

6. An intelligent outbound calling device based on emotion recognition, used to implement the intelligent outbound calling method based on emotion recognition as described in any one of claims 1 to 5, characterized in that, The device includes: The feature extraction module is used to acquire customer audio data, extract the acoustic features of the audio data, perform speech recognition on the audio data, obtain speech recognition results, and extract the text features of the speech recognition results. The emotion classification module is used to classify the acoustic features from the dimension of emotion fluctuation using a preset classification model to obtain a first-dimensional emotion category; and to classify the text features from the dimension of emotion positivity using the preset classification model to obtain a second-dimensional emotion category. The emotion fusion module is used to analyze the first dimension emotion category and the second dimension emotion category using a preset two-dimensional emotion model to obtain the customer's emotional state; The select response module is used to select the corresponding response text to reply to the customer based on the emotional state.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the intelligent outbound calling method based on emotion recognition as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent outbound calling method based on emotion recognition as described in any one of claims 1 to 5.

Images