User emotion perception method and device, equipment, medium and product

Abstract

The invention provides a user emotion perception method and device, equipment, a medium and a product, and relates to the technical field of voice signal processing, and the method comprises the steps: obtaining a target voice signal of a target user and historical behavior data of the target user; obtaining a target feature in the target voice signal, and cutting the target voice signal according to the target feature to obtain multiple segments of effective voice signals; and determining a target emotion tag of the target user according to the multiple segments of effective voice signals and historical behavior data of the target user. According to the invention, while the emotion features are directly extracted from the voice signal, the historical behavior data of the speaker is combined to carry out auxiliary judgment on the emotion, so that the accuracy of emotion judgment is further improved, the service and the product are optimized, and the user satisfaction is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of speech signal processing, and in particular to a user emotion perception method, device, equipment, medium and product. BACKGROUND

[0002] Speech emotion recognition refers to classifying the emotion of a speech in a speech. In the prior art, speech recognition models are used to determine the emotion of a user, such as a Contextual Long Short-Term Memory (C-LSTM) model based on context modeling, a CMN model based on speaker modeling, and a Multimodal Emotion Recognition Dataset (MELD) model. However, the foregoing models still have the following shortcomings: first, lack of real-time performance, i.e., due to the low computational efficiency of effective speech recognition, the emotion recognition result cannot be given in time while the user is speaking; second, lack of accuracy, i.e., different emotion categories and degrees cannot be accurately distinguished, but only rough classification or scoring can be given; and third, lack of universality, i.e., different business types and different product uses cannot be adapted to the characteristics of users, but need to be trained and optimized for specific user groups. SUMMARY

[0003] Embodiments of the present application provide a user emotion perception method, device, equipment, medium and product to solve the problem of lack of real-time performance, accuracy and universality of existing speech recognition models.

[0004] To solve the above technical problems, the present application is implemented as follows:

[0005] In a first aspect, embodiments of the present application provide a user emotion perception method, comprising:

[0006] obtaining a target speech signal of a target user and historical behavior data of the target user;

[0007] obtaining a target feature in the target speech signal, and cutting the target speech signal according to the target feature to obtain a plurality of effective speech signals;

[0008] determining a target emotion label of the target user according to the plurality of effective speech signals and the historical behavior data of the target user.

[0009] Optionally, the obtaining a target feature in the target speech signal according to the target speech signal, and cutting the target speech signal according to the target feature to obtain a plurality of effective speech signals comprises:

[0010] determine a speech frame length of the target speech signal according to a sampling rate of the sound, cut the target speech signal according to the speech frame length to obtain a plurality of first speech signals;

[0011] convert the first speech signal from a time domain signal to a frequency domain signal, and determine a target feature of each frame of the first speech signal, the target feature including energy and a zero-crossing rate;

[0012] if the energy of the first speech signal meets a preset energy threshold and the zero-crossing rate meets a preset zero-crossing rate threshold, determine that the first speech signal is a valid speech signal.

[0013] Optionally, the preset energy threshold is determined according to an average energy of all first speech signals, and the zero-crossing rate threshold is determined according to an average zero-crossing rate of all first speech signals.

[0014] Optionally, the historical behavior data includes at least one of product ordering information, historical transaction information, historical call records, product preferences, and basic portrait information of the user.

[0015] Optionally, determining a target emotion label of the target user according to the plurality of valid speech signals and the historical behavior data of the target user includes:

[0016] obtaining acoustic features and semantic features in the valid speech signal, and splicing the acoustic features and the semantic features in a time step alignment manner to obtain a plurality of first feature vectors, and splicing the plurality of first feature vectors to obtain a first feature sequence;

[0017] inputting the first feature sequence into a perception prediction model, the perception prediction model outputting a time sequence feature according to the first feature sequence, wherein the time sequence feature includes a hidden state of each time step in the first feature sequence, and obtaining a second feature vector according to the time sequence feature and the historical behavior data of the target user;

[0018] inputting the second feature vector into an emotion classification model, the emotion classification model classifying the obtained second feature vector in terms of emotion tendency and outputting an emotion prediction vector;

[0019] obtaining input parameters of a multi-layer perception machine according to the emotion prediction vector;

[0020] inputting the input parameters into the multi-layer perception machine, the multi-layer perception machine outputting the target emotion label.

[0021] Optionally, obtaining the input parameters of the multi-layer perception machine according to the emotion prediction vector includes:

[0022] determine a similarity of each of the first feature vectors to all elements in the first feature sequence, and determine an attention weight according to the similarity, and obtain a weighted context vector according to the attention weight and the corresponding first feature vector;

[0023] obtain the input parameter according to the weighted context vector and the sentiment pre-judgment vector.

[0024] In a second aspect, an apparatus for user emotion perception is provided, comprising:

[0025] a first obtaining module configured to obtain a target voice signal of a target user and historical behavior data of the target user;

[0026] a first processing module configured to obtain target features in the target voice signal, and cut the target voice signal according to the target features to obtain a plurality of valid voice signals;

[0027] a second processing module configured to determine a target emotion label of the target user according to the plurality of valid voice signals and the historical behavior data of the target user.

[0028] Optionally, the first processing module comprises:

[0029] a first processing submodule configured to determine a voice frame length of the target voice signal according to a sampling rate of the voice, and cut the target voice signal according to the voice frame length to obtain a plurality of first voice signals;

[0030] a second processing submodule configured to convert the first voice signals from time domain signals to frequency domain signals, and determine target features of each of the first voice signals, the target features comprising energy and zero-crossing rate; if the energy of the first voice signal satisfies a preset energy threshold and the zero-crossing rate satisfies a preset zero-crossing rate threshold, the first voice signal is determined as a valid voice signal.

[0031] Optionally, the preset energy threshold is determined according to an average energy of all first voice signals, and the zero-crossing rate threshold is determined according to an average zero-crossing rate of all first voice signals.

[0032] Optionally, the historical behavior data comprises at least one of the following: product ordering information of the user, historical transaction information, historical call records, product preferences, and basic portrait information of the user.

[0033] Optionally, the second processing module comprises:

[0034] The third processing sub-module is configured to acquire acoustic features and semantic features in the valid speech signal, splice the acoustic features and the semantic features in a time step alignment manner to obtain a plurality of first feature vectors, and splice the plurality of first feature vectors to obtain a first feature sequence;

[0035] The first model processing sub-module is configured to input the first feature sequence into a perception prediction model, and output a time sequence feature from the perception prediction model according to the first feature sequence, wherein the time sequence feature comprises a hidden state of each time step in the first feature sequence, and obtain a second feature vector from the time sequence feature and historical behavior data of the target user;

[0036] The second model processing sub-module is configured to input the second feature vector into an emotion classification model, and output an emotion prediction vector from the emotion classification model according to an emotion tendency of the second feature vector;

[0037] The fourth processing sub-module is configured to obtain an input parameter of a multi-layer perception machine according to the emotion prediction vector.

[0038] The third model processing sub-module is configured to input the input parameter into the multi-layer perception machine, and output the target emotion label from the multi-layer perception machine.

[0039] Optionally, the fourth processing sub-module comprises:

[0040] The fifth processing sub-module is configured to determine a similarity between each first feature vector and all elements of the first feature sequence, determine an attention weight according to the similarity, obtain a weighted context vector according to the attention weight and the corresponding first feature vector, and obtain the input parameter according to the weighted context vector and the emotion prediction vector.

[0041] In a third aspect, an electronic device is provided, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, and the program or instructions, when executed by the processor, implement the steps in the method for user emotion perception according to any one of the first aspect.

[0042] In a fourth aspect, a readable storage medium is provided, which stores a program or instructions, and the program or instructions, when executed by a processor, implement the steps in the method for user emotion perception according to any one of the first aspect.

[0043] In a fifth aspect, a computer program product is provided, which includes computer instructions, and the computer instructions, when executed by a processor, implement the steps in the method for user emotion perception according to any one of the first aspect.

[0044] In the present application, the target speech signal of the target user and the historical behavior data of the target user are acquired; so as to subsequently extract the emotional features directly from the speech signal while combining the historical behavior data of the speaker for auxiliary determination of the emotion; the target features in the target speech signal are acquired, and the target speech signal is cut according to the target features to obtain multiple effective speech signals; and the target emotional label of the target user is determined according to the multiple effective speech signals and the historical behavior data of the target user. By combining the historical behavior data of the speech signal of the user to jointly determine the target emotional label of the target user, the accuracy of emotion judgment is further improved, the service and product are optimized, the user satisfaction is improved, and the problems of lack of real-time performance, accuracy and universality of the existing speech recognition model are solved. BRIEF DESCRIPTION OF DRAWINGS

[0045] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments with reference made to the accompanying drawings. The drawings are for purposes of illustration only and are not intended to be limiting in any respect. Moreover, the use of the same reference symbols in different drawings indicates similar or identical items.

[0046] Figure 1 is a flowchart of a user emotion perception method provided by an embodiment of the present application;

[0047] Figure 2 is a data combination schematic diagram of a user emotion perception method provided by an embodiment of the present application;

[0048] Figure 3 is a model training schematic diagram of a user emotion perception method provided by an embodiment of the present application;

[0049] Figure 4 is a structural schematic diagram of a user emotion perception device provided by an embodiment of the present application;

[0050] Figure 5 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0052] Reference should be made to Figure 1The embodiment of the present application provides a user emotion perception method, comprising:

[0053] Step 11: obtaining a target voice signal of a target user and historical behavior data of the target user;

[0054] In the embodiment of the present application, before obtaining the target voice signal of the target user and the historical behavior data of the target user, the target user needs to be authenticated, which can be but is not limited to guiding the user through text or voice, so as to authenticate the identity of the user according to the user's mobile phone number, user number or order number and the like, thereby effectively preventing unauthorized access, protecting the privacy and data security of the user and preventing information leakage; and the voice signal of the target user can be monitored in real time in the user interaction process, so as to realize dynamic updating of the emotion label of the user and provide instant emotional analysis feedback.

[0055] In the embodiment of the present application, optionally, the historical behavior data comprises at least one of the following: product ordering information of the user, historical transaction information, historical call records, product preferences and basic portrait information of the user.

[0056] In the embodiment of the present application, after the identity of the user is confirmed, the historical behavior data of the target user is screened from a user data pool according to the identity information of the target user, the user data pool is a pre-trained static data set, and a data set formed by sorting the business data of all users, mainly comprising: product ordering information of the user, historical transaction information, historical call records, product preferences and basic portrait information of the user, so as to jointly determine the target emotion label of the target user by combining the historical behavior data and the voice signal of the user in the future, further improving the accuracy of emotion judgment, optimizing services and products and improving user satisfaction.

[0057] Step 12: obtaining a target feature in the target voice signal, and cutting the target voice signal according to the target feature to obtain a plurality of effective voice signals;

[0058] In the embodiment of the present application, the target feature in the target voice signal, such as energy and zero-crossing rate, judges the effective part and the invalid part in the voice signal, and when judging, the voice signal is cut, that is, the long-time voice data is decomposed into smaller segments, so as to reduce the consumption of computing resources and improve the processing speed; and cutting the voice signal can help to remove irrelevant background noise and focus on effective voice information, so as to improve the accuracy of voice recognition and analysis.

[0059] Step 13: determining a target emotion label of the target user according to the plurality of effective voice signals and the historical behavior data of the target user.

[0060] In the embodiment of the present application, the extracted acoustic features can be used as input to train an emotion classification model, and the corresponding emotion labels (such as happiness, sadness, anger, surprise, etc.) can be output by combining the historical behavior data of the target user. By combining multiple data sources, the system can more comprehensively understand the emotional state of the user, avoid misjudgment and false response, improve the accuracy of emotion recognition, reduce the deviation caused by a single signal, accurately identify the user's emotion, provide more targeted services and suggestions for the user, and enhance the overall experience of the user.

[0061] In the embodiment of the present application, the target voice signal of the target user and the historical behavior data of the target user are obtained, so that the emotion features can be extracted from the voice signal directly in the subsequent process, and the historical behavior data of the speaker is combined for auxiliary determination of emotion. The target features in the target voice signal are obtained, and the target voice signal is cut to obtain a plurality of effective voice signals according to the target features. The target emotion label of the target user is determined according to the plurality of effective voice signals and the historical behavior data of the target user. By combining the historical behavior data of the user's voice signal to determine the target emotion label of the target user, the accuracy of emotion judgment is further improved, the service and product are optimized, the user satisfaction is improved, and the problem of lack of real-time, accuracy and universality of the existing voice recognition model is solved.

[0062] In the embodiment of the present application, optionally, the target features in the target voice signal are obtained according to the target voice signal, and the target voice signal is cut to obtain a plurality of effective voice signals according to the target features, comprising:

[0063] The length of the voice frame of the target voice signal is determined according to the sampling rate of the sound, and the target voice signal is cut according to the length of the voice frame to obtain a plurality of first voice signals;

[0064] The first voice signal is converted from a time domain signal to a frequency domain signal, and the target features of each frame of the first voice signal are determined, the target features comprising: energy and zero-crossing rate;

[0065] If the energy of the first voice signal meets the preset energy threshold, and the zero-crossing rate meets the preset zero-crossing rate threshold, the first voice signal is determined as an effective voice signal.

[0066] In the embodiment of the present application, since the characteristics of the signal (such as frequency component, amplitude and phase) are more obvious in the frequency domain, the first voice signal can be converted from a time domain signal to a frequency domain signal for subsequent better extraction and analysis. Specifically, but not limited to, the time domain signal can be converted to a frequency domain signal using discrete wavelet transform or Fourier transform to realize real-time processing of the voice.

[0067] Specifically, before transformation, first determine a value closest to 2 raised to an integer power as the speech frame length according to the sampling rate of the sound Obtaining multiple frames of first speech signals, selecting data blocks of power-of-2 length can more efficiently process and store, thereby improving computational efficiency and being more helpful for processing real-time speech signals

[0068] Converting the first speech signals from time-domain signals to frequency-domain signals, specifically including:

[0069] Applying a window function to each frame for windowing to reduce discontinuity at the frame boundary:

[0070] x w (n)=x(n)·w(n)

[0071] Where x(n) is the original time-domain signal; w(n) is the window function; x w (n) is the windowed signal; n is the discrete time index, representing the values of the signal x(n) and the window function w(n) at the discrete time point n;

[0072] Performing discrete wavelet transform (DWT) on each windowed speech signal:

[0073] First, select appropriate wavelet basis functions and scaling functions to generate low-pass filter coefficients h(n) and high-pass filter coefficients g(n):

[0074] h(n)=∫φ(t)φ(t-n)dt

[0075] g(n)=∫ψ(t)φ(t-n)dt

[0076] Where t is a continuous variable representing time, and the integral operation is performed on time t, which aims to calculate the degree of overlap of the scaling function φ(t) and the wavelet basis function ψ(t) at different time points, thereby generating filter coefficients;

[0077] Using the generated corresponding filter coefficients to perform convolution operation on the windowed signal, decomposing the signal into low-frequency and high-frequency components, specifically as follows:

[0078] Low-frequency component:

[0079]

[0080] High-frequency component:

[0081]

[0082] Wherein, c(j+1, k) is the low frequency coefficient of the j+1 layer; d(j+1, k) is the high frequency coefficient of the j+1 layer; j is the scale parameter controlling the stretching of the wavelet transform, and k is the translation parameter controlling the position of the wavelet transform on the time axis.

[0083] In the embodiment of the present application, optionally, the preset energy threshold is determined according to the average energy of all first speech signals, and the zero-crossing rate threshold is determined according to the average zero-crossing rate of all first speech signals.

[0084] Specifically, the target feature of each frame of the first speech signal is determined, including:

[0085] The energy and the zero-crossing rate of each frame are calculated, and the energy E is:

[0086]

[0087] Wherein, x(n) is the frame signal after DWT; the zero-crossing rate ZCR is:

[0088]

[0089] Wherein (sgn) is a sign function, and (x(n)) is the frame signal after DWT;

[0090] The average energy and the average zero-crossing rate of all speech frames are calculated, and the energy threshold and the zero-crossing rate threshold are determined according to an empirical formula;

[0091]

[0092] E t =k1E m

[0093] Z t =k2Z m

[0094] Wherein, E i is the energy of the i-th speech frame, Z i is the zero-crossing rate of the i-th speech frame, N is the number of all speech frames, E m is the average energy, Z m is the average zero-crossing rate, E t is the energy threshold, Z t is the zero-crossing rate threshold, and k1 and k2 are empirical coefficients.

[0095] Specifically, if the energy of the first speech signal meets the preset energy threshold and the zero-crossing rate meets the preset zero-crossing rate threshold, the first speech signal is determined as an effective speech signal, including:

[0096] For each speech frame, if the energy and zero-crossing rate are greater than a threshold, the frame is considered as valid speech; otherwise, the frame is considered as invalid noise; for continuous valid speech frames or invalid noise frames, if the number of frames exceeds a given minimum length, the segment is considered as speech activity or silence state; otherwise, the segment is considered as a transition state.

[0097]

[0098] wherein S i is the valid speech flag of the i-th speech frame, T i is the speech activity flag of the i-th speech frame, and L is the minimum length. According to the above judgment, the start and end points of each segment of speech or silence state are determined.

[0099] In the embodiment of the present application, by effectively identifying and excluding the silence state and noise frames, the interference on the speech recognition system can be reduced, thereby improving the accuracy of recognition, and only processing valid speech frames, the processing of irrelevant information is reduced, and the efficiency of the system is improved; the clear start and end points can also facilitate subsequent speech segmentation and analysis, and facilitate speech synthesis, emotion analysis and other tasks, and the clear boundaries of speech and silence state help to improve the accuracy and consistency of labeling.

[0100] In the embodiment of the present application, optionally, the target emotion label of the target user is determined according to the plurality of valid speech signals and historical behavior data of the target user, comprising:

[0101] Acoustic features and semantic features in the valid speech signal are obtained, and the acoustic features and semantic features are spliced in a time step alignment manner to obtain a plurality of first feature vectors, and the plurality of first feature vectors are spliced to obtain a first feature sequence;

[0102] The first feature sequence is input into a perception prediction model, and the perception prediction model outputs a time sequence feature according to the first feature sequence, wherein the time sequence feature includes a hidden state of each time step in the first feature sequence, and a second feature vector is obtained according to the time sequence feature and the historical behavior data of the target user;

[0103] The second feature vector is input into an emotion classification model, and the emotion classification model classifies the obtained second feature vector according to emotion tendency to output an emotion prediction vector;

[0104] Input parameters of a multi-layer perception machine are obtained according to the emotion prediction vector;

[0105] The input parameters are input into the multi-layer perception machine, and the multi-layer perception machine outputs the target emotion label.

[0106] In the embodiment of the present application, for each piece of voice signal S i after cutting, the Mel frequency cepstral coefficient (MFCC) technology is used to extract the acoustic feature vector F i , and the automatic speech recognition (ASR) technology is used to convert S i into a set of word vectors W i representing semantic content and voice text, the key information related to the user's business is extracted from the text converted from the voice by the pre-trained model, and the historical data of the user in the user pool is matched, as shown in Figure 2 , the emotion tendency judgment is generated according to the deviation degree of the voice text information "I hope to change the package to 90 yuan" and the historical data "the current package is 200 yuan", that is, "downgrade -> negative deviation", so as to more comprehensively understand the emotional state of the user, avoid misjudgment and wrong response, improve the accuracy of emotion recognition, reduce the deviation caused by a single signal, accurately identify the user's emotion, provide more targeted services and suggestions for the user, and enhance the overall experience of the user.

[0107] Before using the above model, a large amount of voice data and corresponding emotion labels are collected, and relevant features such as transaction records and query history are extracted from the historical data of the user, and the above model is trained, as shown in Figure 3 , first, the acoustic features and semantic features of the collected data are taken as inputs to train a perception prediction model RNN to capture the time sequence characteristics of the emotion; then, the historical data features of the user and the output of the RNN model are taken as inputs to train an emotion classification model RF to classify the emotion tendency, and finally the most possible emotion label is output.

[0108] In actual use, specifically, the acoustic features and semantic features in the effective voice signal are acquired, and the acoustic features and semantic features are spliced in a time step alignment manner to obtain a plurality of first feature vectors, and the plurality of first feature vectors are spliced to obtain a first feature sequence, comprising:

[0109] The extracted acoustic features F and semantic features W are spliced in a time step alignment manner to generate a combined feature vector of the time step, that is, a first feature vector X.

[0110] Suppose: at each time step t (t = 1, 2,..., T), the corresponding acoustic feature vector F t and semantic feature vector W t are spliced to form a combined feature vector X t of the time step, that is, X t = [F t , W t]; and then the entire feature sequence X={x1, x2,..., x T} is obtained by concatenating in sequence.

[0111] Specifically, the first feature sequence is input into a perception prediction model, and the perception prediction model outputs a time sequence feature according to the first feature sequence, wherein the time sequence feature comprises a hidden state of each time step in the first feature sequence, and a second feature vector is obtained according to the time sequence feature and historical behavior data of the target user, comprising:

[0112] The first feature vector X is input into the perception prediction model RNN, and the hidden state h t of the first feature vector X is updated by the following formula:

[0113] h t =RNN(h t-1 ,x t )

[0114] The time sequence feature H={h1, h2,..., h T} is output, wherein h t is a hidden state of a time step t, RNN is an activation function, each hidden state h t in the time sequence feature H output by the RNN is combined with a user historical data feature u t to form a second feature vector z t =[h t ;u t ].

[0115] Specifically, the second feature vector is input into an emotion classification model, the emotion classification model classifies the obtained second feature vector in terms of emotional tendency, and outputs an emotional prediction vector, comprising:

[0116] The second feature vector is input into an emotion classification model RF model to classify the emotional tendency:

[0117] p t =RF(z t )

[0118] wherein p t is an emotional prediction vector, representing a probability distribution of different emotional tendencies, and the emotion classification model RF is a pre-trained random forest model based on historical data of business users and labeled.

[0119] In the embodiment of the application, optionally, the input parameters of the multilayer perceptron obtained according to the emotional prediction vector comprise:

[0120] determine the similarity of each first feature vector with all elements in the first feature sequence, and determine the attention weight according to the similarity, and obtain a weighted context vector according to the attention weight and the corresponding first feature vector;

[0121] obtain the input parameter according to the weighted context vector and the sentiment prediction vector.

[0122] Specifically, the input parameter of a multi-layer perceptron is obtained according to the sentiment prediction vector, the input parameter is input into the multi-layer perceptron, and the multi-layer perceptron outputs the target emotion label, including:

[0123] use a self-attention mechanism to capture the input sequence x t The long-distance dependence relationship enhances the understanding of complex relationships in user voice and behavior data, better understands and predicts the emotional state of the user, and the specific logic is as follows:

[0124] For each element x t in the sequence, calculate the similarity score e tj of all elements in the sequence:

[0125]

[0126] wherein W is a trainable weight matrix, which is used for linear transformation of the input vector; j is an index variable, which is used to represent the position of the element in the sequence;

[0127] use a softmax function to calculate the attention weight a tj :

[0128] Finally, the weighted context vector c t is calculated:

[0129] The self-attention mechanism understands complex relationships and enhances the accuracy of emotion recognition. At the same time, the prediction vector is used to improve the calculation speed of emotion judgment, and the overall real-time performance of the model is enhanced.

[0130] Therefore, the weighted context vector c t and the sentiment prediction vector p t are combined to form a new feature vector, i.e. the input parameter: v t = [c t ; p t ];

[0131] v tInput into a multi-layer perception (MLP), which is a neural network composed of multiple fully connected layers, which presets the parameters of the emotion label according to the pre-trained data model, for example, 0 represents "neutral", 1 represents "happy", 2 represents "doubt", 3 represents "angry", etc. Through the MLP model, the input data can be nonlinearly transformed and classified, and the probability distribution of the emotion label y is output: t =MLP(v t ), to obtain the final target emotion label.

[0132] In the embodiment of the present application, the acoustic features and semantic features in the speech are extracted, and at the same time, user behavior data and semantic data are introduced for judgment, and the model of recurrent neural network and random forest is trained for hybrid recognition. This method combines the advantages of RNN model in processing time series data and the high efficiency of random forest in classification task, so that the result can be more accurate and fast in the emotion recognition task, and the user's emotional state can be better understood and recognized, providing a more intelligent solution for human-computer interaction. In practical use, it can be applied to customer service system, intelligent voice system, for example, monitoring the service quality of intelligent voice robot, discovering customer negative emotions in time, or realizing reply generation based on user emotion driving, improving customer service quality and satisfaction.

[0133] Please refer to Figure 4 , the embodiment of the present application provides a user emotion perception device, comprising:

[0134] The first acquisition module 41 is used for acquiring the target speech signal of the target user and the historical behavior data of the target user;

[0135] The first processing module 42 is used for acquiring the target feature in the target speech signal, and cutting the target speech signal according to the target feature to obtain a plurality of effective speech signals;

[0136] The second processing module 43 is used for determining the target emotion label of the target user according to the plurality of effective speech signals and the historical behavior data of the target user.

[0137] In the embodiment of the present application, optionally, the first processing module comprises:

[0138] The first processing sub-module is used for determining the speech frame length of the target speech signal according to the sampling rate of the sound, cutting the target speech signal according to the speech frame length, and obtaining a plurality of first speech signals;

[0139] The second processing submodule is configured to convert the first voice signal from a time domain signal to a frequency domain signal, and determine a target feature of each frame of the first voice signal, wherein the target feature comprises energy and a zero-crossing rate; if the energy of the first voice signal satisfies a preset energy threshold, and the zero-crossing rate satisfies a preset zero-crossing rate threshold, it is determined that the first voice signal is a valid voice signal.

[0140] In an embodiment of the present application, optionally, the preset energy threshold is determined according to the average energy of all first voice signals, and the zero-crossing rate threshold is determined according to the average zero-crossing rate of all first voice signals.

[0141] In an embodiment of the present application, optionally, the historical behavior data comprises at least one of the following: product ordering information of the user, historical transaction information, historical call records, product preferences and basic portrait information of the user.

[0142] In an embodiment of the present application, optionally, the second processing module comprises:

[0143] The third processing submodule is configured to obtain acoustic features and semantic features in the valid voice signal, and splice the acoustic features and the semantic features in a time step alignment manner to obtain a plurality of first feature vectors, and splice the plurality of first feature vectors to obtain a first feature sequence;

[0144] The first model processing submodule is configured to input the first feature sequence into a perception prediction model, wherein the perception prediction model outputs a time sequence feature according to the first feature sequence, and the time sequence feature comprises a hidden state of each time step in the first feature sequence; and a second feature vector is obtained according to the time sequence feature and historical behavior data of the target user.

[0145] The second model processing submodule is configured to input the second feature vector into an emotion classification model, wherein the emotion classification model classifies the obtained second feature vector according to an emotional tendency, and outputs an emotional prediction vector.

[0146] The fourth processing submodule is configured to obtain an input parameter of a multi-layer perception machine according to the emotional prediction vector.

[0147] The third model processing submodule is configured to input the input parameter into the multi-layer perception machine, and the multi-layer perception machine outputs the target emotional label.

[0148] In an embodiment of the present application, optionally, the fourth processing submodule comprises:

[0149] The fifth processing submodule is used to determine the similarity between each of the first feature vectors and all elements in the first feature sequence, determine the attention weight based on the similarity, and obtain a weighted context vector based on the attention weight and the corresponding first feature vector; and obtain the input parameters based on the weighted context vector and the sentiment prediction vector.

[0150] The user emotion perception device provided in this embodiment of the invention can achieve... Figure 1 The various processes implemented in the method embodiments achieve the same technical effect, and will not be described again here to avoid repetition.

[0151] This invention provides an electronic device 50, see [link to relevant documentation]. Figure 5 As shown, Figure 5 This is a schematic diagram of an electronic device 50 according to an embodiment of the present invention, including a processor 51, a memory 52, and a program or instructions stored in the memory 52 and executable on the processor 51. When the program or instructions are executed by the processor, they implement the steps of any of the user emotion perception methods of the present invention.

[0152] This invention provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements various processes of the user emotion perception method as described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0153] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0154] Computer-readable media includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0155] It should be noted that in the technical solutions of the present disclosure, the collection, collection, update, analysis, processing, use, transmission, storage and other aspects of user personal information are in line with relevant laws and regulations, are used for legal purposes, and do not violate public order and good customs. Necessary measures are taken to prevent illegal access to user personal information data, and user personal information security and network security are maintained.

[0156] It should be noted that in this paper, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0157] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0158] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, also can be through hardware, but in many cases the former is the better embodiment. Based on such understanding, the technical scheme of the present application essentially or the part which contributes to the prior art can be embodied in the form of software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions to make a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the method described in each embodiment of the present application.

[0159] The above only describes the preferred embodiments of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A method of user sentiment perception, the method comprising: The method comprises the following steps: obtaining a target voice signal of a target user and historical behavior data of the target user; obtaining a target feature in the target voice signal, and cutting the target voice signal according to the target feature to obtain a plurality of effective voice signals; determining a target emotion label of the target user according to the plurality of effective voice signals and the historical behavior data of the target user.

2. The method of user sentiment perception according to claim 1, wherein, The method further comprises the following steps: determining a voice frame length of the target voice signal according to a sampling rate of the voice, and cutting the target voice signal according to the voice frame length to obtain a plurality of first voice signals; converting the first voice signal from a time domain signal to a frequency domain signal, and determining a target feature of each frame of the first voice signal, wherein the target feature comprises energy and a zero-crossing rate; if the energy of the first voice signal meets a preset energy threshold and the zero-crossing rate meets a preset zero-crossing rate threshold, determining that the first voice signal is an effective voice signal.

3. The method according to claim 2, wherein the preset energy threshold is determined according to an average energy of all the first voice signals, and the zero-crossing rate threshold is determined according to an average zero-crossing rate of all the first voice signals.

4. The method according to claim 1, wherein the historical behavior data comprises at least one of the following: product ordering information, historical transaction information, historical call records, product preferences and basic portrait information of the user.

5. The method of user sentiment perception according to claim 1, wherein, The method further comprises the following steps: obtaining an acoustic feature and a semantic feature in the effective voice signal, and splicing the acoustic feature and the semantic feature in a time step alignment manner to obtain a plurality of first feature vectors, and splicing the plurality of first feature vectors to obtain a first feature sequence; inputting the first feature sequence into a perception prediction model, wherein the perception prediction model outputs a time sequence feature according to the first feature sequence, and the time sequence feature comprises a hidden state of each time step in the first feature sequence, and a second feature vector is obtained according to the time sequence feature and the historical behavior data of the target user; inputting the second feature vector into an emotion classification model, wherein the emotion classification model classifies the obtained second feature vector according to an emotion tendency, and outputs an emotion prediction vector; obtaining an input parameter of a multi-layer perception machine according to the emotion prediction vector; inputting the input parameter into the multi-layer perception machine, and the multi-layer perception machine outputs the target emotion label.

6. The method of user sentiment perception according to claim 5, wherein, The method further comprises the following steps: determining a similarity between each first feature vector and all elements of the first feature sequence, and determining an attention weight according to the similarity, and obtaining a weighted context vector according to the attention weight and the corresponding first feature vector. According to the weighted context vector and the sentiment pre-judgment vector, an input parameter is obtained.

7. A device for sensing user emotions, characterized in that, The method comprises the following steps: A first obtaining module is configured to obtain a target voice signal of a target user and historical behavior data of the target user. A first processing module is configured to obtain a target feature in the target voice signal and cut the target voice signal according to the target feature to obtain a plurality of effective voice signals. A second processing module is configured to determine a target emotion label of the target user according to the plurality of effective voice signals and the historical behavior data of the target user.

8. An electronic device, comprising: The computer program product comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps in the method for user emotion perception according to any one of claims 1 to 6.

9. A readable storage medium, characterized by, The readable storage medium stores a program or instruction, and the program or instruction is executed by the processor to implement the steps in the method for user emotion perception according to any one of claims 1 to 6.

10. A computer program product, characterised in that, The computer program product comprises computer instructions, and the computer instructions are executed by the processor to implement the steps in the method for user emotion perception according to any one of claims 1 to 6.

Images