Business processing method, business processing device, electronic equipment and storage medium

Abstract

The invention provides a business processing method and device based on emotion analysis, electronic equipment and a storage medium, and relates to the technical field of emotion recognition. The service processing method based on sentiment analysis comprises the following steps: preprocessing voice data subjected to sentiment type marking to obtain preprocessed data; performing time domain and frequency domain feature extraction on the preprocessed data to obtain original time domain emotion feature data and original frequency domain emotion feature data; performing secondary fitting multi-model training based on the original time domain emotion feature data and the original frequency domain emotion feature data to obtain a plurality of sub-models; splicing the plurality of sub-models to obtain an emotion classification model; and in response to the service mode switching instruction, calling the sentiment classification model to perform sentiment classification on the to-be-recognized user voice to obtain a sentiment analysis result. According to the technical scheme, the influence of the features on the time domain and the frequency domain on sentiment analysis can be considered, and the recognition effect of the sentiment classification model is improved.

Description

Background Art

[0002] In the customer service system, an intelligent emotion robot is used to perform emotion recognition on the acquired user voice, and the emotion classification model is used to implement the emotion classification. The emotion classification model is obtained based on the deep learning model training. However, due to the problems of high noise in the training data and single feature field of the training data in the current model training, the classification model fitting effect is not high, that is, the effect of emotion recognition is not improved, which makes the intelligent emotion robot's emotion judgment vague.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the present disclosure is to provide a business processing method based on sentiment analysis, a business processing device based on sentiment analysis, an electronic device and a computer-readable storage medium, which can at least to some extent improve the problem of fuzzy emotional judgment of intelligent emotional robots in related technologies and the difficulty in making effective measures for classification.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, a business processing method based on sentiment analysis is provided, including: preprocessing speech data after sentiment type labeling to obtain preprocessed data; performing time domain and frequency domain feature extraction on the preprocessed data to obtain original time domain sentiment feature data and original frequency domain sentiment feature data; performing quadratic fitting multi-model training based on the original time domain sentiment feature data and the original frequency domain sentiment feature data to obtain multiple sub-models; splicing the multiple sub-models to obtain a sentiment classification model; in response to a service mode switching instruction, calling the sentiment classification model to perform sentiment classification on the user speech to be recognized to obtain a sentiment analysis result.

[0007] In one embodiment, the multi-model training based on quadratic fitting of the original time-domain emotion feature data and the original frequency-domain emotion feature data to obtain multiple sub-models includes: inputting the original time-domain emotion feature data and the original frequency-domain emotion feature data as training data into the residual neural network model for deep learning of initial fitting, so as to obtain a learning network model based on the learning result as an initial fitting model; obtaining a fusion model, so as to fuse the emotion time-domain features and emotion frequency-domain features output by the learning network model based on the fusion model to obtain a fusion feature; performing quadratic fitting model training on the radial basis function neural network based on the fusion feature to obtain a quadratic fitting model, so as to splice the initial fitting model, the fusion model and the quadratic fitting model as the multiple sub-models to obtain the emotion classification model;

[0008] In one embodiment, the speech data after emotion type labeling is preprocessed to obtain preprocessed data, including: when the speech data is acquired, emotion type labeling is performed on the speech data based on the analyzed emotion type to obtain labeled speech data; primary noise reduction is performed on the labeled speech data based on a Gaussian sliding window to obtain primary processed data; data segmentation is performed on the primary processed data to obtain multiple groups of segmented data; secondary noise reduction is performed on each group of segmented data based on Fourier transform to obtain the preprocessed data.

[0009] In one embodiment, the performing of a secondary denoising based on Fourier transform on each group of the segmented data to obtain the preprocessed data includes: performing a fast Fourier transform on each group of the segmented data to obtain a Fourier transform value; accumulating and summing the Fourier transform values ​​of each group to obtain a Fourier transform sum; calculating an average value of the Fourier transform sum, and performing an inverse Fourier transform based on the average value to obtain the preprocessed data to complete the secondary denoising.

[0010] In one embodiment, the feature extraction of the preprocessed data in the time domain and the frequency domain to obtain original time domain emotion feature data and original frequency domain emotion feature data includes: extracting the original real part and the original imaginary part of the preprocessed data to obtain the original time domain emotion feature data; performing fast Fourier transform on the preprocessed data to obtain frequency domain data; extracting the transformed real part, the transformed imaginary part and the signal energy data of the frequency domain data to obtain the original frequency domain emotion feature data.

[0011] In one embodiment, the original time-domain emotion feature data and the original frequency-domain emotion feature data are respectively input as training data into the residual neural network model for initial fitting deep learning to obtain a learning network model based on the learning results as an initial fitting model, including: inputting the original time-domain emotion feature data into the first residual neural network for model iterative training, performing initial fitting based on the fully connected layer of the first residual neural network, until the model converges based on the emotion type labeling detection, to obtain a first learning network model; inputting the original frequency-domain emotion feature data into the second residual neural network for model iterative training, performing initial fitting based on the fully connected layer of the second residual neural network, until the model converges based on the emotion type labeling detection, to obtain a second learning network model, wherein the first residual neural network and the second residual neural network are both multi-layer neural networks.

[0012] In one embodiment, the fusion model is obtained to fuse the emotion time domain features and emotion frequency domain features output by the learning network model based on the fusion model to obtain the fusion features, including: inputting the original time domain emotion feature data into the first learning network model to output the emotion time domain features; inputting the original frequency domain emotion feature data into the second learning network model to output the emotion frequency domain features; and fusing the emotion time domain features and the emotion frequency domain features based on the fusion model to obtain the fusion features.

[0013] In one embodiment, the fusion model includes a discriminant correlation analysis model, and the emotion time domain features and the emotion frequency domain features are fused based on the fusion model to obtain the fused features, including: determining the time domain vector representation of the emotion time domain features and the frequency domain vector representation of the emotion frequency domain features; calculating the diagonal matrix of the time domain vector representation and the frequency domain vector representation based on the discriminant correlation analysis model; calculating the maximum significant feature based on the diagonal matrix; determining the eigenvector projection of the maximum significant feature matrix; and diagonalizing the eigenvector projection based on singular value decomposition to obtain the fused features.

[0014] In one embodiment, the model training of performing quadratic fitting on the radial basis function neural network based on the fusion features to obtain the quadratic fitting model includes: inputting the fusion features into the radial basis function neural network to perform quadratic fitting; and generating the quadratic fitting model when it is detected that the radial basis function neural network converges to meet the convergence condition.

[0015] In one embodiment, calling the emotion classification model to perform emotion classification on the user voice to be recognized and obtaining the emotion analysis result includes: decomposing the user voice in the time domain and frequency domain to obtain speech time domain data and voice frequency domain data; inputting the speech time domain data into the first initial fitting model to output emotion time domain feature data; inputting the voice frequency domain data into the second initial fitting model to output emotion frequency domain feature data; based on the fusion model, fusing the emotion time domain feature data and the emotion frequency domain feature data to obtain fused feature data; inputting the fused feature data into the secondary fitting model to output the emotion analysis result.

[0016] According to another aspect of the present disclosure, a business processing device based on sentiment analysis is provided, including: a preprocessing module, used to preprocess speech data after sentiment type labeling, to obtain preprocessed data; a feature extraction module, used to perform time domain and frequency domain feature extraction on the preprocessed data, to obtain original time domain sentiment feature data and original frequency domain sentiment feature data; a model training module, used to perform quadratic fitting multi-model training based on the original time domain sentiment feature data and the original frequency domain sentiment feature data, to obtain multiple sub-models; a splicing module, used to splice the multiple sub-models, to obtain a sentiment classification model; a sentiment analysis module, used to call the sentiment classification model to perform sentiment classification on the user speech to be recognized in response to a service mode switching instruction, to obtain a sentiment analysis result.

[0017] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the business processing method based on sentiment analysis described in any one of the first aspect above, or the business processing method based on sentiment analysis described in any one of the second aspect above, by executing the executable instructions.

[0018] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned business processing methods based on sentiment analysis is implemented.

[0019] The business processing solution based on sentiment analysis provided by the embodiments of the present disclosure obtains original time-domain sentiment feature data and original frequency-domain sentiment feature data by performing feature extraction in both the time domain and the frequency domain, and uses the sentiment type labels in the feature data as model training samples for multi-model training through quadratic fitting to obtain multiple sub-models, and then concatenates the multiple sub-models to obtain a sentiment classification model with better effect. Therefore, when the user voice to be recognized is obtained, the quadratic fitting function based on the sentiment classification model can take into account the influence of the features in the time domain and the frequency domain on the sentiment analysis, thereby improving the recognition effect of the sentiment classification model.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0022] Figure 1 A schematic diagram showing a statistical system structure of a service commodity in an embodiment of the present disclosure;

[0023] Figure 2 A flowchart of a business processing method based on sentiment analysis in an embodiment of the present disclosure is shown;

[0024] Figure 3 A schematic diagram showing a business processing solution based on sentiment analysis in an embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of a sub-model of a sentiment classification model based on sentiment analysis in an embodiment of the present disclosure is shown;

[0026] Figure 5 A schematic diagram showing another business processing solution based on sentiment analysis in an embodiment of the present disclosure is shown;

[0027] Figure 6 A schematic diagram showing a sentiment classification model based on sentiment analysis in an embodiment of the present disclosure is shown;

[0028] Figure 7 A schematic diagram showing another business processing solution based on sentiment analysis in an embodiment of the present disclosure;

[0029] Figure 8 A schematic diagram showing another business processing method based on sentiment analysis in an embodiment of the present disclosure;

[0030] Fig. 9 A schematic diagram showing another business processing method based on sentiment analysis in an embodiment of the present disclosure;

[0031] Fig.10 A schematic diagram showing a service processing device based on sentiment analysis in an embodiment of the present disclosure is shown;

[0032] Fig.11 A schematic diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0033] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0034] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0035] There are a large number of recordings in the current customer service system, but these recordings are currently only used as an archive, and this part of the data is not fully utilized. These recordings run through the entire life cycle of the customer, including pre-sales, sales and after-sales, covering consultation, installation, claims and other scenarios, and comprehensively and completely reflect the customer’s real demands for shops, logistics, goods, etc. Therefore, sentiment analysis and keyword extraction play a vital role in the training of intelligent emotional robots, and have an effective risk hedging effect on the situation of insufficient customer service manpower in activities such as big promotions in the e-commerce industry.

[0036] Specifically, since the speech data contains multiple dialects, the same word can be expressed in a variety of ways, and intonation, pauses, and polysemy are all factors that make it difficult to select emotional features, so emotion recognition requires identifying these features.

[0037] In addition, since noise is an important part of data processing, there is a large amount of invalid data in voice data, including environmental sounds, conversation sounds, invalid sentences, etc., which all affect the classification of emotions. Therefore, cleaning the data in advance plays a vital role.

[0038] The above phenomena will lead to vague sentiment judgment and make it difficult to make effective classification measures.

[0039] The solution provided in the present application obtains original time-domain emotion feature data and original frequency-domain emotion feature data by performing feature extraction in both the time domain and the frequency domain, combines the emotion type labels in the feature data as model training samples, performs quadratic fitting multi-model training to obtain multiple sub-models, and concatenates the multiple sub-models to obtain an emotion classification model with better effect. Therefore, when the user voice to be recognized is obtained, the quadratic fitting function of the emotion classification model can take into account the influence of the features in the time domain and the frequency domain on the emotion analysis, thereby improving the recognition effect of the emotion classification model.

[0040] Figure 1 1 is a schematic diagram of a computer system provided by an exemplary embodiment of the present application. The system includes: a plurality of terminals 120 and a server cluster 140.

[0041] The terminal 120 may be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, or the terminal 120 may also be a personal computer (PC), such as a laptop computer and a desktop computer.

[0042] The terminal 120 may be installed with an application for providing statistics of service commodities.

[0043] The terminal 120 is connected to the server cluster 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0044] The server cluster 140 is a server, or is composed of several servers, or is a virtualization platform, or is a cloud computing service center. The server cluster 140 is used to provide background services for applications that provide statistics of service commodities. Optionally, the server cluster 140 undertakes the main computing work and the terminal 120 undertakes the secondary computing work; or, the server cluster 140 undertakes the secondary computing work and the terminal 120 undertakes the main computing work; or, the terminal 120 and the server cluster 140 adopt a distributed computing architecture for collaborative computing.

[0045] In some optional embodiments, the server cluster 140 is used to store statistical program information of service commodities.

[0046] Optionally, the client of the application installed in different terminals 120 is the same, or the client of the application installed on the two terminals 120 is the client of the same type of application of different control system platforms. Based on the different terminal platforms, the specific form of the client of the application may also be different, for example, the application client may be a mobile client, a PC client or a World Wide Web (Web) client.

[0047] Those skilled in the art will appreciate that the number of the terminals 120 may be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or a greater number. The embodiment of the present application does not limit the number and device type of the terminals.

[0048] Optionally, the system may also include a management device ( Figure 1 (not shown), the management device is connected to the server cluster 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0049] Optionally, the above-mentioned wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0050] Below, the business processing method based on sentiment analysis in this example implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0051] like Figure 2 As shown, according to an embodiment of the present disclosure, a business processing method based on sentiment analysis includes:

[0052] Step S202, pre-processing the speech data after the emotion type labeling to obtain pre-processed data.

[0053] Among them, the preprocessing includes but is not limited to noise reduction and segmentation processing of voice data.

[0054] Step S204, extracting features in the time domain and frequency domain from the preprocessed data to obtain original time domain emotion feature data and original frequency domain emotion feature data.

[0055] Among them, the time domain sentiment feature refers to the sentiment feature extracted in the time domain, and the frequency domain sentiment feature refers to the sentiment feature extracted in the frequency domain.

[0056] Step S206, performing multi-model training of quadratic fitting based on the original time-domain emotion feature data and the original frequency-domain emotion feature data to obtain multiple sub-models.

[0057] Among them, multiple sub-models are obtained through multi-model training, so as to realize the quadratic fitting operation in the process of emotion feature classification and recognition based on multiple sub-models, so as to improve the recognition accuracy of the obtained emotion classification model when performing emotion recognition on speech data.

[0058] Step S208, concatenating multiple sub-models to obtain a sentiment classification model.

[0059] Among them, multiple sub-models are spliced, that is, multiple sub-models are arranged in sequence, and the output data of the previous sub-model is used as the input data of the next sub-model until the last sub-model outputs the emotion recognition result.

[0060] Step S210, in response to the service mode switching instruction, calling the sentiment classification model to perform sentiment classification on the user voice to be recognized to obtain a sentiment analysis result.

[0061] In this embodiment, by performing feature extraction in both the time domain and the frequency domain, original time domain emotion feature data and original frequency domain emotion feature data are obtained, and the emotion type labels in the feature data are combined as model training samples for multi-model training through quadratic fitting to obtain multiple sub-models, and the multiple sub-models are spliced ​​to obtain an emotion classification model with better effect. Therefore, when the user voice to be recognized is obtained, the quadratic fitting function based on the emotion classification model can take into account the influence of the features in the time domain and the frequency domain on the emotion analysis respectively, thereby improving the recognition effect of the emotion classification model.

[0062] like Figure 3As shown, in one embodiment, a multi-model training of quadratic fitting is performed based on the original time-domain emotion feature data and the original frequency-domain emotion feature data to obtain multiple sub-models, including:

[0063] Step S302, the original time-domain emotion feature data and the original frequency-domain emotion feature data are respectively input as training data into the residual neural network model for initial fitting deep learning, so as to obtain a learning network model based on the learning results as an initial fitting model.

[0064] Among them, the residual neural network model, the residual neural network model ResNet is constructed by residual blocks (ResidualBuilding Block), which are composed of multiple cascaded convolutional layers and a shortcut connection. After accumulating the output values ​​of the two, the output of the residual block is obtained through the ReLU activation layer. Multiple residual blocks can be connected in series to achieve a deeper network. The last layer of ResNet is a fully connected layer. The initial fitting of the data is achieved based on the fully connected layer, so that the initial fitting model is obtained through model training as the first sub-model.

[0065] Step S304, obtaining a fusion model, and fusing the emotion time domain features and emotion frequency domain features output by the learning network model based on the fusion model to obtain fusion features.

[0066] Among them, the fusion model is the second sub-model.

[0067] Step S306, performing quadratic fitting model training on the radial basis function neural network based on the fusion features to obtain a quadratic fitting model.

[0068] Among them, the quadratic fitting model is the third sub-model.

[0069] Step S308, the initial fitting model, the fusion model and the secondary fitting model are spliced ​​as multiple sub-models to obtain a sentiment classification model.

[0070] In this embodiment, by configuring a fusion model in the emotion classification model, the time domain emotion features and the frequency domain emotion features are fused for identification during emotion classification to improve the recognition rate. Combined with the quadratic fitting operation, a high-precision classifier can be obtained.

[0071] In this embodiment, in order to address the defect that the traditional deep learning network classifier has a low fitting effect, a design scheme based on secondary feature fusion of signal time domain and frequency domain characteristics and a high-precision classifier will improve the fitting effect of the model, thereby improving the recognition performance and recognition rate.

[0072] In one embodiment, the speech data after the emotion type labeling is preprocessed to obtain preprocessed data, including:

[0073] When the voice data is acquired, the voice data is labeled with an emotion type based on the analyzed emotion type to obtain labeled voice data.

[0074] Among them, due to the different ways of expressing emotional attributes and emotional characteristics in different industries, and different definitions of emotions, in order to achieve accurate classification of speech emotions, it is necessary to collect signal data by ourselves. By setting up a data acquisition module, a signal collector with a rate of 1Gsps is used for data collection in the data acquisition module, that is, 1s can collect signal data with a length of 131072 points. In addition, the historical data obtained from logistics customer service has not been marked with good emotional labels. Therefore, there is a lack of good sample labels for model training. Therefore, it is necessary to label the collected speech data with emotional types in order to obtain the data required for subsequent use.

[0075] The marked speech data is subjected to initial noise reduction processing based on the Gaussian sliding window to obtain the initial processed data.

[0076] Among them, the initial noise reduction process for the marked speech data includes: performing signal windowing on the collected speech signal through a Gaussian filter to reduce the signal-to-noise ratio of the marked speech data and reduce the impact of noise.

[0077] The initially processed data is segmented to obtain multiple groups of segmented data.

[0078] Among them, data segmentation: According to the collected voice signal characteristics, the signal length of the voice signal is relatively long, so the data will be segmented by segmenting the voice signal. In order to adapt to the current deep learning model, in this application, the signal can be segmented into 128 segments, that is, the length of each data segment is 1024 points.

[0079] Each group of segmented data is subjected to secondary denoising based on Fourier transform to obtain preprocessed data.

[0080] Among them, for each group of segmented data obtained, Fourier transform is performed, and then accumulation and averaging calculation are performed, and finally inverse Fourier transform is performed, that is, secondary noise reduction is achieved, and low signal-to-noise ratio signal data after noise reduction is obtained.

[0081] In this embodiment, the noise reduction effect is improved by adopting a secondary noise reduction method of signal windowing and Fourier frequency domain cancellation to obtain high-quality training samples, thereby improving the effect of model training and the recognition rate of the network in the process of model training.

[0082] In one embodiment, each group of segmented data is subjected to secondary noise reduction based on Fourier transform to obtain preprocessed data, including:

[0083] Perform fast Fourier transform on each set of segmented data to obtain Fourier transform values.

[0084] The Fourier transform values ​​of each group are accumulated and summed to obtain the Fourier transform sum.

[0085] The average value of the Fourier transform and the inverse Fourier transform is calculated, and the preprocessed data is obtained based on the average value to complete the secondary noise reduction.

[0086] Exemplarily, a fast Fourier transform (FFT) is performed on each group of segmented data to obtain a Fourier transform value, multiple groups of Fourier transform values ​​are accumulated and summed to obtain the sum of the Fourier transforms, the sum of the Fourier transforms is averaged, and an inverse Fourier transform is performed to obtain secondary denoised data.

[0087] In one embodiment, the preprocessed data is subjected to feature extraction in the time domain and the frequency domain to obtain original time domain emotion feature data and original frequency domain emotion feature data, including:

[0088] The original real part and the original imaginary part of the preprocessed data are extracted to obtain the original time-domain emotional feature data.

[0089] Perform fast Fourier transform on the preprocessed data to obtain frequency domain data.

[0090] The transformed real part, transformed imaginary part and signal energy data of the frequency domain data are extracted to obtain the original frequency domain emotion feature data.

[0091] Among them, feature extraction is performed on the preprocessed data, and the extracted features include: the original real and imaginary data of the signal, the real and imaginary data after Fourier transform, and the energy data of the signal, all of which are 1024 points. The data are organized into original two-dimensional time domain data (the original real and imaginary data of the signal) and three-dimensional frequency domain data (the real and imaginary data after Fourier transform, and the energy data of the signal).

[0092] In this embodiment, by performing a fast Fourier transform operation on the preprocessed data, the original time domain emotion feature data and the original frequency domain emotion feature data can be obtained respectively, and the original time domain emotion feature data and the original frequency domain emotion feature data can be applied to the model training process respectively, thereby improving the classification effect of the obtained classification model.

[0093] In one embodiment, the original time-domain emotion feature data and the original frequency-domain emotion feature data are respectively input as training data into the residual neural network model for deep learning of initial fitting, so as to obtain a learning network model based on the learning results as an initial fitting model, including:

[0094] The original time-domain emotional feature data is input into the first residual neural network for model iterative training, and the first fitting is performed based on the fully connected layer of the first residual neural network until the model converges based on the emotional type labeling detection, thereby obtaining the first learning network model.

[0095] The original frequency domain emotion feature data is input into the second residual neural network for iterative model training, and the first fitting is performed based on the fully connected layer of the second residual neural network until the model converges based on the emotion type labeling detection to obtain the second learning network model.

[0096] Wherein, the first residual neural network and the second residual neural network are both multi-layer neural networks.

[0097] like Figure 4 As shown, exemplarily, the real data and imaginary data of the original time-domain sentiment feature data are input into a five-layer ResNet network to perform feature extraction on the original time-domain sentiment features, that is, feature extraction is performed based on the real and imaginary data of the original time-domain sentiment features, and model training is performed by extracting useful features from multiple convolutional layers until the network converges based on the sentiment label. After the network converges, the output of the last convolutional layer is used as the sentiment time-domain feature.

[0098] The real and imaginary data of the original frequency domain sentiment feature data are input into the five-layer ResNet network to extract the original frequency domain sentiment features, that is, feature extraction is performed based on the real and imaginary data of the original frequency domain sentiment features. The model is trained by extracting useful features from multiple convolutional layers until the network converges based on the sentiment label. After the network converges, the output of the last convolutional layer is used as the sentiment frequency domain feature.

[0099] In this embodiment, the first sub-model is obtained by using a residual neural network for model training. In a deep residual network, through the short-circuited paths on the residual network, the gradient can be propagated coherently in a very deep network without being superimposed by too many convolutional layer gradients, thereby ensuring the training effect of the deep neural network.

[0100] In one embodiment, a fusion model is obtained to fuse the emotion time domain features and emotion frequency domain features output by the learning network model based on the fusion model to obtain fusion features, including:

[0101] The original time-domain emotional feature data is input into the first learning network model, and the emotional time-domain features are output.

[0102] The original frequency domain emotion feature data is input into the second learning network model, and the emotion frequency domain features are output.

[0103] Based on the fusion model, the emotional time domain features and the emotional frequency domain features are fused to obtain the fusion features.

[0104] like Figure 5 As shown, in one embodiment, the fusion model includes a discriminant correlation analysis model, and the emotion time domain features and the emotion frequency domain features are fused based on the fusion model to obtain fusion features, including:

[0105] Step S502, determining the time domain vector representation of the emotion time domain feature and the frequency domain vector representation of the emotion frequency domain feature.

[0106] Step S504: Calculate the diagonal matrix of the time domain vector representation and the frequency domain vector representation based on the discriminant correlation analysis model.

[0107] Step S506, calculating the maximum significant feature based on the diagonal matrix.

[0108] Step S508, determining the eigenvector projection of the maximum significant feature matrix.

[0109] Step S510, performing diagonal processing on the feature vector projection based on singular value decomposition to obtain fusion features.

[0110] In this embodiment, the deep learning features extracted by the two networks in step 3 are fused, and the feature fusion algorithm is the DCA algorithm. The features extracted in step 3 are fused by the signal time domain and the emotion frequency domain features to obtain the fused features. The steps are as follows:

[0111] Step 1: Obtain the diagonal matrix of the two characteristic column vectors;

[0112] Step 2: Obtain the maximum significant feature through the diagonal matrix;

[0113] Step 3: Obtain the eigenvector projection of the maximum significant feature matrix;

[0114] Step 4: Use singular value decomposition to diagonalize the projection obtained in step 3 to obtain the fused features.

[0115] In one embodiment, a quadratic fitting model is trained on a radial basis function neural network based on fusion features to obtain a quadratic fitting model, including:

[0116] The fused features are input into the radial basis function neural network for quadratic fitting.

[0117] When it is detected that the radial basis function neural network converges to meet the convergence condition, a quadratic fitting model is generated.

[0118] In this embodiment, the fusion feature uses a RBF neural network with faster fitting speed and better effect for secondary fitting, and the steps of the secondary fitting are:

[0119] The fusion features are input as training samples into the RBF neural network for model training. After the model training reaches network convergence, the model parameters of the network are saved to obtain the third sub-model, namely the quadratic fitting model.

[0120] Furthermore, the trained RBF neural network is spliced ​​with the ResNet model and the fusion model to obtain the entire sentiment classification model.

[0121] like Figure 6 As shown, Figure 4 As shown, exemplarily, the real data and imaginary data of the original time-domain sentiment feature data are input into a five-layer ResNet network to perform feature extraction on the original time-domain sentiment features, that is, feature extraction is performed based on the real and imaginary data of the original time-domain sentiment features, and model training is performed by extracting useful features from multiple convolutional layers until the network converges based on the sentiment label. After the network converges, the output of the last convolutional layer is used as the sentiment time-domain feature.

[0122] The real and imaginary data of the original frequency domain sentiment feature data are input into the five-layer ResNet network to extract the original frequency domain sentiment features, that is, feature extraction is performed based on the real and imaginary data of the original frequency domain sentiment features. The model is trained by extracting useful features from multiple convolutional layers until the network converges based on the sentiment label. After the network converges, the output of the last convolutional layer is used as the sentiment frequency domain feature.

[0123] The emotional time domain features and emotional frequency domain features are input into the fusion model, the fusion features are output, and the RBF neural network model is trained based on the fusion features, and finally the emotional classification model is obtained to produce the classification results.

[0124] like Figure 7 As shown, in one embodiment, the sentiment classification model is called to perform sentiment classification on the user speech to be recognized, and the sentiment analysis result is obtained, including:

[0125] Step S702: Decompose the user's voice in the time domain and the frequency domain to obtain voice time domain data and voice frequency domain data.

[0126] Step S704: input the speech time domain data into the first initial fitting model, and output the emotion time domain feature data.

[0127] Step S706: input the speech frequency domain data into the second initial fitting model, and output the emotion frequency domain feature data.

[0128] Step S708: fusing the emotion time domain feature data and the emotion frequency domain feature data based on the fusion model to obtain fused feature data.

[0129] Step S710, input the fused feature data into the quadratic fitting model and output the sentiment analysis result.

[0130] like Figure 8 As shown, a system for generating a sentiment classification model according to an embodiment of the present disclosure includes:

[0131] The voice collection module 802 is used to collect voice data.

[0132] The data preprocessing module 804 is used to preprocess the speech data.

[0133] The pre-emphasis module 806 is used to enhance the high-frequency components of the characteristic data.

[0134] The frame dividing and windowing module 808 is used to perform frame dividing and windowing on the feature data.

[0135] The feature extraction and emotion recognition module 810 is used to perform feature extraction and emotion recognition on the preprocessed features.

[0136] The result judgment module 812 is used to judge the training result based on the emotion label.

[0137] The model optimization module 814 is used to optimize the model based on the judgment result and finally obtain the sentiment classification model.

[0138] like Fig. 9 As shown in the figure, in the application scenario, the customer service system implements model training and model application through interaction with customers. The interaction process specifically includes:

[0139] Step S902: In the customer service system, voice data is retained based on customer service communication.

[0140] Step S904: pre-process the stored voice data.

[0141] Step S906: perform feature calculation and model training based on the preprocessed data to obtain a sentiment classification model.

[0142] Step S908: Apply the emotion classification model to the intelligent outbound call to receive the user's voice.

[0143] Step S910: predicting the emotion of the user's voice based on the emotion classification model.

[0144] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0145] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0146] Refer to the following Fig.10 To describe the sentiment analysis-based business processing device 1000 according to this embodiment of the present disclosure. Fig.10 The sentiment analysis-based business processing device 1000 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0147] The service processing device 1000 based on sentiment analysis is presented in the form of a hardware module. The components of the service processing device 1000 based on sentiment analysis may include but are not limited to: a preprocessing module 1002, which is used to preprocess the speech data after sentiment type labeling to obtain preprocessed data; a feature extraction module 1004, which is used to extract time domain and frequency domain features of the preprocessed data to obtain original time domain sentiment feature data and original frequency domain sentiment feature data; a model training module 1006, which is used to perform multi-model training based on the original time domain sentiment feature data and the original frequency domain sentiment feature data for quadratic fitting to obtain multiple sub-models; a splicing module 1008, which is used to splice the multiple sub-models to obtain a sentiment classification model; a sentiment analysis module 1010, which is used to respond to a service mode switching instruction, call the sentiment classification model to perform sentiment classification on the user speech to be recognized, and obtain a sentiment analysis result.

[0148] Refer to the following Fig.11 1100 according to this embodiment of the present disclosure is described. Fig.11 The electronic device 1100 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0149] like Fig.11 As shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 may include but are not limited to: at least one processing unit 1110, at least one storage unit 1120, and a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110).

[0150] The storage unit stores program codes, which can be executed by the processing unit 1110, so that the processing unit 1110 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 1110 can execute the following steps: Figure 1 Steps S202 and S210 shown in , and other steps defined in the business processing method based on sentiment analysis of the present disclosure.

[0151] The storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 11201 and / or a cache storage unit 11202 , and may further include a read-only storage unit (ROM) 11203 .

[0152] The storage unit 1120 may also include a program / utility 11204 having a set (at least one) of program modules 11205, such program modules 11205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0153] Bus 1130 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0154] The electronic device 1100 may also communicate with one or more external devices 1160 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1150. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1150. As shown, the network adapter 1150 communicates with other modules of the electronic device 1100 via a bus 1130. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0155] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0156] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0157] According to the program product for implementing the above method in the embodiment of the present disclosure, it can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0158] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0159] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0160] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0161] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of a module or unit described above can be further divided into multiple modules or units to be embodied. In addition, although the various steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution, etc. Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions so that a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) executes the method according to the embodiment of the present disclosure. Those skilled in the art will easily think of other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any modification, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are indicated by the attached claims.

Claims

1. A business processing method based on sentiment analysis, characterized in that: include: Preprocessing the speech data after the emotion type labeling to obtain preprocessed data; Extracting features in the time domain and the frequency domain from the preprocessed data to obtain original time domain emotion feature data and original frequency domain emotion feature data; Performing multi-model training of quadratic fitting based on the original time-domain emotion feature data and the original frequency-domain emotion feature data to obtain multiple sub-models; The multiple sub-models are spliced ​​to obtain a sentiment classification model; In response to the service mode switching instruction, the emotion classification model is called to perform emotion classification on the user voice to be recognized to obtain the emotion analysis result.

2. The business processing method based on sentiment analysis according to claim 1, characterized in that: Based on the original time-domain emotion feature data and the original frequency-domain emotion feature data, a multi-model training of quadratic fitting is performed to obtain multiple sub-models, including: Inputting the original time-domain emotion feature data and the original frequency-domain emotion feature data into the residual neural network model as training data for initial fitting deep learning, so as to obtain a learning network model based on the learning results as an initial fitting model; Acquire a fusion model to fuse the emotion time domain features and emotion frequency domain features output by the learning network model based on the fusion model to obtain fusion features; Based on the fusion features, a quadratic fitting model is trained on the radial basis function neural network to obtain a quadratic fitting model, and the initial fitting model, the fusion model and the quadratic fitting model are spliced ​​as the multiple sub-models to obtain the sentiment classification model.

3. The business processing method based on sentiment analysis according to claim 2 is characterized in that: The speech data after emotion type labeling is preprocessed to obtain preprocessed data, including: When the voice data is acquired, the voice data is marked with an emotion type based on the analyzed emotion type to obtain marked voice data; Performing initial noise reduction processing on the marked speech data based on a Gaussian sliding window to obtain initial processed data; Performing data segmentation on the initially processed data to obtain multiple groups of segmented data; Performing secondary noise reduction based on Fourier transform on each group of the segmented data to obtain the preprocessed data.

4. The business processing method based on sentiment analysis according to claim 3 is characterized in that: Performing secondary noise reduction based on Fourier transform on each group of the segmented data to obtain the preprocessed data, including: Performing a fast Fourier transform on each group of segmented data to obtain a Fourier transform value; Accumulating and summing the Fourier transform values ​​of each group to obtain a Fourier transform sum; An average value of the Fourier transform sum is calculated, and an inverse Fourier transform is performed based on the average value to obtain the preprocessed data to complete the secondary noise reduction.

5. The business processing method based on sentiment analysis according to claim 4 is characterized in that: The preprocessed data is subjected to feature extraction in the time domain and the frequency domain to obtain original time domain emotion feature data and original frequency domain emotion feature data, including: Extracting the original real part and the original imaginary part of the preprocessed data to obtain the original time-domain emotional feature data; Performing a fast Fourier transform on the preprocessed data to obtain frequency domain data; The transformed real part, the transformed imaginary part and the signal energy data of the frequency domain data are extracted to obtain the original frequency domain emotion feature data.

6. The business processing method based on sentiment analysis according to claim 5 is characterized in that: The original time-domain emotion feature data and the original frequency-domain emotion feature data are respectively input as training data into the residual neural network model for deep learning of initial fitting, so as to obtain a learning network model based on the learning results as an initial fitting model, including: Inputting the original time-domain emotion feature data into a first residual neural network for model iterative training, performing initial fitting based on a fully connected layer of the first residual neural network, until the model converges based on the emotion type labeling detection, and obtaining a first learning network model; The original frequency domain emotion feature data is input into the second residual neural network for model iterative training, and the first fitting is performed based on the fully connected layer of the second residual neural network, until the model converges based on the emotion type labeling detection, and the second learning network model is obtained, and the first learning network and the second learning network are used as the initial fitting model. Wherein, the first residual neural network and the second residual neural network are both multi-layer neural networks.

7. The business processing method based on sentiment analysis according to claim 6 is characterized in that: Acquiring a fusion model to fuse the emotion time domain features and emotion frequency domain features output by the learning network model based on the fusion model to obtain fusion features, including: Inputting the original time-domain emotional feature data into the first learning network model, and outputting emotional time-domain features; Inputting the original frequency domain emotion feature data into the second learning network model, and outputting emotion frequency domain features; The emotion time domain feature and the emotion frequency domain feature are fused based on the fusion model to obtain the fusion feature.

8. The business processing method based on sentiment analysis according to claim 7 is characterized in that: The fusion model includes a discriminant correlation analysis model, and the emotion time domain feature and the emotion frequency domain feature are fused based on the fusion model to obtain the fusion feature, including: Determine a time domain vector representation of the emotion time domain feature and a frequency domain vector representation of the emotion frequency domain feature; Calculate the diagonal matrix of the time domain vector representation and the frequency domain vector representation based on the discriminant correlation analysis model; Calculating a maximum salient feature based on the diagonal matrix; Determining an eigenvector projection of the maximum significant feature matrix; The feature vector projection is diagonalized based on singular value decomposition to obtain the fusion feature.

9. The business processing method based on sentiment analysis according to claim 2, characterized in that: Based on the fusion features, a quadratic fitting model is trained on the radial basis function neural network to obtain a quadratic fitting model, including: Inputting the fused features into the radial basis function neural network to perform quadratic fitting; When it is detected that the radial basis function neural network converges to meet the convergence condition, the quadratic fitting model is generated.

10. The business processing method based on sentiment analysis according to any one of claims 2 to 9, characterized in that: The sentiment classification model is called to perform sentiment classification on the user speech to be recognized, and a sentiment analysis result is obtained, including: Decomposing the user's voice in the time domain and the frequency domain to obtain voice time domain data and voice frequency domain data; Input the speech time domain data into a first initial fitting model, and output emotional time domain feature data; Inputting the speech frequency domain data into a second preliminary fitting model, and outputting emotion frequency domain feature data; Based on the fusion model, the emotion time domain feature data and the emotion frequency domain feature data are fused to obtain fused feature data; The fused feature data is input into a quadratic fitting model, and the sentiment analysis result is output.

11. A business processing device based on sentiment analysis, characterized in that: include: A preprocessing module is used to preprocess the speech data after the emotion type is marked to obtain preprocessed data; A feature extraction module is used to extract features in the time domain and the frequency domain of the preprocessed data to obtain original time domain emotion feature data and original frequency domain emotion feature data; A model training module, used for performing multi-model training of quadratic fitting based on the original time-domain emotion feature data and the original frequency-domain emotion feature data to obtain multiple sub-models; A splicing module, used for splicing the multiple sub-models to obtain a sentiment classification model; The sentiment analysis module is used to respond to the service mode switching instruction, call the sentiment classification model to perform sentiment classification on the user voice to be recognized, and obtain the sentiment analysis result.

12. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the sentiment analysis-based business processing method described in any one of claims 1 to 10 by executing the executable instructions.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the business processing method based on sentiment analysis described in any one of claims 1 to 10 is implemented.

Images