Natural language interaction intelligent customer service system

Abstract

The application discloses a natural language interaction intelligent customer service system and relates to the technical field of intelligent interaction; the system comprises a specification module, which is used for collecting original data from a database of an intelligent customer service system and performing text normalization processing on the original data to obtain standard data, wherein the original data comprises text data, voice data and image data; and a word vector module, which is used for converting the standard data into word vectors through a word embedding method; the technical key points are as follows: the relationship of a sentence can be classified more accurately according to the meaning of the sentence, a probability distribution is output by using a Softmax function, the final classification result is given, and the probability of each category is provided, so that the prediction result of the model is more interpretable and reliable, a cross-entropy loss function and the Softmax are combined to provide a very clear and efficient optimization direction for the model, the logarithmic probability of a correct category can be directly maximized, the gradient is clear, and the convergence speed is fast.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of intelligent interaction, and particularly relates to a natural language interaction intelligent customer service system. BACKGROUND

[0002] The natural language interaction intelligent customer service system is a computer system using artificial intelligence technology to simulate human customer service, and performing multi-round dialogue with users through natural language to automatically answer questions and provide services. The principle is to use natural language processing technology to perform word segmentation and syntax analysis on the text or voice input by the user, understand the real intention and key information behind the text or voice, find the most relevant answer in a knowledge base, such as a product manual, a frequently asked question (FAQ) and a database according to the recognized intention, and reply to the user with the found answer or generated text in clear and natural language to complete an interaction. However, in the prior art, the self-attention mechanism is often connected with two layers of forward full connection layers after obtaining multiple spatial matrices. The structure can be regarded as a self-attention mechanism for parameters. However, the structure adds a large number of training parameters to the model, so that the model is difficult to converge, thereby causing the model to fail to achieve an ideal effect. In addition, the problem part is generally directly subjected to average pooling to obtain a vector representation of the problem in the processing process. However, there is an inseparable relationship between the problem and the answer. The determination of the answer cannot be made without considering the similarity with the problem. In order to find an answer closely related to the problem. SUMMARY

[0003] To achieve the above object, the application is implemented by the following technical scheme:

[0004] The natural language interaction intelligent customer service system comprises:

[0005] A specification module is configured to collect original data from a database of the intelligent customer service system, and perform text specification processing on the original data to obtain standard data. The original data comprises text data, voice data and image data.

[0006] A word vector module is configured to convert the standard data into word vectors by using a word embedding method, integrate the word vectors into a meaning matrix based on the original data, and perform residual convolution operation on the meaning matrix to obtain a residual convolution matrix.

[0007] A multi-head attention module is configured to construct a multi-head attention model, and calculate a multi-head output matrix from the residual convolution matrix by using the multi-head attention model.

[0008] A question and answer detection module is configured to construct a question and answer self-attention model based on the multi-head output matrix, and perform similarity difference detection on a question matrix and an answer matrix in the multi-head output matrix by using the question and answer self-attention model, and obtain a correct answer set according to a detection result.

[0009] The emotion recognition module is configured to perform multiple pooling operations on the question matrix, generate feature extraction parameters by using a deep convolutional neural network method, extract features from the question matrix according to the feature extraction parameters, and determine the emotion of the question matrix and obtain an emotional question matrix by using a full connection layer and a softmax function.

[0010] The emotional answer module is configured to take the correct answer set as input, perform multiple pooling operations, full connection layer and softmax function to obtain an emotional answer set composed of multiple emotional answer matrices.

[0011] The language interaction module is configured to calculate the hybrid feature value of the emotional question matrix and the emotional answer set, obtain a fusion vector based on the hybrid feature value, input the fusion vector into a preliminarily constructed classification neural network model to obtain a category probability value, and obtain an optimized natural language interaction system based on a loss value S calculated based on the category probability value.

[0012] Further, the specification module comprises:

[0013] The deduplication module is configured to remove special characters in the text data by using a regular expression method, and quickly deduplicate the text data by using an embedding vector similarity to obtain basic text data.

[0014] The audio conversion module is configured to filter the audio in the original data by calculating the short-time energy to obtain filtered data, and convert the filtered data into audio text data by using an ASR speech recognition method.

[0015] The image conversion module is configured to eliminate blurred image data by using a sharpness detection method to obtain clear image data, and extract image text data from the clear image data by using an OCR technology.

[0016] The multi-modal module is configured to integrate the basic text data, the audio text data and the image text data into multi-modal data, align the multi-modal data on a unified time axis based on a timestamp method to obtain aligned multi-modal data, and fuse the aligned multi-modal data by using a feature fusion method to obtain fused data.

[0017] The standard module is configured to input the fused data into a T language model to correct errors to obtain corrected data, normalize the corrected data according to the standard expressions in the interaction database to obtain standardized data, and insert implicit context labels into the standardized data to obtain standard data.

[0018] Further, the word vector module comprises:

[0019] The conversion module is configured to first convert the standard data into a preliminary word vector by using a one-hot encoding method, and then convert the preliminary word vector into a word vector by using a word embedding method.

[0020] a significance matrix module configured to combine word vectors into a significance matrix according to the question and answer statements in the user interaction in the original data;

[0021] a deep convolution module configured to determine a filter function according to a dimension of the significance matrix, calculate a single deep convolution vector by multiplying the filter function with each vector in the significance matrix, and splice all single deep convolution vectors of the same significance matrix to obtain a deep convolution matrix;

[0022] a residual convolution module configured to sum the deep convolution matrix and the significance matrix to obtain a residual convolution matrix.

[0023] Further, the word vectors are used to make each word in the question and answer statements in the user interaction in the original data correspond to a low-dimensional vector.

[0024] Further, the multi-head attention module comprises:

[0025] a weight module configured to obtain a number M of heads of the multi-head attention model, determine M groups of weight transfer matrices for mapping the residual convolution matrix to three different spaces based on the number M of heads, wherein each group contains three weight transfer matrices corresponding to different spaces;

[0026] a space module configured to select a group of weight transfer matrices from the M groups of weight transfer matrices, and calculate a first space matrix, a second space matrix and a third space matrix by multiplying each weight transfer matrix in the selected group with the residual convolution matrix respectively;

[0027] a repeated operation module configured to repeatedly calculate the space matrices until all weight transfer matrices are multiplied with the residual convolution matrix to obtain a plurality of space matrices;

[0028] a splicing module configured to splice the plurality of space matrices to obtain a multi-head output matrix.

[0029] Further, the question and answer detection module comprises:

[0030] a division module configured to divide the multi-head output matrix into a question matrix and an answer matrix according to the question and answer statements in the user interaction, and extract column vectors of the question matrix and the answer matrix to form a sub-question vector and a sub-answer vector respectively;

[0031] a model construction module configured to construct a question and answer self-attention model and calculate a question transfer matrix of the question matrix and an answer transfer matrix of the answer matrix by a correlation similarity method respectively;

[0032] The attention weight module is configured to calculate attention weights according to the question transition matrix and the answer transition matrix, and the calculation method is to multiply the question transition matrix by the sub-question vector, add the answer transition matrix multiplied by the sub-answer vector to obtain a question and answer vector, calculate an arithmetic average matrix of the question transition matrix and the answer transition matrix, and obtain the question attention weight by using the question and answer vector dot product of the arithmetic average matrix.

[0033] The question proportion module is configured to determine an exponential function, calculate a question proportion value based on the exponential function, and obtain a correct answer set by using a similarity difference method according to the question proportion value.

[0034] Further, the question proportion module comprises:

[0035] The question and answer attention module is configured to multiply the question proportion value by each sub-answer vector and sum to obtain a question and answer attention value of the answer and the question.

[0036] The average pooling module is configured to perform average pooling on the question matrix to obtain a question attention value.

[0037] The similarity difference module is configured to calculate a question and answer similarity by using a cosine similarity method according to the question attention value and the question and answer attention value, and calculate a similarity difference by subtracting the question and answer similarity.

[0038] The threshold module is configured to set a corresponding detection threshold for the similarity difference.

[0039] The correct answer module is configured to form a correct question and answer group according to the detection threshold, and form a correct answer set according to the question and answer attention value.

[0040] Further, the emotion recognition module comprises:

[0041] The first pooling module is configured to perform first pooling on the question matrix through a pooling layer, and obtain a first convolutional block.

[0042] The second pooling module is configured to perform second pooling on the first convolutional block, and obtain a second convolutional block.

[0043] The third pooling module is configured to perform third pooling on the second convolutional block, and obtain a third convolutional block.

[0044] The fourth pooling module is configured to perform fourth pooling on the third convolutional block, and obtain a fourth convolutional block.

[0045] The parameter module is configured to perform pooling on the fourth convolutional block, and obtain a fifth convolutional block, and generate a trained feature extraction parameter by using a deep convolutional neural network method.

[0046] The feature module is configured to perform feature extraction on the question matrix through a fifth convolutional block and feature extraction parameters to obtain question features, and convert the question features into normalized probabilities through a fully connected layer and a softmax function.

[0047] The sentiment matrix module is configured to identify the sentiment of the question matrix according to the normalized probabilities and obtain a sentiment question matrix.

[0048] Further, the language interaction module comprises:

[0049] The sentence meaning encoding module is configured to form a to-be-encoded matrix by arranging the sentiment question matrix and the sentiment answer set in the order of the user-interacted question and answer sentences, and perform word sense encoding on the to-be-encoded matrix through an LSTM encoder to obtain a sentence meaning encoding matrix.

[0050] The mixed feature module is configured to extract the maximum value of the vectors in the sentence meaning encoding matrix as a first feature value, input the sentiment question matrix and the sentiment answer set into a multi-head attention model to obtain multi-dimensional feature values, and splice the first feature value and the multi-dimensional feature values to obtain mixed feature values.

[0051] The fusion module is configured to randomly select two of the mixed feature values as a first feature value and a second feature value, perform element-wise subtraction on the first feature value and the second feature value to obtain an absolute value, perform element-wise multiplication on the first feature value and the second feature value to obtain a multiplied value, and splice the first feature value, the second feature value, the absolute value and the multiplied value to obtain a fusion vector.

[0052] The category module is configured to construct a classification neural network model comprising a feedforward neural network layer, an output layer and a classification layer, input the fusion vector, and obtain a category probability value through the feedforward neural network layer, the output layer and the classification layer.

[0053] The loss module is configured to obtain an actual category z of the fusion vector, and calculate a loss value S for measuring the loss of the model.

[0054] The model optimization module is configured to set a threshold value of the loss value S, and if the loss value S is greater than or equal to the threshold value, optimize the model through a back propagation method and a stochastic gradient descent optimizer until the loss value S is less than the threshold value, and obtain a natural language interaction system.

[0055] The natural language interaction intelligent customer service system provided by the application has the following beneficial effects:

[0056] 1) This scheme improves the integrity and accuracy of context understanding through alignment and fusion technology, updates the word segmentation dictionary and entity library through online learning, adapts to the changes of new terms and user expression habits, avoids the problem of insufficient generalization ability caused by traditional static dictionary, dynamically corrects misspelled words based on dialogue history, supplements implicit information, and solves the problem of semantic deviation caused by traditional method only relying on single sentence processing;

[0057] 2) The multi-head attention model without residual structure can make the parameter size in the multi-head attention model no longer be limited by the input dimension size, increase the dimension to map the residual convolution matrix to a higher dimensional space, so that the model fits more complex function relationship and has stronger expressiveness, and from the overall point of view, the multi-head attention model also plays a role of connecting the past and the future, provides a premise for accurate feature extraction for subsequent processing steps, and improves the accuracy of natural language understanding;

[0058] 3) The parameter migration algorithm of the multi-pooling training model migrates the parameters trained by the deep convolutional neural network method, the general features learned by the shallow network are the general features of the image data, and the knowledge learned by the deeper network layer is more characteristic of the data set itself, so the parameters of the general features learned by the shallow network can be migrated and reused on the speech emotion data set;

[0059] 4) The hybrid pooling method of maximum pooling and multi-dimensional attention weighted average is used for the sentence meaning coding matrix to obtain the vector representation of the sentence, and finally the multi-layer forward neural network is used to obtain the final classification result, the experimental results show that compared with other natural language reasoning models based on deep learning, the model can more accurately classify the relationship of the sentence according to the sentence meaning, the Softmax function is used to output the probability distribution, not only the final classification result is given, but also the probability of each class is provided, so that the prediction result of the model is more interpretable and reliable, the cross-entropy loss function and Softmax are combined to provide a very clear and efficient optimization direction for the model, which can directly maximize the logarithmic probability of the correct class, the gradient is clear, and the convergence speed is fast. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 It is a structure block diagram of a natural language interaction intelligent customer service system in the application;

[0061] Figure 2 It is a structure block diagram of a specification module in the application;

[0062] Figure 3 It is a structure block diagram of a word vector module in the application;

[0063] Figure 4A structural block diagram of the multi-head attention module in the application;

[0064] Figure 5 A structural block diagram of the question and answer detection module in the application;

[0065] Figure 6 A structural block diagram of the question proportion module in the application;

[0066] Figure 7 A structural block diagram of the emotion recognition module in the application;

[0067] Figure 8 A structural block diagram of the language interaction module in the application.

[0068] In the figure: 1, specification module; 101, deduplication module; 102, audio conversion module; 103, image conversion module; 104, multi-modal module; 105, multi-modal fusion module; 106, standard module; 2, word vector module; 201, conversion module; 202, meaning matrix module; 203, deep convolution module; 204, residual convolution module; 3, multi-head attention module; 301, weight module; 302, space module; 303, repeated operation module; 304, splicing module; 4, question and answer detection module; 401, division module; 402, model construction module; 403, attention weight module; 404, question proportion module; 4041, question and answer attention module; 4042, average pooling module; 4043, similarity difference module; 4044, threshold module; 4045, correct answer module; 5, emotion recognition module; 501, primary pooling module; 502, secondary pooling module; 503, tertiary pooling module; 504, quaternary pooling module; 505, parameter module; 506, feature module; 507, emotion matrix module; 6, emotional answer module; 7, language interaction module; 701, sentence meaning encoding module; 702, hybrid feature module; 703, fusion module; 704, category module; 705, loss module; 706, model optimization module. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0070] Please refer to Figure 1 The embodiment provides a natural language interaction intelligent customer service system, which comprises:

[0071] The specification module 1 is used for collecting original data from the database of the intelligent customer service system, and performing text normalization processing on the original data to obtain standard data, wherein the original data includes text data, voice data and image data; the original data has not been processed and may contain special characters, advertising text, meaningless repeated sentences and other data, which seriously affects the processing efficiency of the language interaction system and increases the processing time of the data; in order to solve the above problems, the specific implementation steps are as follows:

[0072] The deduplication module 101 is used for removing special characters in the text data by using the regular expression method, and quickly deduplicating the text data by using the embedded vector similarity to obtain basic text data; wherein the interactive database question and answer system first needs to use entity extraction technology to extract entities from the text, and the entity extraction technology is a commonly used interactive question and answer system interaction method, which will not be described here, and is stored in the form of "subject-predicate-object" triplets, and a knowledge base, i.e. the interactive database of the application, is constructed, wherein the original data includes question text data of the user and answer data corresponding to the question text data, and generally one question text data corresponds to multiple answer data. The audio conversion module 102 is used for filtering the audio in the original data by calculating the short-time energy to obtain filtered data, and converting the filtered data into audio text data by using the ASR speech recognition method; the image conversion module 103 is used for removing blurred image data by using the sharpness detection method to obtain clear image data, and extracting the clear image data by using the OCR technology to obtain image text data; the multi-modal module 104 is used for integrating the basic text data, the audio text data and the image text data into multi-modal data; the multi-modal fusion module 105 is used for aligning the multi-modal data on a unified time axis based on the timestamp method to obtain aligned multi-modal data, and fusing the aligned multi-modal data by using the feature fusion method to obtain fused data; the standard module 106 is used for inputting the fused data into the T5 language model to correct the wrong characters to obtain corrected data, performing normalization processing on the corrected data according to the standard language in the interactive database to obtain standard data, and inserting implicit context labels in the standard data to obtain standard data. The implicit context label is an interactive method of automatically retrieving related content through keywords and sending it to the user, for example: the user asks "traffic balance" in the previous sentence, and the current sentence automatically supplements "[query context] traffic remaining 500MB". The traditional method only processes text data, while the method integrates voice, image, behavior log and other multi-source information, improves the integrity and accuracy of context understanding through alignment and fusion technology, updates the word segmentation dictionary and entity library through online learning, adapts to the changes of new terms and user expression habits, avoids the problem of insufficient generalization ability caused by the traditional static dictionary, dynamically corrects wrong characters based on the dialogue history, supplements implicit information, and solves the semantic deviation problem caused by the traditional method of relying on single sentence processing.

[0073] a word embedding method, and the word embedding method is a commonly used distributed word vector representation method. When the word embedding method is used, a large amount of text data needs to be collected and preprocessed first, so as to construct a corpus, and then the word vectors corresponding to the words contained in the corpus are trained by using the corpus. An existing commonly used method for converting standard data into vectors is a one-hot encoding method. If a discrete feature can take three values of excellent, good and medium, the one-hot encoding vectors corresponding to the three values are [1, 0, 0], [0, 1, 0] and [0, 0, 1] respectively. However, when the one-hot encoding method is used to represent words, each word in a word table is regarded as a value of a discrete feature. If the size of the word table is 10,000, the word vector will be as high as 10,000 dimensions, and only the dimension corresponding to the word to be represented is 1, and the other dimensions are 0. Such a method leads to a very large word vector dimension, which is very wasteful of storage space and has a high computational complexity but a small part actually participates in the calculation. In order to solve this problem, the specific implementation method is as follows:

[0074] The conversion module 201 is configured to convert the standard data into a preliminary word vector by the one-hot encoding method first, and then convert the preliminary word vector into a word vector by the word embedding method. The word vector is used to make each word in the user interaction question and answer sentence in the original data correspond to a low-dimensional vector. In practice, the dimension of the word is generally represented by the One-hot method, which is a 4-dimensional vector. The low dimension is less than or equal to 4 dimensions. Compared with the preliminary word vector generated by the one-hot encoding, the word vector converted by the word embedding method can be more conducive to operation, and can also contain information about the semantic similarity between words. The meaning matrix module 202 is configured to combine the word vectors into a meaning matrix based on the user interaction question and answer sentence in the original data. An example is as follows: the user interaction question and answer sentence is: Who are you? As a noun, it can be vector A, as a predicate verb, it can be vector B, and who is the object, it can be vector C. Therefore, the meaning matrix is ABC. The deep convolution module 203 is configured to determine a filter function according to the dimension of the meaning matrix, calculate the product of the filter function and each vector in the meaning matrix to obtain a single deep convolution vector, and splice all the single deep convolution vectors of the same meaning matrix to obtain a deep convolution matrix. The residual convolution module 204 is configured to sum the deep convolution matrix and the meaning matrix to obtain a residual convolution matrix. In order to enable the high-layer self-attention layer to obtain the information in the word vector, the residual method is used in this model. This method can be regarded as using a depth separable convolution to make up for the lack of information in the word vector, so as to fit the residual. At the same time, in the training process, the residual can also alleviate the problem of gradient disappearance to a certain extent.

[0075] The multi-head attention module 3 is used to construct a multi-head attention model and calculate a multi-head output matrix by the multi-head attention model through the residual convolution matrix. The multi-head attention model is constructed after the local feature extraction of the user question and answer sentences by using the residual convolution matrix. The global feature of the user question and answer sentences needs to be extracted by using the multi-head attention mechanism. The specific implementation steps are as follows:

[0076] The weight module 301 is used to obtain the number M of heads of the multi-head attention model by an artificial preset method, determine M groups of weight transfer matrices for mapping the residual convolution matrix to three different spaces based on the number M of heads, and each group contains three weight transfer matrices corresponding to different spaces. For example, there are M groups of weight transfer matrices. The Mth group of weight transfer matrices is A M , B M , and C M . The multi-head attention model is a common algorithm model for obtaining matrix features. The training and learning process and the construction method steps are not described here. The space module 302 is used to select a group of weight transfer matrices from the M groups of weight transfer matrices, so that each weight transfer matrix in the selected group is multiplied by the residual convolution matrix to calculate a first space matrix, a second space matrix, and a third space matrix. The repeated operation module 303 is used to repeatedly calculate the space matrix until all weight transfer matrices are multiplied by the residual convolution matrix to obtain a plurality of space matrices. The splicing module 304 is used to splice the plurality of space matrices to obtain a multi-head output matrix. In the prior art, the self-attention mechanism is often connected with two layers of forward full connection layers after obtaining the plurality of space matrices. This structure can be regarded as a self-attention mechanism for parameters. However, this structure adds a large number of training parameters to the model, making it difficult for the model to converge, thereby causing the model to fail to achieve the ideal effect.

[0077] Although the use of the residual structure can solve the problem of gradient disappearance to some extent, there are only a small number of training parameters above the layer, and the gradient disappearance is not easy to occur. Therefore, the multi-head attention model does not use the residual structure. At the same time, the use of the residual structure can make the parameter size in the multi-head attention model no longer limited by the input dimension size. Increasing the dimension can map the residual convolution matrix to a higher-dimensional space, thereby enabling the model to fit more complex function relationships and obtain stronger expressiveness. From the overall point of view, the multi-head attention model also plays a role in connecting the past and the future, providing a premise for accurate feature extraction for subsequent processing steps and improving the accuracy of natural language understanding.

[0078] The question and answer detection module 4 is configured to construct a question and answer self-attention model according to the multi-head output matrix, and perform similarity difference detection on the question matrix and the answer matrix in the multi-head output matrix through the question and answer self-attention model, so as to obtain a correct answer set according to a detection result. In the result of the multi-head output matrix, the prior art generally directly performs average pooling on the question part to obtain a vector representation of the question. However, there is an inseparable relationship between the question and the answer, and the determination of the answer cannot be made without considering the similarity with the question. In order to find the answer closely related to the question, the specific steps for solving the problem are as follows:

[0079] The division module 401 is configured to divide the multi-head output matrix into a question matrix and an answer matrix according to the question and answer sentences of the user interaction, and extract column vectors of the question matrix and the answer matrix to form a sub-question vector and a sub-answer vector, respectively. In the question and answer sentences of the user interaction, the length of a general question is short, and the length of an answer is much greater than that of the question. Therefore, the method can distinguish the question and answer sentences in the question and answer sentences of the user interaction, and the multi-head output matrix of the user question and the multi-head output matrix of the answer can also be determined by a reverse tracing method. The model construction module 402 is configured to construct a question and answer self-attention model, and calculate a question transition matrix of the question matrix and an answer transition matrix of the answer matrix by using a correlation similarity method. The attention weight module 403 is configured to calculate an attention weight according to the question transition matrix and the answer transition matrix. The calculation method is to multiply the question transition matrix by the sub-question vector, add the answer transition matrix multiplied by the sub-answer vector to obtain a question and answer vector, calculate an arithmetic average matrix of the question transition matrix and the answer transition matrix, and obtain a question attention weight by using the question and answer vector dot product arithmetic average matrix. The question proportion module 404 is configured to determine an exponential function by using an artificial preset method, calculate a question proportion value based on the exponential function, calculate a similarity difference based on the question proportion value and perform similarity difference detection, and obtain a correct answer set based on a detection result. The question and answer attention module 4041 is configured to multiply the question proportion value by each sub-answer vector and sum to obtain a question and answer attention value of the answer and the question. The average pooling module 4042 is configured to perform average pooling on the question matrix to obtain a question attention value. The similarity difference module 4043 is configured to calculate a question and answer similarity by using a cosine similarity method according to the question attention value and the question and answer attention value, and calculate a similarity difference by subtracting the question and answer similarity. The threshold module 4044 is configured to set a corresponding detection threshold for the similarity difference to represent a detection standard of whether the question and the answer have a high similarity. The detection threshold can be set according to actual needs. In this embodiment, an example is given: the detection threshold is set to 0.15. If the question and answer similarity is 0.8, 1 minus 0.8 is equal to 0.2, and 0.2 is greater than 0.15, which indicates that the similarity difference is large and the similarity is not high. If the question and answer similarity is 0.95, 1 minus 0.95 is equal to 0.05, and 0.05 is less than 0.15, which indicates that the similarity difference is small and the similarity is very high. The correct answer module 4045 is configured to form a correct question and answer group according to the detection threshold, and form a correct answer set according to the question and answer attention value. Using the cosine similarity as a similarity measurement function will not add exponential, logarithmic and other relatively complex operators to the operation of the model, can avoid increasing additional parameters, thereby reducing the complexity of the model, and greatly improving the operation efficiency.

[0080] The emotion recognition module 5 is used for performing multiple pooling operations on the question matrix, generating trained feature extraction parameters through a deep convolutional neural network method, performing feature extraction on the question matrix according to the feature extraction parameters, determining the emotion of the question matrix through a full connection layer and a softmax function, and obtaining an emotional question matrix. The prior art generally performs emotion recognition through a deep learning training network, but the deep learning training network often needs a large amount of data and parameters, and overfitting problems are prone to occur on a small data set of emotions. In order to solve this problem, the specific implementation steps are as follows:

[0081] The first pooling module 501 is configured to perform first pooling on the question matrix through a pooling layer to obtain a first convolutional block. In this embodiment, an example is given: the question matrix is taken as input to be convolved twice through 64 convolutional kernels with a size of 3x3 and a channel of 3 and a step of 1, and the size of the output is 224x224x64; the image size is halved through a max pooling layer with a pooling kernel of 2x2 and a step of 2, and the size of the pooled image is 112x112x64, which is the first convolutional block of the improved processing model; the second pooling module 502 is configured to perform second pooling on the first convolutional block to obtain a second convolutional block; the second pooling is performed in the following steps: the first convolutional block is input to be convolved twice through 128 convolutional kernels with a size of 3x3, and the size of the image is changed to 112x112x128 after ReLU activation; the image is further pooled through a max pooling layer, and the size of the pooled image is changed to 56x56x128, which is the second convolutional block; the third pooling module 503 is configured to perform third pooling on the second convolutional block to obtain a third convolutional block; the third pooling is performed in the following steps: the second convolutional block is input to be convolved three times through 256 convolutional kernels with a size of 3x3, and the size of the image is changed to 56x56x256; the image is further pooled through a max pooling layer, and the size of the pooled image is changed to 28x28x256, which is the third convolutional block; the fourth pooling module 504 is configured to perform fourth pooling on the third convolutional block to obtain a fourth convolutional block; the fourth pooling is performed in the following steps: the third convolutional block is input to be convolved three times through 512 convolutional kernels with a size of 3x3, and the size of the image is changed to 28x28x512; the image is further pooled through a max pooling layer, and the size of the pooled image is changed to 14x14x512, which is the fourth convolutional block; the parameter module 505 is configured to perform pooling on the fourth convolutional block to obtain a fifth convolutional block, and generate trained feature extraction parameters through a deep convolutional neural network method; the fifth pooling is performed in the following steps: the fourth convolutional block is taken as input to be convolved three times through 512 convolutional kernels with a size of 3x3, and the size of the image is changed to 14x14x512; the image is further pooled through a max pooling layer, and the size of the pooled image is changed to 7x7x512, which is the fifth convolutional block; the feature module 506 is configured to extract features of the question matrix through the fifth convolutional block and the feature extraction parameters to obtain question features, convert the question features into normalized probabilities through a full connection layer and a softmax function, and identify the sentiment of the question matrix according to the normalized probabilities to obtain a sentiment question matrix; the convolutional block is used as a feature extractor to extract features of the question matrix, which includes parameter migration of a shallow network and feature extraction of sentiment features in the question matrix by a deep network, and the extracted features include both general features extracted from the pooling model and representative features of the speech emotion dataset; then, a 1024-dimensional full connection layer and a 7-dimensional full connection layer representing output categories are connected, the image is converted into normalized probabilities representing categories through a softmax function, and finally, the sentiment categories are determined according to the normalized probabilities. ;

[0082] wherein, , set p1 represents the type of excitement with very high expectations for the result, p2 represents the type of happiness that is very accepting of the result and gives praise, p3 represents the type of calm that uses calm or rational words, and does not show positive or negative emotions, p4 represents the type of sadness that uses negative words and contains pessimistic expressions, p5 represents the type of anger that has more extreme words in the speech, and the punctuation also has emotional implications, p6 represents the type of fear that has more interrogative and hypothetical words in the speech, and has negative expectations for the result that has not occurred, and p7 represents the type of disgust that uses explicit negative or negative words for the result, the seven normalized probabilities are arranged in ascending order, and the emotion corresponding to the highest normalized probability is taken as the emotion of the question matrix, for example, p5 is the largest among the seven normalized probabilities, so the anger corresponding to p5 is the emotion of the question matrix, the prior art generally uses CNN for convolution and pooling, but the parameters of CNN are generally more, and the commonly used training data requires tens of thousands of data or more, while the voice emotion database is mostly a small data set of a few hundred, training in the CNN network is prone to overfitting, therefore, the present application uses a parameter migration algorithm of a multiple pooling training model to migrate the parameters trained by the deep convolutional neural network method, the knowledge learned by the shallow network is the general features of image data, and as the number of network layers increases, the knowledge learned by the deeper network layers also has more characteristics of the data set itself, therefore, the parameters of the general features learned by the shallow network can be migrated and reused on the voice emotion data set.

[0083] The emotion answering module 6 is configured to repeatedly pass the correct answer set through multiple pooling operations, fully connected layers and a softmax function to obtain an emotion answer set composed of multiple emotion answer matrices, since the input steps are completely the same, no further description is given here, the specific emotions of the question and the answer can be grasped through emotion recognition, but since the answers of the answer are all from the system, the emotion detection of the answer is not the focus, and no detailed description is given here.

[0084] The language interaction module 7 is used for calculating a mixed characteristic value of an emotional question matrix and an emotional answer set, obtaining a fusion vector based on the mixed characteristic value, inputting the fusion vector into a preliminarily constructed classification neural network model to obtain a category probability value, obtaining an optimized natural language interaction system based on a loss value S calculated based on the category probability value; after multiple rounds of interaction with the question and answer system, the user sometimes makes inconsistent descriptions about a certain thing, which will affect the judgment of the system, resulting in that the system cannot give an accurate answer; therefore, it is necessary for the question and answer system to have the ability to judge whether the input of the user is contradictory, and when the description of the user is contradictory, the system can timely feedback, in order to solve this problem, the specific implementation steps are as follows:

[0085] The sentence meaning coding module 701 is configured to form a to-be-coded matrix according to the order of the question and answer sentences of the user interaction, and to obtain a sentence meaning coding matrix by performing semantic coding on the to-be-coded matrix through an LSTM encoder; the LSTM encoder is an encoder composed of an LSTM model, and can code the emotional question matrix and the emotional answer set containing emotions, facilitating the operation of subsequent steps, which will not be described herein; the mixed feature module 702 is configured to extract the maximum value of the vector in the sentence meaning coding matrix as a first feature value, input the emotional question matrix and the emotional answer set into a multi-head attention model to obtain a multi-dimensional feature value, and splice the first feature value and the multi-dimensional feature value to obtain a mixed feature value; the maximum pooling is configured to obtain the most significant features, and can extract the most important features in each feature extractor, however, the maximum pooling method only extracts the part that best reflects the features of the sentence from each dimension in the entire sentence, and directly selects the discarded mode for the less important features, and finally the obtained vector is not strong enough to express the sentence, therefore, the multi-dimensional attention model is used to calculate a score for each feature of each word, which can select the features that best describe the specific meaning of the word in any given context, and include the information in the sentence coding output, and splicing the two features can compensate for each other, and improve the accuracy of the answer reply in the repeated multi-round question and answer process; the fusion module 703 is configured to randomly select two mixed feature values as a first feature value and a second feature value, obtain an absolute value by performing element-by-element subtraction on the first feature value and the second feature value, obtain a multiplication value by performing element-by-element multiplication on the first feature value and the second feature value, and splice the first feature value, the second feature value, the absolute value and the multiplication value to obtain a fusion vector; the mixed feature value, the first feature value and the second feature value are all vectors containing multiple elements, and therefore the elements in the vector can be operated, and the first feature value and the second feature value are often highly similar, in order to improve the difference between the two sentences and make the classification more accurate, the two need to be fused into sentence vectors; the category module 704 is configured to construct a classification neural network model containing a feedforward neural network layer, an output layer and a classification layer, take the fusion vector as an input, and obtain a category probability value through the feedforward neural network layer, the output layer and the classification layer, wherein the activation function of the feedforward neural network layer is a ReLU function, the output layer is a linear layer with the same number of neurons as the total number of categories in the data set, and the activation function of the classification layer is a softmax function; the loss module 705 is configured to obtain an actual category z of the fusion vector, and calculate a loss value S for measuring the loss of the model, and the calculation formula is:

[0086]

[0087] wherein, The actual category probability value representing the fusion vector; the formula of the loss value S is a simplified formula, because the fusion vector itself is an encoding vector, so only the real category probability value needs to be calculated to obtain the loss value S; the model optimization module 706 is used for setting a threshold value of the loss value S according to an empirical method, if the loss value S is greater than or equal to the threshold value, the accuracy of the model is low, and the model can be optimized through the back propagation method and the stochastic gradient descent optimizer until the loss value S is less than the threshold value, and the natural language interaction system is obtained.

[0088] The application uses a hybrid pooling method of weighted average of maximum pooling and multi-dimensional attention on the sentence meaning encoding matrix to obtain the vector representation of the sentence, and finally obtains the final classification result through a multi-layer feedforward neural network. Experimental results show that, compared with other natural language reasoning models based on deep learning, the model can more accurately classify the relationship of the sentence according to the sentence meaning, uses a Softmax function to output a probability distribution, not only gives the final classification result, but also provides the probability of each category, so that the prediction result of the model is more interpretable and reliable, and the combination of the cross-entropy loss function and the Softmax provides a very clear and efficient optimization direction for the model, which can directly maximize the logarithmic probability of the correct category, the gradient is clear, and the convergence speed is fast.

[0089] The above embodiments can be realized wholly or partially by software, hardware, firmware or any combination thereof. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product. Those skilled in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solutions.

[0090] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, which can be located in one place or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment according to actual needs.

[0091] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A natural language interactive intelligent customer service system, characterized in that, The system comprises: A specification module (1) for collecting original data from the database of the intelligent customer service system and performing text normalization processing on the original data to obtain standard data, wherein the original data includes text data, voice data and image data; A word vector module (2) for converting the standard data into word vectors through word embedding, integrating the word vectors into a meaning matrix based on the original data, and performing residual convolution operation on the meaning matrix to obtain a residual convolution matrix; A multi-head attention module (3) for constructing a multi-head attention model and calculating a multi-head output matrix from the residual convolution matrix through the multi-head attention model; An question and answer detection module (4) for constructing a question and answer self-attention model based on the multi-head output matrix, and performing similarity difference detection on the question matrix and the answer matrix in the multi-head output matrix through the question and answer self-attention model to obtain a correct answer set according to the detection result; An emotion recognition module (5) for performing multiple pooling operations on the question matrix, generating feature extraction parameters through a deep convolutional neural network, extracting features from the question matrix according to the feature extraction parameters, and determining the emotion of the question matrix through a full connection layer and a softmax function to obtain an emotional question matrix; An emotional answer module (6) for taking the correct answer set as input to obtain an emotional answer set composed of multiple emotional answer matrices through multiple pooling operations, a full connection layer and a softmax function; A language interaction module (7) for calculating the hybrid feature value of the emotional question matrix and the emotional answer set, obtaining a fusion vector based on the hybrid feature value, inputting the fusion vector into a preliminarily constructed classification neural network model to obtain a class probability value, and obtaining an optimized natural language interaction system based on the loss value S calculated from the class probability value; The language interaction module (7) comprises: A sentence meaning encoding module (701) for constructing a to-be-encoded matrix from the emotional question matrix and the emotional answer set in the order of the question and answer sentences of user interaction, and performing word sense encoding on the to-be-encoded matrix through an LSTM encoder to obtain a sentence meaning encoding matrix; A hybrid feature module (702) for extracting the maximum value of the vector in the sentence meaning encoding matrix as a first feature value, inputting the emotional question matrix and the emotional answer set into a multi-head attention model to obtain a multi-dimensional feature value, and splicing the first feature value and the multi-dimensional feature value to obtain a hybrid feature value; A fusion module (703) for randomly selecting two of the multiple hybrid feature values as a first feature value and a second feature value, obtaining the absolute value of the result of the element-by-element subtraction of the first feature value and the second feature value, obtaining a multiplication value by multiplying the first feature value and the second feature value element by element, and splicing the first feature value, the second feature value, the absolute value and the multiplication value to obtain a fusion vector; A class module (704) for constructing a classification neural network model containing a feedforward neural network layer, an output layer and a classification layer, inputting the fusion vector, and obtaining a class probability value through the feedforward neural network layer, the output layer and the classification layer; A loss module (705) for obtaining the actual class z of the fusion vector and calculating the loss value S for measuring the loss of the model. The model optimization module (706) is configured to set a threshold value of the loss value S, and optimize the model by using a back propagation method and a stochastic gradient descent optimizer if the loss value S is greater than or equal to the threshold value, until the loss value S is less than the threshold value, and obtain the natural language interaction system. 2.The natural language interactive intelligent customer service system of claim 1, wherein: The specification module (1) comprises: The deduplication module (101) is configured to remove special characters in the text data by using a regular expression method, and quickly deduplicate the text data by using an embedding vector similarity to obtain basic text data; The audio conversion module (102) is configured to filter audio in the original data by using a short-time energy calculation method to obtain filtered data, and convert the filtered data into audio text data by using an ASR speech recognition method; The image conversion module (103) is configured to remove blurred image data by using a sharpness detection method to obtain clear image data, and extract the clear image data by using an OCR technology to obtain image text data; The multi-modal module (104) is configured to integrate the basic text data, the audio text data and the image text data into multi-modal data; The multi-modal fusion module (105) is configured to align the multi-modal data on a unified time axis based on a timestamp method to obtain aligned multi-modal data, and fuse the aligned multi-modal data by using a feature fusion method to obtain fused data; The standard module (106) is configured to input the fused data into a T5 language model to correct errors in the data to obtain corrected data, normalize the corrected data according to specification language in an interaction database to obtain standardized data, and insert implicit context labels into the standardized data to obtain standard data. 3.The natural language interactive intelligent customer service system of claim 1, wherein: The word vector module (2) comprises: The conversion module (201) is configured to convert the standard data into a preliminary word vector by using a one-hot encoding method, and convert the preliminary word vector into a word vector by using a word embedding method; The meaning matrix module (202) is configured to combine the word vector into a meaning matrix according to a question and answer sentence in user interaction in the original data; The deep convolution module (203) is configured to determine a filter function according to a dimension of the meaning matrix, calculate a product of the filter function and each vector in the meaning matrix to obtain a single deep convolution vector, and splice all single deep convolution vectors of the same meaning matrix to obtain a deep convolution matrix; The residual convolution module (204) is configured to sum the deep convolution matrix and the meaning matrix to obtain a residual convolution matrix. 4.The natural language interaction intelligent customer service system of claim 3, wherein: The word vector is configured to make each word in the question and answer sentence in the user interaction in the original data correspond to a low-dimensional vector.

5. The natural language interactive intelligent customer service system of claim 1, wherein: The multi-head attention module (3) comprises: The weight module (301) is configured to obtain a number M of heads of a multi-head attention model, determine M groups of weight transfer matrices for mapping the residual convolution matrix to three different spaces based on the number M of heads, and each group contains three weight transfer matrices corresponding to different spaces; The space module (302) is configured to select a group of weight transfer matrices from the M groups of weight transfer matrices, and calculate a first space matrix, a second space matrix and a third space matrix by multiplying each weight transfer matrix in the selected group with the residual convolution matrix, respectively. The repeating operation module (303) is configured to repeatedly calculate the spatial matrix until all weight transfer matrices are multiplied by the residual convolution matrix to obtain a plurality of spatial matrices. The splicing module (304) is configured to splice the plurality of spatial matrices to obtain a multi-head output matrix.

6. The natural language interactive intelligent customer service system of claim 1, wherein: The question and answer detection module (4) comprises: The division module (401) is configured to divide the multi-head output matrix into a question matrix and an answer matrix according to the question and answer sentences of user interaction, and extract column vectors of the question matrix and the answer matrix to form a sub-question vector and a sub-answer vector, respectively. The model construction module (402) is configured to construct a question and answer self-attention model, and calculate a question transfer matrix of the question matrix and an answer transfer matrix of the answer matrix by a correlation similarity method. The attention weight module (403) is configured to calculate an attention weight according to the question transfer matrix and the answer transfer matrix, and the calculation method is to multiply the question transfer matrix by the sub-question vector, add the answer transfer matrix multiplied by the sub-answer vector to obtain a question and answer vector, calculate an arithmetic average matrix of the question transfer matrix and the answer transfer matrix, and obtain the question attention weight by using the question and answer vector dot product of the arithmetic average matrix. The question proportion module (404) is configured to determine an exponential function, calculate a question proportion value based on the exponential function, and obtain a correct answer set by a similarity difference method according to the question proportion value.

7. The natural language interactive intelligent customer service system of claim 6, wherein: The question proportion module (404) comprises: The question and answer attention module (4041) is configured to multiply the question proportion value by each sub-answer vector and sum to obtain a question and answer attention value of the answer and the question. The average pooling module (4042) is configured to perform average pooling on the question matrix to obtain a question attention value. The similarity difference module (4043) is configured to calculate a question and answer similarity by a cosine similarity method according to the question attention value and the question and answer attention value, and calculate a similarity difference by subtracting the question and answer similarity. The threshold module (4044) is configured to set a corresponding detection threshold for the similarity difference. The correct answer module (4045) is configured to form a correct question and answer group according to the detection threshold, and form a correct answer set according to the question and answer attention value. 8.The natural language interaction intelligent service system according to claim 1, characterized in that: The sentiment recognition module (5) comprises: The first pooling module (501) is configured to perform first pooling on the question matrix through a pooling layer to obtain a first convolution block. The second pooling module (502) is configured to perform second pooling on the first convolution block to obtain a second convolution block. The third pooling module (503) is configured to perform third pooling on the second convolution block to obtain a third convolution block. The fourth pooling module (504) is configured to perform fourth pooling on the third convolution block to obtain a fourth convolution block. The parameter module (505) is configured to perform pooling on the fourth convolution block to obtain a fifth convolution block, and generate a trained feature extraction parameter by a deep convolutional neural network method. The feature module (506) is configured to perform feature extraction on the question matrix by the fifth convolution block and the feature extraction parameter to obtain a question feature, and convert the question feature by a full connection layer and a softmax function to obtain a normalized probability. The emotion matrix module (507) is configured to identify the emotion of the problem matrix according to the normalized probability and obtain an emotion problem matrix.

Images