Natural language interaction intelligent customer service system

Abstract

The invention discloses a natural language interaction intelligent customer service system, and relates to the technical field of intelligent interaction. The system comprises a standardization module used for collecting original data from a database of an intelligent customer service system and performing text standardization processing on the original data to obtain standard data, the original data including text data, voice data and image data; the word vector module is used for converting the standard data into word vectors through a word embedding method; the method is technically characterized in that the relations of sentences can be classified more accurately according to sentence meanings, probability distribution is output by using a Softmax function, a final classification result is given, the probability of each category is provided, the prediction result of the model is more interpretable and reliable, and the prediction efficiency of the model is improved. The combination of the cross entropy loss function and Softmax provides a very clear and efficient optimization direction for the model, the logarithmic probability of a correct category can be directly maximized, the gradient is clear, and the convergence speed is high.

Description

Technical Field

[0001] This invention relates to the field of intelligent interaction technology, specifically to a natural language interactive intelligent customer service system. Background Technology

[0002] A natural language interactive intelligent customer service system is a computer system that uses artificial intelligence technology to simulate human customer service, engaging in multi-turn dialogues with users through natural language to automatically answer questions and provide services. Its principle involves using natural language processing technology to segment and syntactically analyze the text or speech input by the user, understanding the underlying intent and key information. Based on the identified intent, it searches for the most relevant answer in a knowledge base such as product manuals, frequently asked questions (FAQs), or databases, and then replies to the user with the found answer or generated text in clear and natural language, completing an interaction. However, in existing technologies, self-attention mechanisms often connect two forward fully connected layers after obtaining multiple spatial matrices. This structure can be seen as a parameter-based self-attention mechanism, but it adds a large number of training parameters to the model, making it difficult for the model to converge and thus preventing it from achieving ideal results. Furthermore, the question portion is typically directly average-pooled to obtain the question's vector representation. However, there is an inseparable connection between questions and answers, and the determination of the answer cannot ignore its similarity to the question. Therefore, it is crucial to find answers closely related to the question. Summary of the Invention

[0003] To achieve the above objectives, the present invention provides the following technical solution: A natural language interactive intelligent customer service system, the system comprising: The standardization module is used to collect raw data from the database of the intelligent customer service system and perform text standardization processing on the raw data to obtain standard data. The raw data includes text data, voice data and image data. The word vector module is used to convert standard data into word vectors through word embedding, integrate the word vectors into a meaning matrix based on the original data, and perform residual convolution operation on the meaning matrix to obtain a residual convolution matrix; The multi-head attention module is used to construct a multi-head attention model and calculate the multi-head output matrix from the residual convolution matrix through the multi-head attention model. The question-answering detection module is used to construct a question-answering self-attention model based on the multi-head output matrix, and to perform similarity difference detection on the question matrix and answer matrix in the multi-head output matrix through the question-answering self-attention model, and obtain the set of correct answers based on the detection results; The emotion recognition module performs multiple pooling operations on the question matrix and generates feature extraction parameters using a deep convolutional neural network. Based on the feature extraction parameters, it extracts features from the question matrix and then uses a fully connected layer and a softmax function to determine the emotion of the question matrix and obtain the emotion question matrix. The sentiment response module is used to take the set of correct responses as input and pass it through multiple pooling operations, fully connected layers, and the softmax function to obtain a sentiment response set consisting of multiple sentiment response matrices; The language interaction module is used to calculate the mixed feature values ​​of the sentiment question matrix and the sentiment response set, and to obtain a fusion vector based on the mixed feature values. The fusion vector is then input into the initially constructed classification neural network model to obtain the category probability value. The loss value S calculated based on the category probability value is used to obtain the optimized natural language interaction system.

[0004] Furthermore, the specification module includes: The deduplication module is used to remove special characters from text data using regular expressions and to quickly deduplicat text data by embedding vector similarity to obtain the basic text data. The audio conversion module is used to filter the audio in the original data by calculating short-time energy to obtain filtered data, and then convert the filtered data into audio text data using ASR speech recognition. The image conversion module is used to remove blurry image data through sharpness detection to obtain clear image data, and then use OCR technology to extract image text data from the clear image data. The multimodal module is used to integrate basic text data, audio text data, and image text data into multimodal data; based on the timestamp method, the multimodal data is aligned on a unified time axis to obtain aligned multimodal data, and the aligned multimodal data is fused using the feature fusion method to obtain fused data; The standard module is used to input the fused data into the T language model to correct typos and obtain corrected data. It then performs standardization processing on the corrected data according to the standard terms in the interactive database to obtain standard data, and inserts implicit context labels into the standard data to obtain standard data.

[0005] Furthermore, the word vector module includes: The conversion module is used to first convert standard data into preliminary word vectors using one-hot encoding, and then convert the preliminary word vectors into word vectors using word embedding. The meaning matrix module is used to combine word vectors into a meaning matrix based on the question and answer statements of user interactions in the original data. The deep convolution module is used to determine the filter function based on the dimension of the meaning matrix, calculate the product of the filter function and each vector in the meaning matrix to obtain a single deep convolution vector, and concatenate all the single deep convolution vectors of the same meaning matrix to obtain a deep convolution matrix. The residual convolution module is used to sum the depthwise convolution matrix and the meaning matrix to obtain the residual convolution matrix.

[0006] Furthermore, the word vectors are used to assign a low-dimensional vector to each word in the question-and-answer statements of user interaction in the original data.

[0007] Furthermore, the multi-head attention module includes: The weight module is used to obtain the number M of heads in the multi-head attention model. Based on the number M of heads, M sets of weight transfer matrices are determined to map the residual convolution matrix to three different spaces. Each set contains three weight transfer matrices corresponding to different spaces. The spatial module is used to select a set of weight transfer matrices from M sets of weight transfer matrices, and multiply each weight transfer matrix in the selected set by the residual convolution matrix to calculate the first spatial matrix, the second spatial matrix, and the third spatial matrix. The repetitive computation module is used to repeatedly compute the spatial matrix until all weight transition matrices are multiplied by the residual convolution matrix to obtain multiple spatial matrices; The splicing module is used to splice multiple spatial matrices to obtain a multi-head output matrix.

[0008] Furthermore, the question-answering detection module includes: The partitioning module is used to divide the multi-head output matrix into a question matrix and an answer matrix based on the user's interactive question and answer statements, and extract the column vectors of the question matrix and the answer matrix respectively to form a sub-question vector and a sub-answer vector; The model building module is used to build a question-answering self-attention model and calculate the question transition matrix of the question matrix and the answer transition matrix of the answer matrix using the association similarity method. The attention weight module is used to calculate attention weights based on the question transition matrix and the answer transition matrix. The calculation method is to multiply the question transition matrix by the sub-question vector, add the answer transition matrix multiplied by the sub-answer vector to obtain the question-answer vector, calculate the arithmetic mean matrix of the question transition matrix and the answer transition matrix, and use the dot product of the question-answer vectors to obtain the question attention weight. The question proportion module is used to determine an exponential function and calculate the question proportion value based on the exponential function. The correct answer set is obtained by using the similarity difference method based on the question proportion value.

[0009] Furthermore, the problem proportion module includes: The question-answering attention module is used to multiply the question proportion value by each sub-answer vector and sum them to obtain the question-answering attention value between the answer and the question. The average pooling module is used to perform average pooling on the problem matrix to obtain the problem attention value; The similarity difference module is used to calculate the question-answer similarity based on the question attention value and the question-answer attention value using the cosine similarity method, and to calculate the similarity difference by subtracting the question-answer similarity from the answer similarity. The threshold module is used to set the corresponding detection threshold for the similarity difference; The correct answer module is used to form correct question-answer groups based on the question attention value and the question-answer attention value according to the detection threshold, and to form the answer set corresponding to the question-answer attention value into a correct answer set.

[0010] Furthermore, the emotion recognition module includes: The initial pooling module is used to perform the first pooling of the problem matrix through the pooling layer to obtain the first convolutional block; The secondary pooling module is used to perform a second pooling on the first convolutional block to obtain the second convolutional block; The third pooling module is used to perform a third pooling on the second convolutional block to obtain a third convolutional block; The fourth pooling module is used to perform a fourth pooling on the third convolutional block to obtain a fourth convolutional block; The parameter module is used to pool the fourth convolutional block to obtain the fifth convolutional block, and generate trained feature extraction parameters using a deep convolutional neural network method. The feature module is used to extract features from the problem matrix using the fifth convolutional block and feature extraction parameters to obtain problem features. Based on the problem features, normalized probabilities are obtained by transformation through a fully connected layer and a softmax function. The sentiment matrix module is used to identify the sentiment of a question matrix based on normalized probabilities and obtain the sentiment question matrix.

[0011] Furthermore, the language interaction module includes: The sentence semantic encoding module is used to construct the matrix to be encoded from the sentiment question matrix and the sentiment answer set according to the order of the question and answer statements in the user interaction. The LSTM encoder is used to perform word semantic encoding on the matrix to be encoded to obtain the sentence semantic encoding matrix. The hybrid feature module is used to extract the maximum value of the vector in the semantic encoding matrix as the first feature value, input the sentiment question matrix and sentiment response set into the multi-head attention model to obtain multi-dimensional feature values, and concatenate the first feature value and multi-dimensional feature values ​​to obtain the hybrid feature value; The fusion module is used to randomly select two features from multiple mixed features as the first feature and the second feature. The absolute value is obtained by subtracting the first feature and the second feature element by element. The product value is obtained by multiplying the first feature and the second feature element by element. The fusion vector is obtained by concatenating the first feature, the second feature, the absolute value and the product value. The category module is used to build a classification neural network model containing a feedforward neural network layer, an output layer, and a classification layer. It takes the fused vector as input and obtains the category probability value through the feedforward neural network layer, the output layer, and the classification layer. The loss module is used to obtain the actual class z of the fused vector and calculate the loss value S used to measure the loss of the model; The model optimization module is used to set a threshold for the loss value S. If the loss value S is greater than or equal to the threshold, the model is optimized using backpropagation and stochastic gradient descent optimizer until the loss value S is less than the threshold, and a natural language interaction system is obtained.

[0012] This invention provides a natural language interactive intelligent customer service system, which has the following beneficial effects: 1) This solution improves the completeness and accuracy of contextual understanding through alignment and fusion technology. It updates the word segmentation dictionary and entity database through online learning to adapt to changes in new terms and user expression habits, avoiding the problem of insufficient generalization ability caused by traditional static dictionaries. It also dynamically corrects typos and supplements implicit information based on dialogue history, solving the semantic bias problem caused by traditional methods that only rely on single-sentence processing. 2) This scheme adopts a multi-head attention model without residual structure, which makes the parameter size in the multi-head attention model no longer limited by the size of the input dimension. Increasing the dimension can map the residual convolution matrix to a higher-dimensional space, thereby enabling the model to fit more complex functional relationships and obtain stronger expressive power. From an overall perspective, the multi-head attention model also plays a bridging role, providing the premise for accurate feature extraction in subsequent processing steps and improving the accuracy of natural language understanding. 3) This scheme uses a parameter transfer algorithm for training the model through multiple pooling. It transfers the parameters trained by the deep convolutional neural network method. The knowledge learned by the shallow network is the general feature of the image data. As the number of network layers increases, the knowledge learned by the deeper network layers is more characteristic of the dataset itself. Therefore, the parameters of the general features learned by the shallow network can be transferred and reused on the speech emotion dataset. 4) This scheme uses a hybrid pooling method of weighted averaging of max pooling and multidimensional attention to obtain the vector representation of the sentence semantic encoding matrix. Finally, the final classification result is obtained through a multi-layer feedforward neural network. Experimental results show that compared with other deep learning-based natural language inference models, this model can classify the relationship of sentences more accurately according to the sentence meaning. Using the Softmax function to output the probability distribution not only provides the final classification result, but also provides the probability of each category, making the model's prediction results more interpretable and reliable. The combination of cross-entropy loss function and Softmax provides the model with a very clear and efficient optimization direction, which can directly maximize the log probability of the correct category, with clear gradient and fast convergence speed. Attached Figure Description

[0013] Figure 1 This is a structural block diagram of a natural language interactive intelligent customer service system according to the present invention; Figure 2 This is a structural block diagram of the specification module in this invention; Figure 3 This is a structural block diagram of the word vector module in this invention; Figure 4 This is a structural block diagram of the multi-head attention module in this invention; Figure 5 This is a structural block diagram of the question-answering detection module in this invention; Figure 6 This is a structural block diagram of the problem ratio module in this invention; Figure 7 This is a structural block diagram of the emotion recognition module in this invention; Figure 8 This is a structural block diagram of the language interaction module in this invention.

[0014] In the diagram: 1. Standardization Module; 101. Deduplication Module; 102. Audio Conversion Module; 103. Image Conversion Module; 104. Multimodal Module; 105. Multimodal Fusion Module; 106. Standard Module; 2. Word Vector Module; 201. Conversion Module; 202. Meaning Matrix Module; 203. Depthwise Convolution Module; 204. Residual Convolution Module; 3. Multi-head Attention Module; 301. Weight Module; 302. Spatial Module; 303. Repeated Operation Module; 304. Concatenation Module; 4. Question Answering Detection Module; 401. Segmentation Module; 402. Model Building Module; 403. Attention Weight Module; 404. Question Ratio Module ; 4041, Question Answering Attention Module; 4042, Average Pooling Module; 4043, Similarity Difference Module; 4044, Threshold Module; 4045, Correct Answer Module; 5, Sentiment Recognition Module; 501, Primary Pooling Module; 502, Secondary Pooling Module; 503, Tertiary Pooling Module; 504, Quaternary Pooling Module; 505, Parameter Module; 506, Feature Module; 507, Sentiment Matrix Module; 6, Sentiment Response Module; 7, Language Interaction Module; 701, Sentence Semantic Encoding Module; 702, Hybrid Feature Module; 703, Fusion Module; 704, Category Module; 705, Loss Module; 706, Model Optimization Module. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0016] Please see Figure 1 This embodiment provides a natural language interactive intelligent customer service system, which includes: Standardization Module 1 is used to collect raw data from the database of the intelligent customer service system and perform text standardization processing on the raw data to obtain standard data. The raw data includes text data, voice data, and image data. Since the raw data has not been processed, it may contain special characters, advertising text, meaningless repetitive sentences, etc., which seriously affects the processing efficiency of the language interaction system and increases the data processing time. To solve the above problems, the specific implementation steps are as follows: The deduplication module 101 is used to remove special characters from the text data using regular expressions and to quickly deduplicate the text data using embedding vector similarity to obtain the basic text data. The interactive database, a question-and-answer system, first needs to utilize entity extraction technology to extract entities from the text. Entity extraction technology is a commonly used interactive question-and-answer system interaction method, which will not be elaborated here. It stores entities in the form of "subject-predicate-object" triples to construct a knowledge base, namely the interactive database of this invention. The original data includes the user's question text data and the corresponding answer data; generally, one question text data corresponds to multiple answer data. The audio conversion module 102 is used to filter the audio in the original data by calculating short-time energy to obtain filtered data, and then convert the filtered data into audio-text data using ASR speech recognition. The image conversion module 103 is used to remove blurry image data by sharpness detection to obtain clear image data, and then extract image-text data by OCR technology from the clear image data. The multimodal module 104 is used to integrate basic text data, audio-text data, and image-text data into multimodal data. The multimodal fusion module 105 is used to align the multimodal data on a unified time axis based on timestamps to obtain aligned multimodal data, and then fuse the aligned multimodal data using feature fusion to obtain fused data. The standardization module 106 is used to input the fused data into a T5 language model for typo correction to obtain corrected data, to standardize the corrected data according to the standard terms in the interactive database to obtain standardized data, and to insert implicit context labels into the standardized data to obtain standard data. Implicit contextual tagging is an interactive method that automatically retrieves relevant content based on keywords and sends it to the user. For example, if the user asked "data balance" in the previous sentence, this sentence will automatically add "[Query context] Data balance 500MB". Traditional methods only process text data, while this method integrates multi-source information such as voice, images, and behavior logs. It improves the completeness and accuracy of contextual understanding through alignment and fusion technology. It updates the word segmentation dictionary and entity database through online learning to adapt to changes in new terms and user expression habits, avoiding the problem of insufficient generalization ability caused by traditional static dictionaries. It dynamically corrects typos and supplements implicit information based on dialogue history, solving the semantic bias problem caused by traditional methods that only rely on single-sentence processing.

[0017] Word vector module 2 is used to convert standard data into word vectors using word embedding. Based on the original data, the word vectors are integrated into a meaning matrix, and residual convolution is performed on the meaning matrix to obtain the residual convolution matrix. Word embedding is a commonly used distributed word vector representation method. When using this method, a large amount of text data needs to be collected and preprocessed to build a corpus. Then, the word vectors corresponding to the words in the corpus are trained using this corpus. The existing common method for converting standard data into vectors is one-hot encoding. The method assumes that a certain discrete feature can take three values: excellent, good, and average. Then, the one-hot encoded vectors corresponding to these three values ​​are [1,0,0], [0,1,0], and [0,0,1], respectively. However, when using one-hot encoding to represent words, each word in the vocabulary is treated as a value of the discrete feature. If the size of the vocabulary is 10,000, then the word vector will be as high as 10,000 dimensions, of which only the dimension corresponding to the word to be represented is 1, and the others are 0. This method results in a very high dimensionality of word vectors, which is very wasteful of storage space and has high computational complexity, but only a small portion of the computation actually participates in the calculation. To solve this problem, the specific implementation method is as follows: The conversion module 201 is used to first convert standard data into preliminary word vectors using one-hot encoding, and then convert the preliminary word vectors into word vectors using word embedding. These word vectors are used to ensure that each word in the user interaction question-and-answer statements in the original data corresponds to a low-dimensional vector. In practice, the dimension of a word is generally represented using one-hot encoding, resulting in a 4-dimensional vector. Low-dimensional means less than or equal to 4 dimensions. Compared to the preliminary word vectors generated by one-hot encoding, the word vectors converted by word embedding are more computationally efficient and also contain information about the semantic similarity between words. The meaning matrix module 202 is used to combine the word vectors into a meaning matrix based on the user interaction question-and-answer statements in the original data. For example, if the user interaction question-and-answer statement is: "Who are you?", the word vectors are used to convert the user interaction question-and-answer statements into a meaning matrix. "You" as a noun can be vector A, "is" as a predicate verb can be vector B, and "who" as an object can be vector C; therefore, the meaning matrix is ​​ABC. The deep convolution module 203 determines the filter function based on the dimension of the meaning matrix, calculates the product of the filter function and each vector in the meaning matrix to obtain a single deep convolution vector, and concatenates all single deep convolution vectors of the same meaning matrix to obtain the deep convolution matrix. The residual convolution module 204 sums the deep convolution matrix and the meaning matrix to obtain the residual convolution matrix. To enable the higher-level self-attention layers to obtain information from the word vectors, this layer uses a residual method. This method can be seen as using deep separable convolution to compensate for the lack of information in the word vectors, thereby fitting the residuals. Simultaneously, during training, the residuals can also alleviate the gradient vanishing problem to some extent.

[0018] Multi-head attention module 3 is used to construct a multi-head attention model and calculate the multi-head output matrix from the residual convolution matrix through the multi-head attention model. After extracting local features from the user's question and answer statements using the residual convolution matrix, the multi-head attention model needs to be constructed. The specific implementation steps are as follows: The weight module 301 is used to obtain the number M of heads in the multi-head attention model through a manually preset method, and to determine M sets of weight transfer matrices based on the number M heads to map the residual convolution matrix to three different spaces. Each set contains three weight transfer matrices corresponding to different spaces. For example, if there are M sets of weight transfer matrices, then the Mth set of weight transfer matrices is A. M B M and C M Multi-head attention is a common algorithm for acquiring matrix features. Its training and learning process, as well as its construction steps, will not be elaborated here. Spatial module 302 is used to select a set of weight transfer matrices from M sets of weight transfer matrices, and multiply each weight transfer matrix in the selected set by the residual convolution matrix to calculate the first spatial matrix, the second spatial matrix, and the third spatial matrix. Repeated computation module 303 is used to repeatedly calculate the spatial matrices until all weight transfer matrices have been multiplied by the residual convolution matrix to obtain multiple spatial matrices. Concatenation module 304 is used to concatenate the multiple spatial matrices to obtain the multi-head output matrix. In existing technologies, self-attention mechanisms often connect two forward fully connected layers after obtaining multiple spatial matrices. This structure can be seen as a parameter-based self-attention mechanism; however, this structure adds a large number of training parameters to the model, making it difficult for the model to converge, thus preventing the model from achieving the desired effect. While using residual structures can mitigate the vanishing gradient problem to some extent, the limited number of trainable parameters above this layer makes it less prone to gradient vanishing. Therefore, multi-head attention models do not employ residual structures. Furthermore, omitting residual structures removes the parameter size limitation imposed by the input dimension. Increasing dimensionality allows the residual convolution matrix to be mapped to a higher-dimensional space, enabling the model to fit more complex functional relationships and achieve stronger expressiveness. From an overall perspective, multi-head attention models also play a crucial role, providing the prerequisite for accurate feature extraction in subsequent processing steps and improving the accuracy of natural language understanding.

[0019] The question-answering detection module 4 is used to construct a question-answering self-attention model based on the multi-head output matrix, and to perform similarity difference detection on the question matrix and answer matrix in the multi-head output matrix using the question-answering self-attention model. Based on the detection results, the correct answer set is obtained. In the results of the multi-head output matrix, existing technologies generally directly perform average pooling on the question portion to obtain the vector representation of the question. However, there is an inseparable relationship between questions and answers, and the determination of answers cannot ignore their similarity to the questions. To find answers closely related to the questions, the specific steps to solve this problem are as follows: The partitioning module 401 is used to partition the multi-head output matrix into a question matrix and an answer matrix based on the user interaction's question-and-answer statements, and extract the column vectors of the question matrix and answer matrix respectively to form sub-question vectors and sub-answer vectors. In the user interaction's question-and-answer statements, the length of the question is generally shorter, while the length of the answer is much longer than the length of the question. Therefore, this method can distinguish between the question and answer statements in the user interaction's question-and-answer statements, and can also determine the multi-head output matrix of the user's question and the multi-head output matrix of the answer through reverse tracing. The model building module 402 is used to build a question-and-answer self-attention model and calculate the correlation similarity method respectively. The system comprises: a question transition matrix and an answer transition matrix; an attention weight module 403, used to calculate attention weights based on the question and answer transition matrices. The calculation method involves multiplying the question transition matrix by a sub-question vector, adding the answer transition matrix multiplied by the sub-answer vector to obtain the question-answer vector, calculating the arithmetic mean of the question and answer transition matrices, and then multiplying the question-answer vectors by the arithmetic mean to obtain the question attention weight; and a question ratio module 404, used to determine an exponential function through a pre-defined method, calculate the question ratio value based on the exponential function, calculate the similarity difference based on the question ratio value, and perform similarity calculations. The similarity difference detection module (4041) obtains the correct answer set based on the detection results. The question-answer attention module (4041) multiplies the question proportion by each sub-answer vector and sums the results to obtain the question-answer attention value between the question and answer. The average pooling module (4042) performs average pooling on the question matrix to obtain the question attention value. The similarity difference module (4043) calculates the question-answer similarity using cosine similarity based on the question attention value and the question-answer attention value, and calculates the similarity difference by subtracting the question-answer similarity value from the similarity value. The threshold module (4044) sets a corresponding detection threshold for the similarity difference, representing the detection standard for whether the question and answer have a high degree of similarity. The detection threshold can be set according to actual needs. In this embodiment, an example is given: the detection threshold is set to 0.15. If the question-answer similarity is 0.8, 1 minus 0.8 equals 0.2. 0.2 is greater than 0.15, indicating a large difference in similarity and low similarity. If the question-answer similarity is 0.95, 1 minus 0.95 equals 0.05. 0.05 is less than 0.15, indicating a small difference in similarity and high similarity. The correct answer module 4045 is used to form correct question-answer groups based on the detection threshold, combining highly similar question attention values ​​and question-answer attention values, and then forming a correct answer set from the corresponding answers. Using cosine similarity as the similarity metric function avoids adding computationally complex operators like exponentials and logarithms to the model's calculations, while also avoiding additional parameters, thus reducing model complexity and significantly improving computational efficiency.

[0020] The sentiment recognition module 5 performs multiple pooling operations on the question matrix and generates trained feature extraction parameters using a deep convolutional neural network. Features are extracted from the question matrix based on these parameters, and then the sentiment of the question matrix is ​​determined using a fully connected layer and a softmax function to obtain the sentiment question matrix. Existing technologies generally use deep learning networks for sentiment recognition; however, training deep learning networks often requires a large amount of data and parameters, and overfitting is prone to occur on small sentiment datasets. To address this issue, the specific implementation steps are as follows: The initial pooling module 501 is used to perform a first pooling of the problem matrix through a pooling layer to obtain the first convolutional block. In this embodiment, an example is given: the problem matrix is ​​used as input and passed through a 3x3 convolution kernel with 3 channels and a stride of 1, convolved twice, resulting in an output size of 224×224×64. After passing through a max pooling layer with a 2x2 kernel and a stride of 2, the image size is halved, and the pooled size becomes 112×112×64, which is the first convolutional block of the improved processing model. The second pooling module 502 is used to perform a second pooling of the first convolutional block to obtain the second convolutional block. The steps of the second pooling are: passing the first convolutional block through a 128-channel pooling layer... The second convolutional block is formed by two 3×3 convolutional kernels, followed by ReLU activation, resulting in a size of 112×112×128. It then undergoes max pooling, reducing the size to 56×56×128. A third pooling module 503 performs a third pooling operation on this second convolutional block to obtain the third convolutional block. The third pooling process involves the second convolutional block being processed by 256 3×3 convolutional kernels, resulting in a size of 56×56×256. After max pooling, the size becomes 28×28. The third convolutional block is 8×256. The fourth pooling module 504 performs a fourth pooling operation on the third convolutional block to obtain the fourth convolutional block. The fourth pooling process involves: the input third convolutional block is processed by 512 3×3 convolutional kernels, resulting in three convolutions, changing the size to 28×28×512. After max pooling, the size becomes 14×14×512, forming the fourth convolutional block. The parameter module 505 pools the fourth convolutional block to obtain the fifth convolutional block, generating the training data using a deep convolutional neural network. Good feature extraction parameters; the fifth pooling step is as follows: the fourth convolutional block is taken as input and passed through 512 3×3 convolutional kernels, convolved three times, the size becomes 14×14×512, after passing through the max pooling layer, the size becomes 7×7×512, this is the fifth convolutional block; feature module 506 is used to extract features from the question matrix through the fifth convolutional block and feature extraction parameters to obtain question features, and the normalized probability is obtained by transforming the question features through a fully connected layer and a softmax function; sentiment matrix module 507 is used to identify the sentiment of the question matrix according to the normalized probability and obtain the sentiment question matrix; the convolutional block is used as a feature extractor to extract features from the question matrix, which includes parameter transfer of the shallow network and feature extraction of the sentiment features in the question matrix by the deep network. The extracted features include both the general features extracted from the pooling model and the representative features of the speech sentiment dataset. Then, a 1024-dimensional fully connected layer and a 7-dimensional fully connected layer representing the output category are connected, and then passed through a softmax function. The function is transformed into normalized probabilities representing the categories, and finally, the sentiment category is determined based on the normalized probabilities. The specific steps of sentiment judgment are: obtaining the 7-dimensional normalized probabilities. ; in, Let's define the emotions as follows: p1 represents excitement, indicating a high level of anticipation for the outcome; p2 represents happiness, indicating strong acceptance of the outcome and a praising attitude; p3 represents calmness, using calm or rational language, without revealing any positive or negative emotions; p4 represents sadness, using relatively negative language and expressing pessimism; p5 represents anger, using relatively extreme words and punctuation that carries emotional connotations; and p6 represents fear, using many rhetorical questions and hypothetical statements, indicating a negative expectation of an outcome that has not yet occurred. In this paper, p7 represents the emotion of disgust, which is the type of expression that uses explicit negation or negative words to describe the result. The seven normalized probabilities are sorted in ascending order, and the emotion corresponding to the highest normalized probability is taken as the emotion of the question matrix. For example, p5 has the highest normalized probability among the seven, so the emotion corresponding to p5 is anger. Existing technologies generally use CNNs for convolutional pooling, but CNNs usually have a large number of parameters, and their commonly used training data requires tens of thousands or even more data points. However, most speech emotion databases are small datasets with only a few hundred data points. Training in a CNN network is prone to overfitting. Therefore, this invention adopts an algorithm for parameter transfer by training the model through multiple pooling. The parameters trained by the deep convolutional neural network are transferred. The knowledge learned by the shallow network is the general feature of the image data. As the number of network layers increases, the knowledge learned by the deeper network layers is more characteristic of the dataset itself. Therefore, the parameters of the general features learned by the shallow network can be transferred and reused on the speech emotion dataset.

[0021] The sentiment response module 6 is used to repeatedly process the correct response set as input through multiple pooling operations, fully connected layers, and the softmax function to obtain a sentiment response set composed of multiple sentiment response matrices. Since the steps are exactly the same regardless of the input, they will not be elaborated further here. Sentiment recognition can grasp the specific sentiment of the questions and answers. However, since the answers all come from the system, sentiment detection of the answers is not the focus, and will not be elaborated in detail here.

[0022] The language interaction module 7 is used to calculate the mixed feature values ​​of the sentiment question matrix and the sentiment response set, and to obtain a fusion vector based on the mixed feature values. This fusion vector is then input into the initially constructed classification neural network model to obtain class probability values. The loss value S calculated based on the class probability values ​​yields the optimized natural language interaction system. After multiple rounds of interaction with the question-answering system, users may sometimes provide inconsistent descriptions of something, which will affect the system's judgment and prevent it from providing an accurate answer. Therefore, the question-answering system needs to have the ability to determine whether the user's input is contradictory. When the user's description is contradictory, the system should be able to provide timely feedback. To solve this problem, the specific implementation steps are as follows: The semantic encoding module 701 is used to construct the encoding matrix from the sentiment question matrix and the sentiment response set according to the order of the user's interactive question and answer statements. An LSTM encoder is then used to perform semantic encoding on the encoding matrix to obtain the semantic encoding matrix. The LSTM encoder is an encoder composed of an LSTM model that can encode the sentiment question matrix and sentiment response set containing sentiment, facilitating subsequent computations; details are omitted here. The hybrid feature module 702 is used to extract the maximum value of the vector in the semantic encoding matrix as the first feature value. The sentiment question matrix and sentiment response set are input into a multi-head attention model to obtain multi-dimensional feature values. The first feature value and the multi-dimensional feature values ​​are then processed... Row concatenation yields mixed feature values; max pooling aims to extract the most salient features, extracting the most important features from each feature extractor. However, max pooling only extracts the most representative features from each dimension of the entire sentence, discarding less important features. The resulting vector is not sufficiently expressive of the sentence. Therefore, a multidimensional attention model is used to calculate a score for each feature of each word. This model selects the features that best describe the specific meaning of a word in any given context and includes this information in the sentence encoding output. Concatenating two features can compensate for each other. To improve the accuracy of answer responses during repeated, multi-round question-and-answer sessions, the fusion module 703 is used to randomly select two features from multiple mixed features as the first and second features. The absolute values ​​of the first and second features are obtained by subtracting each feature element by element, and then the product values ​​are obtained by multiplying each feature element by element. The first feature, second feature, absolute value, and product value are then concatenated to obtain the fusion vector. Since the mixed features, first feature, and second feature are all vectors containing multiple elements, operations can be performed on the elements of the vectors. However, the first and second features often have high similarity, so to differentiate between two sentences... To improve accuracy and improve classification, sentence vector fusion is required. Category module 704 constructs a classification neural network model containing a feedforward neural network layer, an output layer, and a classification layer. The fused vector is used as input, and the class probability values ​​are obtained through these layers. The activation function of the feedforward neural network layer is ReLU, the output layer is a linear layer with the same number of neurons as the total number of classes in the dataset, and the activation function of the classification layer is softmax. Loss module 705 obtains the actual class z of the fused vector and calculates the loss value S, which measures the model loss. The calculation formula is as follows:

[0023] in, The actual class probability value of the fused vector is represented; the formula for the loss value S is a simplified formula because the fused vector itself is an encoded vector, so only the true class probability value needs to be calculated to obtain the loss value S; the model optimization module 706 is used to set a threshold for the loss value S based on empirical methods. If the loss value S is greater than or equal to the threshold, the accuracy of the model is low. The model can be optimized by backpropagation and stochastic gradient descent optimizer until the loss value S is less than the threshold, and a natural language interaction system is obtained.

[0024] This invention employs a hybrid pooling method combining max pooling and multidimensional attention for weighted averaging of the sentence semantic encoding matrix to obtain the vector representation of the sentence. Finally, a multi-layer feedforward neural network is used to obtain the final classification result. Experimental results show that, compared with other deep learning-based natural language inference models, this model can more accurately classify the relationships of sentences based on their semantics. The use of the Softmax function to output the probability distribution not only provides the final classification result but also the probability of each category, making the model's prediction results more interpretable and reliable. The combination of the cross-entropy loss function and Softmax provides the model with a very clear and efficient optimization direction, which can directly maximize the log probability of the correct category, with a clear gradient and fast convergence speed.

[0025] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0026] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0027] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A natural language interactive intelligent customer service system, characterized in that, The system includes: The standardization module (1) is used to collect raw data from the database of the intelligent customer service system and perform text standardization processing on the raw data to obtain standard data. The raw data includes text data, voice data and image data. The word vector module (2) is used to convert standard data into word vectors through word embedding, integrate word vectors into a meaning matrix based on the original data, and perform residual convolution operation on the meaning matrix to obtain a residual convolution matrix; The multi-head attention module (3) is used to construct a multi-head attention model and calculate the multi-head output matrix by passing the residual convolution matrix through the multi-head attention model. The question-answering detection module (4) is used to construct a question-answering self-attention model based on the multi-head output matrix, and to perform similarity difference detection on the question matrix and answer matrix in the multi-head output matrix through the question-answering self-attention model, and to obtain the correct answer set based on the detection results; The emotion recognition module (5) is used to perform multiple pooling operations on the question matrix and generate feature extraction parameters through a deep convolutional neural network. Based on the feature extraction parameters, the question matrix is ​​used to extract features. Then, the emotion of the question matrix is ​​determined through a fully connected layer and a softmax function to obtain the emotion question matrix. The sentiment response module (6) is used to take the correct response set as input and obtain a sentiment response set consisting of multiple sentiment response matrices through multiple pooling operations, fully connected layers and softmax function; The language interaction module (7) is used to calculate the mixed feature values ​​of the emotion question matrix and the emotion response set, and to obtain the fusion vector based on the mixed feature values. The fusion vector is then input into the initially constructed classification neural network model to obtain the category probability value. The loss value S calculated based on the category probability value is used to obtain the optimized natural language interaction system.

2. The natural language interactive intelligent customer service system according to claim 1, characterized in that: The specification module (1) includes: The deduplication module (101) is used to remove special characters from text data using regular expressions and to quickly deduplicat text data by embedding vector similarity to obtain basic text data. The audio conversion module (102) is used to filter the audio in the original data by calculating short-time energy to obtain filtered data, and convert the filtered data into audio text data by ASR speech recognition. The image conversion module (103) is used to remove blurry image data by sharpness detection to obtain clear image data, and to extract image text data by OCR technology from the clear image data; A multimodal module (104) is used to integrate basic text data, audio text data, and image text data into multimodal data; The multimodal fusion module (105) is used to align multimodal data on a unified time axis based on the timestamp method to obtain aligned multimodal data, and to fuse the aligned multimodal data using the feature fusion method to obtain fused data; The standard module (106) is used to input the fused data into the T5 language model to correct typos and obtain corrected data. The corrected data is then normalized according to the standard terms in the interactive database to obtain standard data. Implicit context labels are then inserted into the standard data to obtain standard data.

3. The natural language interactive intelligent customer service system according to claim 1, characterized in that: The word vector module (2) includes: The conversion module (201) is used to first convert the standard data into preliminary word vectors through one-hot encoding, and then convert the preliminary word vectors into word vectors through word embedding. The semantic matrix module (202) is used to combine word vectors into a semantic matrix based on the question and answer statements of user interaction in the original data. The deep convolution module (203) is used to determine the filter function based on the dimension of the meaning matrix, calculate the product of the filter function and each vector in the meaning matrix to obtain a single deep convolution vector, and concatenate all the single deep convolution vectors of the same meaning matrix to obtain a deep convolution matrix. The residual convolution module (204) is used to sum the depth convolution matrix and the meaning matrix to obtain the residual convolution matrix.

4. The natural language interactive intelligent customer service system according to claim 3, characterized in that: The word vectors are used to assign a low-dimensional vector to each word in the question-and-answer statements of user interaction in the original data.

5. The natural language interactive intelligent customer service system according to claim 1, characterized in that: The multi-head attention module (3) includes: The weight module (301) is used to obtain the number of heads M of the multi-head attention model. Based on the number of heads M, M sets of weight transfer matrices are determined to map the residual convolution matrix to three different spaces. Each set contains three weight transfer matrices corresponding to different spaces. The spatial module (302) is used to select a set of weight transfer matrices from the M sets of weight transfer matrices, and to multiply each weight transfer matrix in the selected set with the residual convolution matrix to calculate the first spatial matrix, the second spatial matrix and the third spatial matrix respectively; The repeated computation module (303) is used to repeatedly compute the spatial matrix until all weight transfer matrices are multiplied by the residual convolution matrix to obtain multiple spatial matrices; The splicing module (304) is used to splice multiple spatial matrices to obtain a multi-head output matrix.

6. The natural language interactive intelligent customer service system according to claim 1, characterized in that: The question-and-answer detection module (4) includes: The partitioning module (401) is used to divide the multi-head output matrix into a question matrix and an answer matrix according to the question and answer statements of the user interaction, and extract the column vectors of the question matrix and the answer matrix respectively to form a sub-question vector and a sub-answer vector; The model building module (402) is used to build a question-answering self-attention model and calculate the question transition matrix of the question matrix and the answer transition matrix of the answer matrix respectively by the association similarity method; The attention weight module (403) is used to calculate the attention weight based on the question transition matrix and the answer transition matrix. The calculation method is to multiply the question transition matrix by the sub-question vector, add the answer transition matrix multiplied by the sub-answer vector to obtain the question-answer vector, calculate the arithmetic mean matrix of the question transition matrix and the answer transition matrix, and use the dot product of the question-answer vectors to obtain the question attention weight. The question proportion module (404) is used to determine an exponential function and calculate the question proportion value based on the exponential function. The correct answer set is obtained by using the similarity difference method based on the question proportion value.

7. The natural language interactive intelligent customer service system according to claim 6, characterized in that: The problem scaling module (404) includes: The question-answering attention module (4041) is used to multiply the question proportion value by each sub-answer vector and sum them to obtain the question-answering attention value between the answer and the question. The average pooling module (4042) is used to perform average pooling on the problem matrix to obtain the problem attention value; The similarity difference module (4043) is used to calculate the question-answer similarity based on the question attention value and the question-answer attention value using the cosine similarity method, and to calculate the similarity difference by subtracting the question-answer similarity from one. The threshold module (4044) is used to set the corresponding detection threshold for the similarity difference; The correct answer module (4045) is used to form a correct question and answer group based on the question attention value and the question and answer attention value according to the detection threshold, and to form a correct answer set by combining the answer sets corresponding to the question and answer attention values.

8. The natural language interactive intelligent customer service system according to claim 1, characterized in that: The emotion recognition module (5) includes: The initial pooling module (501) is used to perform the first pooling of the problem matrix through the pooling layer and obtain the first convolutional block; The secondary pooling module (502) is used to perform a second pooling on the first convolutional block to obtain a second convolutional block; The third pooling module (503) is used to perform a third pooling on the second convolutional block to obtain a third convolutional block; The fourth pooling module (504) is used to perform a fourth pooling on the third convolutional block to obtain a fourth convolutional block; The parameter module (505) is used to pool the fourth convolutional block to obtain the fifth convolutional block and generate trained feature extraction parameters through a deep convolutional neural network. The feature module (506) is used to extract features from the problem matrix through the fifth convolutional block and feature extraction parameters to obtain problem features, and to obtain normalized probabilities through a fully connected layer and a softmax function based on the problem features; The sentiment matrix module (507) is used to identify the sentiment of the question matrix based on the normalized probability and obtain the sentiment question matrix.

9. The natural language interactive intelligent customer service system according to claim 1, characterized in that: The language interaction module (7) includes: The sentence semantic encoding module (701) is used to construct the matrix to be encoded from the sentiment question matrix and the sentiment answer set according to the order of the question and answer statements of the user interaction, and to obtain the sentence semantic encoding matrix by performing word semantic encoding on the matrix to be encoded through the LSTM encoder. The hybrid feature module (702) is used to extract the maximum value of the vector in the semantic encoding matrix as the first feature value, input the sentiment question matrix and sentiment answer set into the multi-head attention model to obtain multi-dimensional feature values, and concatenate the first feature value and multi-dimensional feature values ​​to obtain the hybrid feature value; The fusion module (703) is used to randomly select two features from multiple mixed features as the first feature and the second feature, subtract the first feature and the second feature element by element to obtain their absolute values, multiply the first feature and the second feature element by element to obtain the multiplicative value, and concatenate the first feature, the second feature, the absolute value and the multiplicative value to obtain the fusion vector. The category module (704) is used to construct a classification neural network model containing a feedforward neural network layer, an output layer, and a classification layer. The fused vector is used as input, and the category probability value is obtained through the feedforward neural network layer, the output layer, and the classification layer. The loss module (705) is used to obtain the actual class z of the fusion vector and calculate the loss value S used to measure the loss of the model; The model optimization module (706) is used to set a threshold for the loss value S. If the loss value S is greater than or equal to the threshold, the model is optimized by backpropagation and stochastic gradient descent optimizer until the loss value S is less than the threshold, and a natural language interaction system is obtained.

Images