A question and answer response method, system and related device for a customer service robot

Abstract

The present application provides a method for answering questions of a customer service robot, including: obtaining a user question; using a self-supervised neural network to determine the representation information of the user question; querying the Euclidean distance between the representation information and the existing representation information in the knowledge base; taking the existing representation information with the smallest Euclidean distance as the target representation information; and returning an answer to the user question according to the target representation information. The present application uses a self-supervised neural network to calculate the representation information of the user question, and then compares the representation information between questions, so that the granularity control of text semantic expression is more delicate, the semantic expression is more reasonable, the semantic rationality of the representation is improved, the recognition accuracy of the user question can be improved, and further the accuracy of question response can be improved. The present application also provides a question-answering response system, a computer-readable storage medium and an electronic device of a customer service robot, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and particularly relates to a question and answer response method, system and related devices for a customer service robot. Background Art

[0002] With the continuous development and in-depth of intelligent technology, the application of intelligent customer service robots has become more and more common. For a long time, researchers' designs for calculating text sentence representations have stayed on supervised neural networks. Supervised learning relies on a large amount of manually labeled data to train the model. The model's prediction and the real labels of the data are used for backpropagation (calculating gradients and updating parameters). Through continuous learning, the ability to recognize new samples can be finally obtained. This design method seriously depends on the quantity of labeled data. If the data volume is too small and the model is too large, overfitting will occur, resulting in the model performing well during the training process but having poor results in the test data, thus preventing the model from developing in a deeper and wider direction. Summary of the Invention

[0003] The purpose of the present application is to provide a question and answer response method, question and answer response system, computer-readable storage medium and electronic device for a customer service robot, which can improve the accuracy of the question and answer response of the customer service robot.

[0004] To solve the above technical problems, the present application provides a question and answer response method for a customer service robot, and the specific technical solution is as follows:

[0005] Obtain a user's question;

[0006] Use a self-supervised neural network to determine the representation information of the user's question;

[0007] Query the Euclidean distance between the representation information and the existing representation information in the knowledge base;

[0008] Take the existing representation information with the smallest Euclidean distance as the target representation information;

[0009] Return the answer to the user's question according to the target representation information.

[0010] Optionally, before querying the Euclidean distance between the representation information and the existing representation information in the knowledge base, it further includes:

[0011] Obtain a preset text database;

[0012] Use the natural language processing tool nltk to perform sentence tokenization on the text in the preset text data to obtain sample data composed of tokens;

[0013] Take N attention probability parameters in the dropout layer of the self-supervised neural network, and use the self-supervised neural network to calculate the sample data to obtain N representation information corresponding to the same sample; N is a preset quantity;

[0014] Construct the knowledge base according to the N representation information.

[0015] Optionally, it further includes:

[0016] Adopt the model design structure of the BERT-like model as the model structure of the self-supervised neural network model;

[0017] Calculate the model parameters and determine the self-supervised neural network model; the self-supervised neural network includes 1 fully connected layer, 6 attention mechanism models and the dropout layer.

[0018] Optionally, calculating the model parameters and determining the self-supervised neural network model includes:

[0019] Use the redundancy reduction loss function to calculate the model parameters and determine the self-supervised neural network model according to the model parameters;

[0020] The redundancy reduction loss function is

[0021] where λ is a positive constant used to balance the ratio between the first term and the second term in the redundancy reduction loss function in, is the cross-correlation matrix, and the cross-correlation matrix is calculated from the positive sample set at the corresponding positions obtained by the same sample through the self-supervised neural network respectively, and The dimension of is the same as the dimension of the representation information output by the self-supervised neural network.

[0022] Optionally, before using the redundancy reduction loss function to calculate the model parameters, it further includes:

[0023] Determine the cross-correlation matrix through the cross-correlation matrix calculation formula;

[0024] The cross-correlation matrix calculation formula is

[0025] where b represents each original in the same batch of samples, i, j indicate different dimensions of the representation vector, and h is the representation information.

[0026] Optionally, after using the redundancy reduction loss function to calculate the model parameters and before determining the self-supervised neural network model according to the model parameters, it further includes:

[0027] Use the preset optimizer to minimize the loss of the redundancy reduction loss function;

[0028] Correspondingly, determining the self-supervised neural network model according to the model parameters includes:

[0029] Taking the value when the loss of the redundancy reduction loss function is minimized as the model parameter, and determining the self-supervised neural network model.

[0030] This application also provides a question and answer response system for a customer service robot, including:

[0031] An acquisition module, configured to acquire user questions;

[0032] A characterization calculation module, configured to use a self-supervised neural network to determine the characterization information of the user question;

[0033] A distance calculation module, configured to query the Euclidean distance between the characterization information and the existing characterization information in the knowledge base;

[0034] A target characterization determination module, configured to use the existing characterization information with the smallest Euclidean distance as the target characterization information;

[0035] A response module, configured to return an answer to the user question according to the target characterization information.

[0036] Optionally, it further includes:

[0037] A knowledge base construction module, configured to acquire a preset text database; use the natural language processing tool nltk to perform sentence tokenization on the text in the preset text data to obtain sample data composed of tokens; take N attention probability parameters in the dropout layer of the self-supervised neural network, and use the self-supervised neural network to calculate the sample data to obtain N characterization information corresponding to the same sample; N is a preset quantity; construct the knowledge base according to the N characterization information.

[0038] This application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0039] This application also provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the method described above are implemented.

[0040] This application provides a question and answer response method for a customer service robot, including: acquiring user questions; using a self-supervised neural network to determine the characterization information of the user question; querying the Euclidean distance between the characterization information and the existing characterization information in the knowledge base; using the existing characterization information with the smallest Euclidean distance as the target characterization information; returning an answer to the user question according to the target characterization information.

[0041] This application uses a self-supervised neural network to calculate the representation information of user questions, and then compares the representation information between questions, making the control of the text semantic expression granularity more delicate, the semantic expression more reasonable, improving the semantic rationality of the representation, being able to improve the recognition accuracy of user questions, and further improving the accuracy of question responses.

[0042] This application also provides a question and answer response system, a computer-readable storage medium, and an electronic device for a customer service robot, which have the above beneficial effects and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0044] Figure 1 It is a flowchart of a question and answer response method for a customer service robot provided by an embodiment of the present application;

[0045] Figure 2 It is a schematic structural diagram of a question and answer response system for a customer service robot provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0047] Please refer to Figure 1 , Figure 1 It is a flowchart of a question and answer response method for a customer service robot provided by an embodiment of the present application. The method includes:

[0048] S101: Obtain a user question;

[0049] The purpose of this step is to obtain a user question. Here, there is no limitation on how to obtain the user question, and the user question can be obtained through text, voice, or user input, etc. Of course, there are no limitations on the language and method used for the user question, etc. However, it should be noted that the user question should be recognizable content.

[0050] S102: Determine the representation information of the user question using a self-supervised neural network;

[0051] In this step, a self-supervised neural network is used to calculate the representation information of the user question. The so-called representation information refers to the part of the semantics with the optimal meaning expressed in the user question, such as noun information, question type, etc. in the user question. The self-supervised neural network in this step is a trained self-supervised neural network. In this embodiment, it is default that the self-supervised neural network has been generated before this step is executed, but the specific generation process of the self-supervised neural network is not limited.

[0052] S103: Query the Euclidean distance between the representation information and the existing representation information in the knowledge base;

[0053] The purpose of this step is to calculate the Euclidean distance between the representation information and the existing representation information in the knowledge base. Of course, when calculating the Euclidean distance, the Euclidean distance can be calculated for all the existing representation information in the knowledge base, or the representation information can be classified first, and then calculated among the existing representation information under the corresponding classification in the knowledge base, so as to reduce the calculation amount of the Euclidean distance and improve the response speed.

[0054] S104: Take the existing representation information with the smallest Euclidean distance as the target representation information;

[0055] S105: Return the answer to the user question according to the target representation information.

[0056] Take the existing representation information with the smallest Euclidean distance as the representation information closest to the user question, and thus return the question answer corresponding to the target representation information in the knowledge base.

[0057] When executing step S105, it is also possible to optimize the response reply based on the question answer corresponding to the target representation information, such as adding polite words, etc.

[0058] In the embodiment of this application, a self-supervised neural network is used to calculate the representation information of the user question, and then the representation information between questions is compared, so that the control of the text semantic expression granularity is more delicate, the semantic expression is more reasonable, the semantic rationality of the representation is improved, the recognition accuracy of the user question can be improved, and thus the accuracy of the question response can be improved.

[0059] The following explains how to obtain the self-supervised neural network and how to construct the knowledge base. Constructing the knowledge base includes the following steps:

[0060] The first step: Obtain a preset text database;

[0061] The second step: Use the natural language processing tool nltk to perform sentence tokenization on the text in the preset text data to obtain sample data composed of tokens;

[0062] Step 3: Take N attention probability parameters in the dropout layer of the self-supervised neural network, and use the self-supervised neural network to calculate the sample data to obtain N representation information corresponding to the same sample; N is a preset quantity.

[0063] Step 4: Construct the knowledge base according to the N representation information.

[0064] Obviously, in the whole process of intelligent customer service question answering, obtaining the semantic representation of sentences is the most core link. The rationality of sentence semantic representation directly determines the subsequent ability to calculate the similarity with questions in the knowledge base. Therefore, the representation of sentence semantics needs to be reasonable, measurable for similar texts, and not a trivial solution that makes the distance between the input and the texts in the knowledge base equal. This application constructs a knowledge base for storing common and reasonable representations, which can be used in the question understanding part of the intelligent customer service robot.

[0065] In addition, this application designs a self-supervised neural network based on a redundancy reduction loss function to represent sentence semantics. One of the advantages of the self-supervised neural network is that it uses unlabeled data for training and uses the designed proxy task to supervise the training of model parameters. Therefore, there are no special requirements for the training text data, and data crawled from the Internet or existing public datasets can be used. For example, the publicly available wikipedia web page content data, abbreviated as wiki_dataset, can be used.

[0066] After that, use the nltk language model to first split the text into sentences and then into words to obtain several words. Convert the words into tokens according to the word list. The position information of each word in the token is designed as [0, 1, 2,...] to represent the position information of the word itself. It should be noted that the word list can be set by those skilled in the art themselves and is only used to mark the digital number information corresponding to each word.

[0067] Finally, through the calculation of the self-supervised neural network f θ take different attention probabilities in the dropout layer to obtain different representations of the same sample where z is a random mask in the dropout layer, that is, the attention probability. Since the representations obtained from different masks are all from the same sample, the different representations H obtained can be positive samples of each other.

[0068] This application adopts a model design structure similar to BERT as the self-supervised neural network f θ, thereafter, the model parameters are calculated and the self-supervised neural network model is determined. The self-supervised neural network includes 1 fully connected layer, 6 attention mechanism models and the dropout layer, and the dropout layer is used to obtain different representations of the same sample. In this application, a symmetric network structure is used to calculate sentence similarity, and only sample positive examples are used as inputs of the self-supervised neural network, thus avoiding unnecessary overhead caused by maintaining negative examples. Therefore, how to avoid the self-supervised neural network that only uses positive examples to converge to a trivial solution is the most important part of building a self-supervised neural network.

[0069] To this end, this application provides a redundancy reduction loss function As shown in formula (1):

[0070]

[0071] Among them, λ is the loss function used to balance redundancy reduction The positive constant of the ratio between the first and second terms in , is a cross-correlation matrix, which is calculated by the positive sample sets of corresponding positions obtained by the self-supervised neural network for the same samples, and The dimension of is the same as the dimension of the representation information output by the self-supervised neural network.

[0072] Before using the redundancy reduction loss function to calculate the model parameters, it is necessary to first determine the cross-correlation matrix using the cross-correlation matrix calculation formula;

[0073] The cross-correlation matrix calculation formula is:

[0074] Among them, b represents each original in the same batch of samples, i and j indicate different dimensions of the representation vector, and h is the representation information.

[0075] According to the above formula, is the self-supervised neural network f θ Output Characterization A square matrix of the same dimension, with values between -1 and 1.

[0076] Intuitively, the redundancy reduction loss function The identity term in constrains the diagonal elements of the cross-correlation matrix to be equal to 1, so that the text representation remains unchanged in different augmentation results, retaining the most semantic core content in the text. At the same time, the redundancy reduction term constrains the off-diagonal elements of the cross-correlation matrix to be equal to 0, and decorrelates the different vector components of the text representation. This decorrelation reduces the redundancy between output units, thereby reducing the non-redundant information about the sample contained in the output representation.

[0077] In addition, after calculating the model parameters using the redundancy reduction loss function and before determining the self-supervised neural network model based on the model parameters, the preset optimizer can be used to minimize the loss of the redundancy reduction loss function. Then, when determining the self-supervised neural network model, the value at which the loss of the redundancy reduction loss function is minimized can be used as the model parameters, and the self-supervised neural network model can be determined.

[0078] This application obtains the self-supervised neural network model parameters by minimizing the redundancy reduction loss function and obtains a reasonable semantic representation of the input text through the model parameters to support the subsequent process of similarity calculation.

[0079] Specifically, during training, if a sentence set is given where the sample x i and the sample are positively correlated samples with semantic associations, and this application only involves positively correlated samples, there is no need to construct a queue or increase the batch size value during network training to increase the number of negative samples. h i and are the representations of the sample x i and the sample , usually the value output by the penultimate hidden layer after fixing the network parameters after training is stable. Note: After the neural network training is completed, fix the network parameters, and for any input sentence x i its semantic representation h i can be obtained. Further calculating the Euclidean distance between different h i can obtain the semantic similarity of two sentences. Therefore, the quality of the semantic representation h i determines the accuracy of similarity calculation.

[0080] Considering that in the process of deep neural networks, training is carried out in batches, so the above symbols are rewritten here. Given a batch of samples X in the dataset, this batch of samples obtains data X A and X B by adopting different attention probability parameters. Different data augmentation methods, denoted as τ, bring different dimensional forms of performance to the sample X. Since they are all transformed from the same sample, they can be positive samples for each other. After obtaining the augmented data X A and X B , they are fed into the designed neural network f θ to obtain the representations H A and H B of the same sample.

[0081] The following introduces a question and answer response system for a customer service robot provided by an embodiment of the present application. The question and answer response system described below can be mutually corresponding and referred to with a question and answer response method for a customer service robot described above.

[0082] See Figure 2 , Figure 2 which is a schematic structural diagram of a question and answer response system for a customer service robot provided by an embodiment of the present application. The present application also provides a question and answer response system for a customer service robot, including:

[0083] An acquisition module, configured to acquire user questions;

[0084] A characterization calculation module, configured to determine characterization information of the user question by using a self-supervised neural network;

[0085] A distance calculation module, configured to query the Euclidean distance between the characterization information and existing characterization information in a knowledge base;

[0086] A target characterization determination module, configured to use the existing characterization information with the smallest Euclidean distance as target characterization information;

[0087] A response module, configured to return an answer to the user question according to the target characterization information.

[0088] Based on the above embodiments, as a preferred embodiment, it further includes:

[0089] A knowledge base construction module, configured to acquire a preset text database; perform sentence word segmentation on the text in the preset text data by using a natural language processing tool nltk to obtain sample data composed of word segments; take N attention probability parameters in the dropout layer of the self-supervised neural network, and use the self-supervised neural network to calculate the sample data to obtain N characterization information corresponding to the same sample; N is a preset quantity; construct the knowledge base according to the N characterization information.

[0090] Based on the above embodiments, as a preferred embodiment, it further includes:

[0091] A model construction module, configured to adopt a model design structure similar to BERT as the model structure of the self-supervised neural network model; calculate model parameters, and determine the self-supervised neural network model; the self-supervised neural network includes 1 fully connected layer, 6 attention mechanism models, and the dropout layer.

[0092] Based on the above embodiments, as a preferred embodiment, the model construction module includes:

[0093] Calculate the model parameters by using a redundancy reduction loss function, and determine the self-supervised neural network model according to the model parameters;

[0094] The redundancy reduction loss function is

[0095] where λ is a positive constant used to balance the proportion between the first term and the second term in the redundancy reduction loss function and is a cross-correlation matrix, which is calculated from the set of positive samples at corresponding positions obtained by passing the same samples through the self-supervised neural network respectively, and the dimension of is the same as the dimension of the representation information output by the self-supervised neural network. Based on the above embodiments, as a preferred embodiment, it further includes:

[0096] A cross-correlation matrix calculation module for determining the cross-correlation matrix through the cross-correlation matrix calculation formula;

[0097] The cross-correlation matrix calculation formula is

[0098] where b represents each original in the same batch of samples, i and j indicate different dimensions of the representation vector, and h is the representation information.

[0099] Based on the above embodiments, as a preferred embodiment, it further includes:

[0100] An optimization module for minimizing the loss of the redundancy reduction loss function by using a preset optimizer;

[0101] This application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the steps provided by the above embodiments can be implemented. The storage medium may include: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0102] This application also provides an electronic device, which may include a memory and a processor. When the processor calls the computer program stored in the memory, the steps provided by the above embodiments can be implemented. Of course, the electronic device may also include various network interfaces, power supplies and other components.

[0103] In the description of the specification, the embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the system provided by the embodiment, since it corresponds to the method provided by the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0104] ​

[0105] In this text, specific examples are used to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

[0106] It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the said element.

Claims

1. A question and answer response method for a customer service robot, characterized in that, Including: Obtain the user's question; Use a self-supervised neural network to determine the representation information of the user's question; The representation information refers to the content with the optimal meaning in the semantics expressed in the user's question, including: noun information and question type in the user's question; the self-supervised neural network includes 1 fully connected layer, 6 attention mechanism models and a dropout layer, and the dropout layer is used to obtain different representations of the same sample; Query the Euclidean distance between the representation information and the existing representation information in the knowledge base; Take the existing representation information with the smallest Euclidean distance as the target representation information; the knowledge base includes the question answer corresponding to the target representation information; Return the answer to the user's question according to the target representation information and optimize the answer; Among them, before querying the Euclidean distance between the representation information and the existing representation information in the knowledge base, it also includes: Obtain a preset text database; Use the natural language processing tool nltk to tokenize the text in the preset text data to obtain sample data composed of tokens; Take N attention probability parameters in the dropout layer of the self-supervised neural network and use the self-supervised neural network to calculate the preset text data to obtain N representation information corresponding to the same sample; N is a preset quantity; Construct the knowledge base according to the N representation information.

2. The Q&A response method according to claim 1, wherein It also includes: Adopt the model design structure similar to BERT as the model structure of the self-supervised neural network model; Calculate the model parameters and determine the self-supervised neural network model.

3. The Q&A response method according to claim 2, wherein Calculating the model parameters and determining the self-supervised neural network model includes: Use the redundancy reduction loss function to calculate the model parameters and determine the self-supervised neural network model according to the model parameters; The redundant reduction loss function is ; Among them, is a positive constant used to balance the proportion between the first term and the second term in the redundancy reduction loss function ; is a cross-correlation matrix, which is calculated from the set of positive samples at corresponding positions obtained by passing the same samples through a self-supervised neural network respectively, and has the same dimension as the dimension of the representation information output by the self-supervised neural network.

4. The question and answer response method according to claim 3, wherein Before using the redundancy reduction loss function to calculate the model parameters, it also includes: Determine the cross-correlation matrix through the cross-correlation matrix calculation formula; The cross-correlation matrix calculation formula is ; Among them, b represents each original in the same batch of samples, indicating different dimensions of the representation vector, which is the representation information.

5. The Q&A response method according to claim 1, wherein After using the redundancy reduction loss function to calculate the model parameters and before determining the self-supervised neural network model according to the model parameters, it also includes: Use a preset optimizer to minimize the loss of the redundancy reduction loss function; Correspondingly, the determining the self-supervised neural network model according to the model parameters includes: Take the value when the loss of the redundancy reduction loss function is minimized as the model parameters and determine the self-supervised neural network model.

6. A question-and-answer response system for a customer service robot, characterized in that, Including: An acquisition module for obtaining the user's question; A representation calculation module for using a self-supervised neural network to determine the representation information of the user's question; The representation information refers to the content with the optimal meaning in the semantics expressed in the user's question, including: noun information and question type in the user's question; the self-supervised neural network includes 1 fully connected layer, 6 attention mechanism models and a dropout layer, and the dropout layer is used to obtain different representations of the same sample; A distance calculation module for querying the Euclidean distance between the representation information and the existing representation information in the knowledge base; A target representation determination module, configured to use the existing representation information with the smallest Euclidean distance as the target representation information; the knowledge base includes the question answer corresponding to the target representation information; A response module, configured to return the answer to the user question according to the target representation information and optimize the answer; A knowledge base construction module, configured to obtain a preset text database; use the natural language processing tool nltk to perform sentence tokenization on the text in the preset text data to obtain sample data composed of tokens; take N attention probability parameters in the dropout layer of the self-supervised neural network, and use the self-supervised neural network to calculate the sample data to obtain N representation information corresponding to the same sample; N is a preset quantity; construct the knowledge base according to the N representation information.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the question and answer response method of the customer service robot according to any one of claims 1-5 are implemented.

8. An electronic device, characterized in that, It includes a memory and a processor. A computer program is stored in the memory. When the processor calls the computer program in the memory, the steps of the question and answer response method of the customer service robot according to any one of claims 1-5 are implemented.

Images