Question and answer method, electronic device, and program product

Abstract

Embodiments of the present disclosure relate to a question and answer method, an electronic device and a computer program product. The method comprises: determining an answer base associated with a question; determining a restricted Boltzmann machine associated with the answer base, the restricted Boltzmann machine being used to determine a question set that the answer base can answer and a relationship between a question in the question set and the answer base; and determining, using the restricted Boltzmann machine, description information associated with the question for the answer base. Using the technical solution of the present disclosure, the description information for the answer base can be determined at the same time as the answer base associated with the question is determined, and the customer service personnel can be made to have a more thorough understanding of the determined answer base and obtain targeted recommendation information by providing the determined description information to the customer service personnel, thereby improving the user experience of the customer service personnel using the question and answer system.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure generally relate to artificial intelligence technology, and in particular, to a question and answer method, an electronic device and a computer program product, which can be used in the field of customer service. BACKGROUND

[0002] At present, with the development of artificial intelligence technology, computer-aided forms have been widely used in service industries and other industries to help customer service personnel answer user questions. For example, an answer bank for different questions or different types of questions can be pre-stored in a customer service system, and the answer bank can include answers to some common questions or questions that have been raised by users. When a user raises a question, which can also be referred to as a service request (SR), about the use of an application to a customer service personnel through a customer service system, the customer service system can determine an answer bank associated with the question raised by the user based on the question raised by the user, and push the determined answer bank or the identifier of the answer bank to the customer service personnel. After receiving the pushed answer bank or the identifier of the answer bank, the customer service personnel can refer to the answer to the question raised by the user in the answer bank to answer the user. This form of customer service system determining the answer bank according to the user's question belongs to the recommendation function, and can be constructed as a recommendation system. A good recommendation system can be used to guide junior customer service personnel, thereby saving the time spent by junior customer service personnel in asking senior customer service personnel and finding answers.

[0003] However, the traditional recommendation system only provides the answer bank associated with the user's question, and does not provide other useful information or reasonable suggestions, so it is difficult to provide more effective assistance to the customer service personnel using the recommendation system. SUMMARY

[0004] Embodiments of the present disclosure provide a question and answer method, an electronic device and a computer program product

[0005] In a first aspect of the present disclosure, a question and answer method is provided. The method comprises: determining an answer bank associated with a question; determining a restricted Boltzmann machine (RBM) associated with the answer bank, the restricted Boltzmann machine being used to determine a set of questions that can be answered by the answer bank and an association between a question in the set of questions and the answer bank; and determining, using the restricted Boltzmann machine, description information associated with the question for the answer bank.

[0006] In a second aspect of the disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, cause the device to perform actions including determining a bank of answers associated with a question; determining a restricted Boltzmann machine associated with the bank of answers, the restricted Boltzmann machine being used to determine a set of questions that the bank of answers can answer and an association of questions in the set of questions with the bank of answers; and determining, using the restricted Boltzmann machine, explanatory information associated with the question for the bank of answers.

[0007] In a third aspect of the disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer readable medium and comprises machine executable instructions that, when executed, cause a machine to perform any of the steps of the method described according to the first aspect of the disclosure.

[0008] The summary is provided to introduce a selection of concepts, in a simplified form, that are further described below in the detailed description. The summary is not intended to identify key or essential features of embodiments of the disclosure, nor is it intended to limit the scope of embodiments of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other objects, features and advantages of the disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures, and in which:

[0010] Figure 1 A schematic diagram illustrating an example question and answer environment 100 in which devices and / or methods according to embodiments of the disclosure can be implemented is shown;

[0011] Figure 2 A flowchart illustrating a question and answer method 200 according to embodiments of the disclosure is shown;

[0012] Figure 3 A schematic diagram illustrating a restricted Boltzmann machine 300 according to embodiments of the disclosure is shown;

[0013] Figure 4 A schematic diagram illustrating an accuracy comparison 400 according to embodiments of the disclosure is shown; and

[0014] Figure 5 A schematic block diagram illustrating an example device 800 that can be used to implement embodiments of the disclosure is shown.

[0015] In the various drawings, like or corresponding reference numbers identify like or corresponding parts. Detailed Implementation

[0016] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0017] The term "comprising" and its variations as used herein indicate an open-ended inclusion, for example, "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "an example embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0018] As mentioned earlier, a good recommendation system can guide junior customer service staff, saving them the time spent consulting senior staff and searching for answers. However, traditional recommendation systems only provide a database of answers related to user questions, without offering other useful information or reasonable suggestions, thus failing to provide more effective assistance to customer service staff using recommendation systems.

[0019] To at least partially address one or more of the aforementioned problems and other potential issues, embodiments of this disclosure propose a collaborative filtering recommendation method based on a Restricted Boltzmann Machine (RBM). This method uses a set of questions that can be answered by an answer database as positive samples and a set of questions that cannot be answered by the answer database as negative samples. Training associations between the positive and negative samples and the answer database are obtained, and the RBM is trained using the positive and negative samples and the training associations. This trained RBM can then be used to determine the set of questions that the answer database can answer and the associations between the questions in the set and the answer database. The trained RBM can then determine the explanatory information associated with the questions and the answer database. By providing this determined explanatory information to customer service personnel, they can gain a more thorough understanding of the identified answer database and receive targeted recommendations, thereby improving the user experience for customer service personnel using the question-and-answer system.

[0020] Figure 1A schematic block diagram of an example question answering environment 100 in which a question answering method in certain embodiments of the present disclosure can be implemented is shown. According to embodiments of the present disclosure, the question answering environment 100 can be a cloud environment.

[0021] As shown in Figure 1 The question answering environment 100 includes a computing device 110. In the question answering environment 100, a question 120, which can also be referred to as a service request, is provided to the computing device 110 as an input of the computing device 110, for example, including a user's use of a certain application through a customer service system to a customer service personnel. According to embodiments of the present disclosure, the question 120 can include a title and an associated description of a service request filled by the user, where the title can be freely filled by the user or options are provided to the user by the customer service system and selected by the user, and the associated description can be a specific detailed question description of the user for the question 120.

[0022] After receiving the question 120, the computing device 110 can determine an answer base associated with the question, determine a restricted Boltzmann machine associated with the answer base, and determine, using the restricted Boltzmann machine, the explanatory information associated with the question for the answer base.

[0023] It should be understood that the example question answering environment 100 is merely exemplary and not limiting, and it is scalable or scalable. For example, more computing devices 110 can be included in the example environment 100, and more questions 120 can be provided to the computing devices 110 as inputs, so that more users can meet the needs of simultaneously using more computing devices 110 to simultaneously or non-simultaneously obtain recommended answer bases and associated explanatory information for more questions 120.

[0024] The following describes in detail the specific operations performed by the computing device 110 after receiving the question 120 with reference to the question answering method 200 shown in Figure 1 Figure 2 The following describes in detail the specific operations performed by the computing device 110 after receiving the question 120 with reference to the question answering method 200 shown in

[0025] Figure 2 A flowchart of the question answering method 200 according to embodiments of the present disclosure is shown. The question answering method 200 can be implemented by the computing device 110 shown in Figure 1 It should be understood that the question answering method 200 can also include additional steps not shown and / or can omit the steps shown, and the scope of embodiments of the present disclosure is not limited in this regard.

[0026] ​At block 202, the computing device 110 determines an answer base associated with the question 120. According to embodiments of the present disclosure, the question 120 can be issued by a user to a customer service staff through a customer service system. The question 120 can include a title of a service request filled by the user and an associated description, where the title can be freely filled by the user or options are provided to the user by the customer service system and selected by the user, and the associated description can be a specific detailed question description by the user for the question 120.

[0027] According to some embodiments of the present disclosure, the computing device 110 can determine the answer base associated with the question 120 based on at least one of the relevance of the question 120 to questions that can be answered by each of the answer bases and the relevance of the question 120 to the answer base. For example, the computing device 110 can concatenate the words included in the question 120 together, convert all the words to lower case if the words are in a foreign language, and remove all the punctuation marks to form a word sequence corresponding to the question 120. Then, the computing device 110 can determine the relevance between the word sequence corresponding to the question 120 and the questions that can be answered by the answer bases or the answer bases themselves, for example, by calculating the semantic relevance between the word sequence and the questions that can be answered by the answer bases or the answer bases themselves, and further determine the answer base associated with the question 120 among the answer bases.

[0028] At block 204, the computing device 110 can further determine a restricted Boltzmann machine associated with the answer base determined to be associated with the question 120 at block 202. According to embodiments of the present disclosure, the computing device 110 can pre-construct a restricted Boltzmann machine for each answer base. Therefore, after the computing device 110 determines the answer base associated with the question 120, the computing device 110 can further determine the restricted Boltzmann machine associated with the answer base.

[0029] According to embodiments of the present disclosure, the restricted Boltzmann machine can be used to determine a set of questions that can be answered by the answer base and the relevance of the questions in the set of questions to the answer base, where the above-mentioned capabilities of the restricted Boltzmann machine are associated with the training process of the restricted Boltzmann machine.

[0030] A restricted Boltzmann machine is a stochastic generative neural network that can learn a probability distribution through an input data set. The restricted Boltzmann machine can be applied in dimensionality reduction, classification, collaborative filtering, feature learning, and topic modeling, etc. According to different tasks, the restricted Boltzmann machine can be trained using supervised learning or unsupervised learning methods.

[0031] In particular, a restricted Boltzmann machine is a special topology of a Boltzmann machine. The principle of a Boltzmann machine originates from statistical physics and is a modeling approach based on an energy function and capable of describing high-order interactions between variables. The learning algorithm of a Boltzmann machine is complex and is a symmetrically coupled stochastic feedback type binary unit neural network composed of a visible layer and multiple hidden layers. The nodes in the visible layer can be referred to as visible units and the nodes in the hidden layers can be referred to as hidden units. In a Boltzmann machine, a learning model of a stochastic network with a stochastic environment is expressed by visible units and hidden units, and the correlation between units is expressed by weights.

[0032] A restricted Boltzmann machine is a variant of a Boltzmann machine, but the model must be bipartite. The model contains visible units corresponding to input parameters and hidden units corresponding to training results, and each edge must connect a visible unit and a hidden unit. This restriction makes it possible to have a more efficient training algorithm than a general Boltzmann machine, in particular, a gradient-based contrastive divergence algorithm. Therefore, a restricted Boltzmann machine can be used for a deep learning network.

[0033] In a restricted Boltzmann machine, each element in a weight matrix W = (w i，j ) specifies a weight of an edge between a hidden layer unit h j and a visible layer unit v i . In addition, there is a bias a i for each visible layer unit v i and a bias b j for each hidden layer unit h j . Under this definition, the "energy" of a restricted Boltzmann machine configuration, i.e., given values of each unit, (v, h) is defined as:

[0034]

[0035] or expressed in matrix form as follows:

[0036] E(v, h) = -a T v - b T h - h T Wv (2)

[0037] In a general restricted Boltzmann machine, a joint probability distribution between the hidden layer and the visible layer can be given by the following energy function:

[0038]

[0039] where Z is a partition function or a normalization factor, which is defined as e -E(v，h)and, i.e. a normalization constant that makes the probability distribution sum to 1. Similarly, the marginal distribution of the visible layer values can be obtained by summing over all hidden layer configurations:

[0040]

[0041] Since the restricted Boltzmann machine is a bipartite graph, there are no edges within the layers, so the activation of the hidden layer is conditionally independent given the values of the visible layer nodes. Similarly, the activation of the visible layer nodes is conditionally independent given the values of the hidden layer. That is, for m visible layer nodes and n hidden layer nodes, the conditional probability of the visible layer configuration v given the hidden layer configuration h is:

[0042]

[0043] Similarly, the conditional probability of h given v is

[0044]

[0045] The activation probability of a single node is

[0046]

[0047] where σ denotes the logistic function.

[0048] The following will describe the restricted Boltzmann machine 300 used in the embodiments of the present disclosure with specific reference to Figure 3

[0049] Figure 3 A schematic diagram of the restricted Boltzmann machine 300 according to an embodiment of the present disclosure is shown. As shown in Figure 3 The restricted Boltzmann machine 300 includes a hidden layer 310, a first visible layer 320, and a second visible layer 330. The hidden layer 310 includes hidden nodes 310-1 and 310-2, the first visible layer 320 includes visible layer nodes 320-1 to 320-N, and the second visible layer 330 includes visible layer nodes 330-1 to 330-N. According to an embodiment of the present disclosure, the first visible layer 320 and the second visible layer 330 can also be merged into a single hidden layer.

[0050] ​According to embodiments of the present disclosure, the visible layer nodes 320-1 to 320-N included in the first visible layer 320 correspond to each question for which the answer library corresponding to the restricted Boltzmann machine 300 can be answered, where the value of the visible layer node corresponding to the question that can be answered by the answer library can be set to 1, and the question corresponding to these visible layer nodes can be considered as a positive sample for training the restricted Boltzmann machine 300. Conversely, the value of the visible layer node corresponding to the question that cannot be answered by the answer library can be set to 0, and the question corresponding to these visible layer nodes can be considered as a negative sample for training the restricted Boltzmann machine 300.

[0051] According to embodiments of the present disclosure, the computing device 110 obtains the log files collected by the customer service system, which can include information about each answer library and the questions that can be answered by the answer library. With respect to the log files of the customer service system, the title and description of the question can be filled in and submitted as a service request to describe the question, and then the customer service personnel responsible for answering the question can attach the appropriate answer library to this question. Not all questions can be labeled with the corresponding answer library, and NA can also be labeled to indicate that no suitable answer library is found. The dataset containing all questions can be defined as SR ALL , and the dataset containing all labeled questions can be defined as LSR ALL .

[0052] As mentioned earlier, the title and description can be concatenated as a complete service request, which is used as input. In the case of foreign languages, all letters in the title and description are converted to lowercase, and punctuation marks are removed.

[0053] The computing device 110 can divide the entire log dataset into a training dataset and an evaluation dataset according to the creation timestamp of the question. Accordingly, SR ALL can be divided into a question training dataset SR TRAIN and a question evaluation dataset SR EVAL . The question training dataset can be used as a training corpus to train the restricted Boltzmann machine 300. The restricted Boltzmann machine 300 can be evaluated on the question evaluation dataset in the form of the previous n result accuracy. When constructing the restricted Boltzmann machine 300 for one answer library, some other questions can be selected as visible layer nodes from another answer library, and the values of these visible layer nodes can be 0 because the questions from another answer library cannot be answered by the current answer library.

[0054] According to an embodiment of the present disclosure, the visible layer nodes 330-1 to 330-N included in the second visible layer 330 respectively correspond to each visible layer node in the second visible layer 320, and are used to represent the training association relationship between the question in each visible layer node in the second visible layer 320 and the answer bank corresponding to the restricted Boltzmann machine 300. The training association relationship can be used to represent the degree to which the question can be used to illustrate the answer bank. For example, if a certain question is very representative in the answer bank, for example, it is a very typical question in a category of questions to which the answer bank is directed, then the degree of the training association relationship between the question and the answer bank is very high. For another example, if a certain question has a very high association with other questions in the answer bank, for example, the question has a very high semantic association with the most questions in the answer bank, then the degree of the training association relationship between the question and the answer bank can also be considered to be very high.

[0055] According to an embodiment of the present disclosure, the training association relationship between the question and the answer bank corresponding to the restricted Boltzmann machine 300 can be manually labeled, or can be determined by the following formula:

[0056] E i = sim(SR, SR i ) x dis i，c x r i (8)

[0057] In formula (8), E i represents the training association relationship between the question SR i with the answer bank, and sim(SR, SR i ) refers to the similarity between the question SR i with all questions SR in the answer bank, where sim represents that the similarity is in the form of cosine similarity. According to another embodiment of the present disclosure, the similarity between the question SR i with all other questions SR in the answer bank can also be represented in the form of semantic similarity, and the protection scope of the present disclosure is not limited thereto.

[0058] Further, in formula 8, dis i，c represents the representative value of the question SR i to all questions SR in the answer bank. It can be determined by, for example, K-means method or K-nearest neighbor method.

[0059] The K-means method can also be referred to as the K-means algorithm, which is an iterative algorithm that attempts to divide a dataset into K predefined different non-overlapping subgroups or clusters, where each data point belongs to only one cluster. The K-means method attempts to make the data points within a cluster as similar as possible while keeping the clusters as different as possible. The K-means method assigns data points to a cluster such that the sum of the squared distances between the data point and the cluster centroid, i.e., the arithmetic mean of all data points belonging to the cluster, is minimized, because the smaller the difference in the cluster, the more similar the data points in the same cluster are. The K-means method can work in a way that includes specifying the number of clusters K, initializing the centroids by first shuffling the dataset and then randomly selecting K data points as centroids without replacement, and continuing iteration until the centroids do not change, i.e., the assignment of data points to clusters does not change. In this process, the sum of the squared distances between the data points and all centroids can be calculated, each data point is assigned to the nearest cluster or centroid, and the centroid of the cluster is calculated by taking the average of all data points belonging to each cluster.

[0060] The method by which the K-means method solves the problem is known as expectation maximization. The expectation step or E-step is the assignment of data points to the nearest cluster. The maximization step or M-step is the calculation of the centroid of each cluster.

[0061] As can be seen, the K-means method essentially recalculates the centroid of each cluster to reflect the new assignments. Thus, using the K-means method, a representative value for the question SR i i can be determined.

[0062] Further, in equation (8), r i i is a question SR i whether the question SR i i can be solved by the answer base. According to embodiments of the present disclosure, if the question SR i i can be solved by the answer base, the value of r i i cannot be solved by the answer base, the value of r i i cannot be solved by the answer base, the value of r

[0063] As can be seen, the value of E i i obtained by equation (8) is between 0 and 1.

[0064] According to some other embodiments of the present disclosure, the form of equation (8) can also be further adjusted. For example, only one of sim(SR, SR i ) and dis i，c may be included in equation (8), so that a simplified equation (8) can be obtained.

[0065] According to embodiments of the present disclosure, the process of training the restricted Boltzmann machine 300 by the computing device 110 can be summarized as follows: obtaining a set of questions that can be answered by the answer base as positive samples; obtaining a set of questions that cannot be answered by the answer base as negative samples; obtaining a training association relationship between the samples in the positive samples and the negative samples and the answer base; and training the restricted Boltzmann machine 300 using the positive samples, the negative samples, and the training association relationship.

[0066] Further, according to embodiments of the present disclosure, the computing device 110 can determine the training association relationship based on whether a sample can be answered by the answer base and one of the following: a similarity between the sample and the positive samples and the negative samples; and a representative value of the sample to the positive samples and the negative samples. The similarity between the sample and the positive samples and the negative samples can be determined using cosine similarity, and the representative value of the sample to the positive samples and the negative samples can be determined using the K-nearest neighbor method or the K-means method.

[0067] According to some embodiments of the present disclosure, training the restricted Boltzmann machine 300 can define a joint distribution over (v, h) based on the aforementioned association relationship. j i where a, b, and c are biases, f and n are the number of hidden units and visible units, respectively. W and D are weights of the logical functions

[0068] According to some embodiments of the present disclosure, an approximation method of the gradient of a different objective function can be used, which can be referred to as "contrastive divergence (CD)":

[0069] ΔW ij W i j i j

[0070] where W ij is an element of the learned matrix that captures the influence of the simulated rating on h.

[0071] Learning D is using CD to capture the influence of h, which is similar in form to:

[0072] ΔD ij D i j i ​​​​​​​​​​​​​​​j >recom) (10)

[0073] At block 206, the computing device 110 determines, using the restricted Boltzmann machine 300, explanatory information associated with the question 120 for the answer bank. According to embodiments of the present disclosure, the explanatory information for the answer bank can take various forms.

[0074] According to some embodiments of the present disclosure, determining, using the restricted Boltzmann machine 300, the explanatory information associated with the question 120 for the answer bank can include determining, based on the associations between the questions in the set of questions and the answer bank, a first threshold number of questions in the set of questions that have the highest association with the answer bank. For example, as previously described, since the associations between each question that the answer bank associated with the restricted Boltzmann machine 300 can answer and the answer bank can be determined using the restricted Boltzmann machine 300, and these associations can be quantified as a value between 0 and 1, it can be easy to determine, by ranking, a number of questions, e.g., 10, 5, 1, in the set of questions that the answer bank associated with the restricted Boltzmann machine 300 can answer that have the highest association.

[0075] According to other embodiments of the present disclosure, determining, using the restricted Boltzmann machine 300, the explanatory information associated with the question 120 for the answer bank can include determining, based on the associations between the questions in the set of questions and the answer bank, questions in the set of questions that have an association with the answer bank that is higher than a threshold association. For example, as previously described, since the associations between each question that the answer bank associated with the restricted Boltzmann machine 300 can answer and the answer bank can be determined using the restricted Boltzmann machine 300, and these associations can be quantified as a value between 0 and 1, it can be easy to determine, by ranking, questions in the set of questions that the answer bank associated with the restricted Boltzmann machine 300 can answer that have an association that reaches a predetermined value, e.g., 0.8.

[0076] According to still other embodiments of the present disclosure, determining, using the restricted Boltzmann machine 300, the explanatory information associated with the question 120 for the answer bank can include determining, a second threshold number of questions in the set of questions that have the highest association with the question 120. For example, it can be determined by semantic similarity, a number of questions, e.g., 10, 5, 1, in the answer bank associated with the restricted Boltzmann machine 300 that are most similar to the question 120 to be solved.

[0077] According to embodiments of the present disclosure, when the computing device 110 determines the answer base associated with the question 120 at block 202 and determines the explanatory information associated with the question 120 for the answer base using the restricted Boltzmann machine 300 at block 206, the computing device 110 can additionally provide the explanatory information for the determined answer base to the customer service personnel. According to some embodiments of the present disclosure, the computing device 110 can provide the explanatory information in the form of, for example, "This answer base can also answer the following questions" and list the questions that can be answered or the identifiers of the questions. According to other embodiments of the present disclosure, the computing device 110 can provide the explanatory information in the form of, for example, "The question 120 is more relevant to the following questions in this answer base" and list the questions or the identifiers of the questions.

[0078] The above description is made by reference to Figures 1 to 3 The example question and answer environment 100 in which the device and / or method according to embodiments of the present disclosure can be implemented, the question and answer method 200 according to embodiments of the present disclosure, and the restricted Boltzmann machine 300 according to embodiments of the present disclosure are described. It should be understood that the above description is made for better illustrating the content recorded in the embodiments of the present disclosure, rather than limiting the protection scope of the embodiments of the present disclosure in any way.

[0079] It should be understood that the number of various elements and the size of physical quantities employed in the embodiments of the present disclosure and in the various drawings are only examples, and are not a limitation on the protection scope of the embodiments of the present disclosure. The above number and size can be arbitrarily set as needed without affecting the normal implementation of the embodiments of the present disclosure.

[0080] By the above description by reference to Figures 1 to 3 , the technical solution according to embodiments of the present disclosure proposes a collaborative filtering recommendation method based on a restricted Boltzmann machine. By using the set of questions that can be answered by the answer base as positive samples and the set of questions that cannot be answered by the answer base as negative samples, the training association relationship between the samples in the positive samples and the negative samples and the answer base is obtained, and the restricted Boltzmann machine is trained using the positive samples, the negative samples, and the training association relationship, so that the trained restricted Boltzmann machine can be used to determine the set of questions that can be answered by the answer base and the association relationship between the questions in the set of questions and the answer base, thereby the trained restricted Boltzmann machine can be used to determine the explanatory information associated with the question for the answer base.

[0081] Specifically, the technical solution according to embodiments of the present disclosure has many advantages over the conventional solution.

[0082] For example, using the technical solution of the present disclosure, the customer service personnel can be made to have a more thorough understanding of the determined answer base and obtain targeted recommended information by providing the determined explanatory information to the customer service personnel, thereby improving the user experience of the customer service personnel using the question and answer system.

[0083] For another example, using the technical solution of the present disclosure, the restricted Boltzmann machine for each answer base can be easily established and trained, thereby the restricted Boltzmann machine which simultaneously takes into account both the question set that the answer base can answer and the association between the question set and the answer base can be easily implemented, and the explanatory information for the answer base associated with the question can be determined and provided according to various preset requirements.

[0084] The technical effect comparison between the question and answer method according to the embodiments of the present disclosure and the conventional solution is described below in combination with a comparative example.

[0085] Figure 4 A schematic diagram of the accuracy rate comparison 400 according to the embodiments of the present disclosure is shown. In Figure 4 , the horizontal coordinate is the number of provided questions, for example, when the question with the highest association between the answer base in the provided question set is provided, and the vertical coordinate is the accuracy rate converted into a decimal, 410 is the accuracy rate achieved using the technical solution of the present disclosure, and 420 is the accuracy rate achieved by the conventional solution without considering the explanatory information for the answer base associated with the question.

[0086] It can be seen that using the technical solution of the present disclosure, the accuracy rate can be significantly better than the conventional technology for different numbers of provided questions. Figure 4 It can be seen that using the technical solution of the present disclosure, the accuracy rate can be significantly better than the conventional technology for different numbers of provided questions.

[0087] Figure 5 FIG. 1 shows a schematic block diagram of an example device 500 that can be used to implement embodiments of the present disclosure. According to embodiments of the present disclosure, Figure 1 The computing device 110 in FIG. 1 can be implemented by the device 500. As shown, the device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 502 or loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0088] A number of components in device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, a CD, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows device 500 to exchange information / data with other devices over a computer network, such as the Internet, and / or various telecommunication networks.

[0089] The various processes and functions described above, such as the method 200, can be performed by the processing unit 501. For example, in some embodiments, the method 200 can be implemented as a computer software program tangibly embodied in a machine readable medium, such as the storage unit 508. In some embodiments, portions or all of the computer program can be loaded onto and / or installed on device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded onto the RAM 503 and executed by the CPU 501, one or more acts of the method 200 described above can be performed.

[0090] Embodiments of the present disclosure can relate to methods, devices, systems and / or computer program products. Computer program products can include computer readable storage media having computer readable program instructions embodied therewith, wherein the instructions are executed by a computer processor to implement aspects of the embodiments.

[0091] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium as a non-exhaustive list include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a ROM, a magnetic track storage and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media, or electrical signals through a wire, cable, or other transmission mediums.

[0092] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0093] Computer readable program instructions for carrying out operations of embodiments of the disclosure can be assembly-level instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of embodiments of the present disclosure.

[0094] Aspects of the embodiments of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0095] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, to cause a series of operational elements or steps to be performed on the computer to produce a computer implemented process. Such instructions can also be stored and / or executed by other computer-readable media. Computer-readable media storing the computer readable instructions can include computers, processors, or other programmable data processing apparatuses capable of receiving, storing, and / or executing instructions.

[0096] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational elements or steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable elements, or other

[0097] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational elements or steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable elements, or other

[0098] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative of the embodiments and not restrictive of the disclosure. Many modifications and variations of the described embodiments are possible in light of this disclosure without departing from the scope and spirit of the described embodiments. The choice of words in this document is intended to best explain the principles of the embodiments, the practical application, or technical improvements over the existing technology, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method of question answering, comprising: implementing a plurality of restricted Boltzmann machines in a processor-based recommendation system for respective ones of a plurality of answer repositories, wherein each of the plurality of restricted Boltzmann machines comprises at least a plurality of hidden nodes, a plurality of visible nodes of a first type corresponding to respective questions used to train the restricted Boltzmann machine, and a plurality of visible nodes of a second type corresponding to respective associations between the questions and the answer repositories used to train the restricted Boltzmann machine, the second type being different from the first type; determining, in the processor-based recommendation system, an answer repository associated with a question corresponding to a service request received by the processor-based recommendation system; determining, in the processor-based recommendation system, a restricted Boltzmann machine associated with the answer repository, the restricted Boltzmann machine being used to determine a set of questions that the answer repository can answer and an association between questions in the set of questions and the answer repository; determining, in the processor-based recommendation system, using the restricted Boltzmann machine associated with the answer repository, descriptive information associated with the question for the answer repository; and generating, in the processor-based recommendation system, a recommendation based on the descriptive information for responding to the service request. 2.The method of claim 1, wherein determining the answer repository associated with the question comprises: determining the answer repository associated with the question based on at least one of: a relevance of the question to questions that can be answered by the answer repository; and an association of the question with the answer repository. 3.The method of claim 1, further comprising: training the restricted Boltzmann machine by: obtaining a set of questions that can be answered by the answer repository as positive samples; obtaining a set of questions that cannot be answered by the answer repository as negative samples; obtaining training associations of samples in the positive samples and the negative samples with the answer repository; and training the restricted Boltzmann machine with the positive samples, the negative samples, and the training associations. 4.The method of claim 3, wherein obtaining the training associations of the samples in the positive samples and the negative samples with the answer repository comprises: determining the training associations based on whether the samples can be answered by the answer repository and one of: a similarity of the samples to the positive samples and the negative samples; and a representative value of the samples to the positive samples and the negative samples. 5.The method of claim 4, further comprising: determining the similarity using cosine similarity. 6.The method of claim 4, further comprising: determining the representative using one of: K-nearest neighbors method; and K-means method. 7.The method of claim 1, wherein determining the descriptive information associated with the question for the answer repository comprises: ​ ​ ​ determine, based on the association between the questions in the question set and the answer library, a first threshold number of questions in the question set that have a highest association with the answer library.

8. The method of claim 1, wherein determining the explanatory information associated with the question for the answer library comprises: determine, based on the association between the questions in the question set and the answer library, a question in the question set that has an association with the answer library that is higher than a threshold association.

9. The method of claim 1, wherein determining the explanatory information associated with the question for the answer library comprises: determine a second threshold number of questions in the question set that have a highest association with the question.

10. An electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which when executed by the at least one processing unit, cause the device to perform acts comprising: implementing, in a processor-based recommendation system, a plurality of restricted Boltzmann machines for respective answer libraries in a plurality of answer libraries, wherein each restricted Boltzmann machine in the plurality of restricted Boltzmann machines comprises at least: a plurality of hidden nodes, a plurality of visible nodes of a first type corresponding to respective questions used to train the restricted Boltzmann machine, and a plurality of visible nodes of a second type corresponding to respective association relationships between the questions and the answer libraries used to train the restricted Boltzmann machine, the second type being different from the first type; determining, in the processor-based recommendation system, an answer library associated with a question corresponding to a service request received by the processor-based recommendation system; determining, in the processor-based recommendation system, a restricted Boltzmann machine associated with the answer library, the restricted Boltzmann machine being used to determine a question set that the answer library can answer and an association relationship between questions in the question set and the answer library; determining, in the processor-based recommendation system, explanatory information associated with the question for the answer library using the restricted Boltzmann machine associated with the answer library; and generating, in the processor-based recommendation system, a recommendation based on the explanatory information for responding to the service request.

11. The electronic device of claim 10, wherein determining the answer library associated with the question comprises: determining the answer library associated with the question based on at least one of: an association of the question with questions that can be answered by the answer library; and an association of the question with the answer library.

12. The electronic device of claim 10, the acts further comprising: training the restricted Boltzmann machine by: obtaining a question set that can be answered by the answer library as positive samples; obtaining a question set that cannot be answered by the answer library as negative samples; ​ obtaining training associations of samples in the positive samples and the negative samples with the answer base; and training the restricted Boltzmann machine using the positive samples, the negative samples, and the training associations.

13. The electronic device of claim 12, wherein obtaining the training associations of the samples in the positive samples and the negative samples with the answer base comprises: determining the training associations based on whether the samples can be answered by the answer base and one of: a similarity of the samples to the positive samples and the negative samples; and a representative value of the samples to the positive samples and the negative samples.

14. The electronic device of claim 13, the acts further comprising: determining the similarity using cosine similarity.

15. The electronic device of claim 13, the acts further comprising: determining the representative using one of: K-Nearest Neighbors; and K-Means.

16. The electronic device of claim 10, wherein determining the informative information associated with the query for the answer base comprises: determining a first threshold number of queries in the set of queries that have a highest association with the answer base based on the associations of the queries in the set of queries with the answer base.

17. The electronic device of claim 10, wherein determining the informative information associated with the query for the answer base comprises: determining queries in the set of queries that have an association with the answer base that is higher than a threshold association based on the associations of the queries in the set of queries with the answer base.

18. The electronic device of claim 10, wherein determining the informative information associated with the query for the answer base comprises: determining a second threshold number of queries in the set of queries that have a highest association with the query.

19. A computer program product, the computer program product being tangibly stored on a non-transient computer readable medium and comprising machine executable instructions that, when executed, cause a machine to perform steps of a method according to any one of claims 1 to 9.

Images