Question matching method and device, question answering method and device

Abstract

The embodiments of the present application provide a method and device for matching questions, and a method and device for answering questions, and relate to the field of artificial intelligence technology. The method for matching questions includes: obtaining questions to be matched, and matching the questions to be matched using a dictionary tree. If the matching fails, multiple candidate questions similar to the questions to be matched are retrieved from a preset question library, and the questions to be matched are matched with the candidate questions using a trained semantic similarity calculation model based on question pairs. Among them, the model includes a coding layer, a local interaction layer, and an aggregation layer. Thus, the semantic similarity of question pairs is calculated using a trained semantic similarity calculation model based on question pairs, and semantics is used as a reference factor for finding similar candidate questions, thereby improving the accuracy of determining similar questions. The embodiments of the present application are used in a question-and-answer system, which runs on electronic devices such as mobile phones, and realizes intelligent answers through natural language processing technology.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a question matching method and device, and a question answering method and device. Background Art

[0002] Question-answering technology refers to the technology in which users ask questions and machines automatically answer them during human-computer interaction. Common question-answering technologies include FAQ (Frequently Asked Questions) and CQA (Community-based Question and Answeri). Among them, FAQ stores frequently asked questions and corresponding answers in the form of a list. After receiving questions raised by users, it searches for similar frequently asked questions in the list and uses the answers corresponding to the frequently asked questions to reply to users.

[0003] In the related art, the stored FAQs are used as training data to train a classifier, and the trained classifier is used to classify the questions raised by users, thereby finding similar FAQs. The classifier classifies questions based on the characters included in the questions, without involving the semantics of the questions, and therefore the determination of similar questions is not accurate enough. Summary of the invention

[0004] The present application provides a question matching method and device, and a question answering method and device, so as to realize the use of a trained semantic similarity calculation model based on question pairs to calculate the semantic similarity of question pairs, and use semantics as a reference factor for finding similar candidate questions, thereby improving the accuracy of determining similar questions.

[0005] In a first aspect, the present application provides a method for matching questions, the method comprising: obtaining questions to be matched; matching the questions to be matched using a dictionary tree; if the matching fails, retrieving multiple candidate questions similar to the questions to be matched from a preset question library; matching the questions to be matched with the candidate questions using a trained semantic similarity calculation model based on question pairs; wherein the semantic similarity calculation model based on question pairs comprises an input layer, an encoding layer, a local interaction layer, an aggregation layer and an output layer, the input layer being used to input word information of the question pair, the encoding layer being used to perform semantic analysis on the questions in the question pair, determine the importance of each word included in the question and learn the structural features of the questions, the local interaction layer being used to perform semantic relevance analysis on the two questions in the question pair, the aggregation layer being used to perform feature extraction and aggregation on the output of the local interaction layer, and the output layer being used to calculate the semantic similarity of the question pair.

[0006] Optionally, before obtaining the question to be matched, the method further includes: updating the dictionary tree, the preset question library, and the corresponding index library.

[0007] Optionally, the use of the dictionary tree to match the questions to be matched includes: removing modal particles in the questions to be matched; unifying punctuation marks in the questions to be matched; performing synonym replacement on the questions to be matched to generate multiple similar questions to the questions to be matched; and using the dictionary tree to match each of the similar questions separately.

[0008] Optionally, the trained question-pair-based semantic similarity calculation model is used to match the question to be matched with the candidate question, including: forming a question pair with each of the candidate question and the question to be matched; inputting the question pair into the question-pair-based semantic similarity calculation model; and matching the question to be matched with the candidate question according to the semantic similarity corresponding to each question pair.

[0009] Optionally, the semantic similarity calculation model based on question pairs is trained by the following steps: obtaining a reference question pair and a reference semantic similarity corresponding to the reference question pair; wherein the reference question pair includes a first reference question and a second reference question; performing word segmentation processing on the first reference question and the second reference question respectively to generate a first reference word set corresponding to the first reference question and a second reference word set corresponding to the second reference question; determining the part of speech corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference part of speech set corresponding to the first reference word set and a second reference part of speech set corresponding to the second reference word set; determining the part of speech corresponding to each reference word in the first reference word set and the second reference word set Synonyms corresponding to the words are generated to generate a first reference synonym set corresponding to the first reference word set and a second reference synonym set corresponding to the second reference word set; the first reference word set, the second reference word set, the first reference part-of-speech set, the second reference part-of-speech set, the first reference synonym set and the second reference synonym set are input into the encoding layer of the semantic similarity calculation model based on question pairs; the parameters of the semantic similarity calculation model based on question pairs are trained according to the output of the output layer of the semantic similarity calculation model based on question pairs and the reference semantic similarity; when the accuracy of the semantic similarity calculation model based on question pairs is greater than a preset threshold, the training of the semantic similarity calculation model based on question pairs is completed.

[0010] Optionally, the encoding layer includes a bidirectional recurrent neural network layer, a first normalization layer and a stacked bidirectional self-attention layer.

[0011] Optionally, the local interaction layer includes a bidirectional multi-angle similarity analysis layer and a second normalization layer.

[0012] In a second aspect, the present application provides a method for answering questions, the method comprising: obtaining a question to be answered; using the aforementioned question matching method to determine a candidate question that matches the question to be answered; and answering using a candidate answer corresponding to the candidate question.

[0013] In a third aspect, the present application provides a question matching device, the device comprising: a first acquisition module, used to acquire questions to be matched; a first matching module, used to match the questions to be matched using a dictionary tree; a retrieval module, used to retrieve multiple candidate questions similar to the questions to be matched from a preset question library when the first matching module fails to match; a second matching module, used to match the questions to be matched with the candidate questions using a trained semantic similarity calculation model based on question pairs; wherein the semantic similarity calculation model based on question pairs comprises an input layer, an encoding layer, a local interaction layer, an aggregation layer and an output layer, the input layer is used to input word information of the question pair, the encoding layer is used to perform semantic analysis on the questions in the question pair, determine the importance of each word included in the question and learn the structural features of the question, the local interaction layer is used to perform semantic relevance analysis on the two questions in the question pair, the aggregation layer is used to perform feature extraction and aggregation on the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair.

[0014] Optionally, the device further includes: an updating module, configured to update the dictionary tree, the preset question library, and a corresponding index library.

[0015] Optionally, the first matching module includes: a removal submodule for removing modal particles in the question to be matched; a unification submodule for unifying punctuation marks in the question to be matched; a replacement submodule for performing synonym replacement on the question to be matched to generate multiple similar questions to the question to be matched; and a first matching submodule for using the dictionary tree to match each of the similar questions separately.

[0016] Optionally, the second matching module includes: a pairing submodule, used to form a question pair with each of the candidate questions and the question to be matched; an input submodule, used to input the question pairs into the semantic similarity calculation model based on question pairs; and a second matching submodule, used to match the question to be matched with the candidate questions according to the semantic similarity corresponding to each of the question pairs.

[0017] Optionally, the device further includes: a second acquisition module, used to acquire a reference question pair, and a reference semantic similarity corresponding to the reference question pair; wherein the reference question pair includes a first reference question and a second reference question; a processing module, used to perform word segmentation processing on the first reference question and the second reference question, respectively, to generate a first reference word set corresponding to the first reference question, and a second reference word set corresponding to the second reference question; a first determination module, used to determine the part of speech corresponding to each reference word in the first reference word set and the second reference word set, to generate a first reference part of speech set corresponding to the first reference word set, and a second reference part of speech set corresponding to the second reference word set; a second determination module, used to determine the part of speech corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference synonym set corresponding to the first reference word set and a second reference synonym set corresponding to the second reference word set; an input module, used to input the first reference word set, the second reference word set, the first reference part-of-speech set, the second reference part-of-speech set, the first reference synonym set and the second reference synonym set into the encoding layer of the semantic similarity calculation model based on question pairs; a training module, used to train the parameters of the semantic similarity calculation model based on question pairs according to the output of the output layer of the semantic similarity calculation model based on question pairs and the reference semantic similarity; a completion module, used to complete the training of the semantic similarity calculation model based on question pairs when the accuracy of the semantic similarity calculation model based on question pairs is greater than a preset threshold.

[0018] Optionally, the encoding layer includes a bidirectional recurrent neural network layer, a first normalization layer and a stacked bidirectional self-attention layer.

[0019] Optionally, the local interaction layer includes a bidirectional multi-angle similarity analysis layer and a second normalization layer.

[0020] In a fourth aspect, the present application provides a device for answering questions, the device comprising: a third acquisition module, used to obtain questions to be answered; a third determination module, used to use the aforementioned question matching method to determine candidate questions that match the questions to be answered; and a reply module, used to reply using candidate answers corresponding to the candidate questions.

[0021] In a fifth aspect, the present application provides a question-and-answer system, which includes: a question-and-answer interface for receiving user input content and displaying generated reply content; a distribution agent for distributing the user's input content to a corresponding reply device based on the type of the user's input content; and a reply device for the aforementioned question, which is used to receive questions to be replied to sent by the distribution agent and determine corresponding answers from a preset question library.

[0022] In a sixth aspect, the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are performed: obtaining a question to be matched; matching the question to be matched using a dictionary tree; if the match fails, retrieving multiple candidate questions similar to the question to be matched from a preset question library; matching the question to be matched with the candidate questions using a trained semantic similarity calculation model based on question pairs; wherein the semantic similarity calculation model based on question pairs comprises an input layer, an encoding layer, a local interaction layer, an aggregation layer, and an output layer, wherein the input layer is used to input word information of the question pair, the encoding layer is used to perform semantic analysis on the questions in the question pair, determine the importance of each word included in the question and learn the structural features of the question, the local interaction layer is used to perform semantic relevance analysis on the two questions in the question pair, the aggregation layer is used to perform feature extraction and aggregation on the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair.

[0023] Optionally, before the electronic device obtains the question to be matched, the following steps are further performed: updating the dictionary tree, the preset question library, and the corresponding index library.

[0024] Optionally, the electronic device uses a dictionary tree to match the questions to be matched, specifically including the following steps: removing modal particles in the questions to be matched; unifying punctuation marks in the questions to be matched; performing synonym replacement on the questions to be matched to generate multiple similar questions to the questions to be matched; and using the dictionary tree to match each of the similar questions separately.

[0025] Optionally, the electronic device uses a trained semantic similarity calculation model based on question pairs to match the question to be matched with the candidate questions, which specifically includes the following steps: each of the candidate questions is respectively combined with the question to be matched to form a question pair; the question pairs are input into the semantic similarity calculation model based on question pairs; and the question to be matched is matched with the candidate questions according to the semantic similarity corresponding to each question pair.

[0026] Optionally, when training the semantic similarity calculation model based on question pairs, the electronic device performs the following steps: obtaining a reference question pair, and a reference semantic similarity corresponding to the reference question pair; wherein the reference question pair includes a first reference question and a second reference question; performing word segmentation processing on the first reference question and the second reference question respectively to generate a first reference word set corresponding to the first reference question, and a second reference word set corresponding to the second reference question; determining the part of speech corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference part of speech set corresponding to the first reference word set, and a second reference part of speech set corresponding to the second reference word set; determining the first reference word set and the second reference word set The method comprises the steps of: selecting synonyms corresponding to the reference words to generate a first reference synonym set corresponding to the first reference word set and a second reference synonym set corresponding to the second reference word set; inputting the first reference word set, the second reference word set, the first reference part-of-speech set, the second reference part-of-speech set, the first reference synonym set and the second reference synonym set into the encoding layer of the semantic similarity calculation model based on question pairs; training parameters of the semantic similarity calculation model based on question pairs according to the output of the output layer of the semantic similarity calculation model based on question pairs and the reference semantic similarity; and completing the training of the semantic similarity calculation model based on question pairs when the accuracy of the semantic similarity calculation model based on question pairs is greater than a preset threshold.

[0027] Optionally, the encoding layer includes a bidirectional recurrent neural network layer, a first normalization layer and a stacked bidirectional self-attention layer.

[0028] Optionally, the local interaction layer includes a bidirectional multi-angle similarity analysis layer and a second normalization layer.

[0029] In a seventh aspect, the present application provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are performed: obtaining a question to be answered; using the method described in the first aspect to determine a candidate question that matches the question to be answered; and replying using a candidate answer corresponding to the candidate question.

[0030] In an eighth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a structural diagram of a classification-based question-answering system in the prior art;

[0032] Figure 2a A flowchart of a method for constructing a deep learning model in the prior art;

[0033] Figure 2b A flowchart of a question-answering method based on a deep learning model in the prior art;

[0034] Figure 3 A flowchart of a matching method for a problem proposed in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of the process flow of the same processing proposed in the embodiment of the present application;

[0036] Figure 5 A flowchart of the training steps of a semantic similarity calculation model based on question pairs provided in an embodiment of the present application;

[0037] Figure 6 A schematic diagram of a process for generating positive training examples and negative training examples according to an embodiment of the present application;

[0038] Figure 7 A flowchart of a problem matching method provided in an embodiment of the present application;

[0039] Figure 8 for Figure 7 Schematic diagram of initialization loading;

[0040] Fig. 9 A schematic diagram of the structure of a semantic similarity calculation model based on question pairs provided in an embodiment of the present application;

[0041] Fig.10 A schematic diagram of a process for answering questions raised in an embodiment of the present application;

[0042] Fig.11 A schematic diagram of the structure of a matching device for solving a problem proposed by an embodiment of the present application;

[0043] Fig.12 This is a schematic diagram of the structure of the first matching module proposed in the embodiment of the present application;

[0044] Fig.13 A schematic diagram of the structure of the second matching module proposed in the embodiment of the present application;

[0045] Fig.14 Another structural schematic diagram of a matching device for solving the problem raised by an embodiment of the present application;

[0046] Fig.15 A schematic diagram of the structure of a device for answering questions raised in an embodiment of the present application;

[0047] Fig.16 A schematic diagram of the structure of a question-answering system proposed in an embodiment of the present application; and

[0048] Fig.17 A schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0049] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0050] The following describes a method and device for matching questions, a method and device for answering questions, a question-answering system, and a computer-readable storage medium according to an embodiment of the present application with reference to the accompanying drawings.

[0051] In order to more clearly illustrate the problem matching method provided in the embodiment of the present application, the relevant technology is first described below.

[0052] Question answering (QA) technology refers to the machine automatically answering questions raised by users to meet their needs for relevant information. It is generally used in a variety of application scenarios such as enterprise customer service and intelligent information. Its typical application is mobile life assistants represented by Apple's Siri, Google Now, and Microsoft's Cortana.

[0053] As an intelligent human-computer interaction technology, question-answering technology can be implemented on various electronic devices. Specifically, electronic devices can include mobile terminals (mobile phones), smart screens, drones, ICVs (Intelligent Connected Vehicles), smart / intelligent cars, or vehicle-mounted devices.

[0054] In traditional search technology, the input is keywords and the output is a list of related documents. Unlike traditional search technology, the input of question-answering technology is usually a question in natural language form, and the output is a concise answer or a list of multiple possible answers.

[0055] In recent years, with the rapid development of artificial intelligence, natural language processing and mobile Internet technology, question-answering technology has gradually evolved from being centered on simple keywords to being centered on deep question-answering technology.

[0056] Based on the different types of target data sources, common question-answering technology systems can be roughly divided into: question-answering systems based on structured data, question-answering systems based on free text, and question-answering systems based on question-answer pairs. Among them, question-answering systems based on question-answer pairs include: question-answering systems based on frequently asked questions (FAQ) and question-answering systems based on community questions (CQA).

[0057] The question-answering system (FAQ) based on the frequently asked questions list has the advantages of large data volume, high question quality and good data structure, so it is very suitable for mobile life assistant applications.

[0058] In the prior art, there are many implementation methods for a question-answering system based on a frequently asked question list.

[0059] The first implementation method is to use the trained classifier to classify the questions raised by the user, and then find similar FAQs from the same type of questions, and use the answers corresponding to the FAQs to reply to the user. Figure 1 FIG. 1 is a schematic diagram of a classification-based question-answering system in the prior art. Figure 1 As shown, an existing classification-based question-answering system divides the frequently asked questions stored in the question file and the corresponding answers stored in the answer file into different question groups. The different question groups are used as training data to train the classifier, so that the classifier can classify different types of questions.

[0060] During use, the user inputs a question through the user interface module, and the classifier determines the category of the question based on the question raised by the user, and then retrieves multiple FAQs similar to the question in the same category. From the answers corresponding to these FAQs, the answer that best matches the question raised by the user is selected and displayed to the customer through the user interface module.

[0061] In the first implementation, the classifier classifies questions based on characters and the number of characters, which does not solve the semantic gap problem in traditional search technology. As the number of FAQs increases, the accuracy of the classifier decreases, and it cannot be applied to different types of application scenarios.

[0062] In addition, as time goes by, new question-answer pairs are constantly generated, old question-answer pairs are deleted or some question-answer pairs are updated. After the training is completed, the above classifier cannot be updated in real time according to the update of the question-answer pairs, so that similar FAQs retrieved by the classifier are relatively old.

[0063] The second implementation method is to crawl the question-and-answer dataset from the Internet and store it as a list of frequently asked questions. From the question-and-answer dataset, multiple frequently asked questions similar to the questions raised by the user and the answers to these questions are found, and the answers are matched with the questions raised by the user using a neural network-based question-and-answer matching model to determine the answer that best matches the question raised by the user. Figure 2a The present invention is a flowchart of a method for constructing a deep learning model in the prior art. Figure 2b FIG. 1 is a flowchart of a question-answering method based on a deep learning model in the prior art. Figure 2a As shown in the figure, in the process of building a deep learning model, network question and answer data are captured from the Internet and stored in a relational database as a question and answer data set, and a full-text retrieval service is established. Chinese word segmentation is performed on the questions stored in the relational database, and BOW (Bag of Words) vectors, TF-IDF (Term Frequency–Inverse Document Frequency) values ​​and word2vec word vectors are generated based on the results of Chinese word segmentation, and then the corresponding text representation vectors are generated. The question and answer data in the question and answer data set are used as training data to train a neural network-based question and answer matching model. As shown in the figure, Figure 2b As shown in the figure, after receiving the user's question, the full-text search service provided by the relational database is used to generate a similar question set from multiple questions similar to the user's question, and the text representation vector of each similar question in the similar question set is obtained. The user's question is segmented into Chinese words, and the BOW vector, TF-IDF value and word2dev word vector are generated based on the result of Chinese word segmentation, and then the text representation vector corresponding to the user's question is generated.

[0064] By calculating the cosine similarity between the text representation vectors of similar questions and the text representation vectors of user questions, multiple similar questions are generated, and the answers corresponding to these similar questions are used as candidate answers. The user questions and candidate answers are input into the trained neural network-based question-answer matching model to determine the answer to the user's question.

[0065] In the second implementation, as business needs and time go by, questions in the question-and-answer dataset need to be constantly added or deleted (for example, new questions need to be added to the question-and-answer dataset, or some questions are repeated or unnecessary and need to be deleted from the question-and-answer dataset). After the question-and-answer dataset changes, the steps of the deep learning model construction method need to be executed again to update the full-text retrieval service, text representation vector, and neural network-based question-and-answer matching model, and real-time dynamic updates cannot be achieved. In other words, as time goes by, the previously established full-text retrieval service, text representation vector, and neural network-based question-and-answer matching model are relatively old, and the answers to user questions determined are also relatively old and do not match the actual situation.

[0066] In addition, the neural network-based question-answering matching model is not suitable for the question-answering system based on frequently asked questions. Specifically, for the question-answering matching model, there is a difference between the semantic space of questions and answers, and it is difficult to set reasonable corresponding semantic features, resulting in inaccurate question-answering matching. Moreover, as the application scenario changes, the answer corresponding to the same question will change. Therefore, the question-answering matching model is closely related to the application scenario and is not universal.

[0067] Based on the above description of the related art, it can be known that when searching for questions similar to those asked by the user in a stored question set in the related art, the determination of similar questions is not accurate enough.

[0068] In order to solve the above problems, the present application embodiment proposes a matching method for the problem: Figure 3 This is a flow chart of a matching method for a problem proposed in an embodiment of the present application. Figure 3 As shown, the method includes:

[0069] Step S101, obtaining questions to be matched.

[0070] Based on the above description of the question-answering system, it can be known that the questions to be matched are questions raised by users.

[0071] There are many ways to obtain questions raised by users, which can be achieved through a human-computer interaction interface. Users can ask questions through the human-computer interaction interface through various input methods such as keyboard input and voice input.

[0072] Step S102, using a dictionary tree to match the questions to be matched.

[0073] It should be noted that the embodiment of the present application matches questions from two aspects: characters and semantics. Specifically, a dictionary tree is used for character matching, and a semantic similarity calculation model based on question pairs is used for semantic matching.

[0074] Among them, the dictionary tree (trie tree), also known as the word search tree, is a tree structure. The root node does not contain any characters. Except for the root node, each node contains only one character. From the root node to a certain node, the characters passed on the path are connected to form the string corresponding to the node. All child nodes of each node contain different characters.

[0075] It can be understood that the embodiment of the present application stores multiple FAQs in advance, and each FAQ is filled into a dictionary tree as a string, and the nodes in the dictionary tree can be matched with the FAQ. The string corresponding to the question to be matched is split into multiple characters, and the multiple characters are matched in sequence using the dictionary tree. If a node matching the string corresponding to the question to be matched can be found in the dictionary tree, it can be determined that the question to be matched is matched with the FAQ corresponding to the node.

[0076] It should be noted that if the dictionary tree can complete the matching of the question to be matched, the character matching of the question to be matched is completed. In other words, if the question raised by the user is exactly the same as a candidate question in terms of content, then in the subsequent reply process, the answer corresponding to the candidate question can be directly used to answer the question raised by the user.

[0077] Step S103: if the matching fails, multiple candidate questions similar to the question to be matched are retrieved from a preset question library.

[0078] It should be noted that if the matching of the question to be matched is not completed in step S102, it means that there is no candidate question in the pre-stored FAQs that is exactly the same as the question raised by the user, and the question to be matched needs to be matched through semantic analysis.

[0079] It can be understood that there are a large number of frequently asked questions in the preset question library. If each frequently asked question is input into the semantic similarity calculation model together with the question to be matched, the amount of calculation will be too large and the efficiency will be very low.

[0080] In order to reduce the amount of calculation in the subsequent process, the frequently asked questions in the preset question library can be preliminarily screened. Specifically, multiple candidate questions similar to the question to be matched are retrieved from the preset question library, and then each candidate question is input into the semantic similarity calculation model together with the question to be matched, and the candidate question matching the question to be matched is determined.

[0081] Unlike the aforementioned use of a dictionary tree for matching, here when searching the preset question library, it is only necessary to retrieve the FAQs that have the same keywords as the questions to be matched, without having to be completely consistent in content form. It can be understood that the more common keywords the FAQs in the preset question library have with the questions to be matched, the higher the similarity in content form between the FAQs and the questions to be matched. Therefore, the embodiment of the present application sorts the search results in reverse order by similarity, and then takes the first N FAQs as candidate questions, where N is a positive integer.

[0082] Step S104: Use the trained semantic similarity calculation model based on question pairs to match the question to be matched with the candidate questions.

[0083] Among them, the semantic similarity calculation model based on question pairs includes an input layer, an encoding layer, a local interaction layer, an aggregation layer and an output layer. The input layer is used to input the word information of the question pair, the encoding layer is used to perform semantic analysis on the questions in the question pair, determine the importance of each word included in the question and learn the structural characteristics of the question, the local interaction layer is used to perform semantic relevance analysis on the two questions in the question pair, the aggregation layer is used to extract and aggregate features of the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair.

[0084] After completing the preliminary screening of candidate questions in terms of content form in step S103, it is necessary to match the questions to be matched in terms of semantics.

[0085] The embodiment of the present application adopts a semantic similarity calculation model based on question pairs to match the questions to be matched with the candidate questions.

[0086] One possible implementation method is to form a question pair with each candidate question and the question to be matched, input the question pair into a semantic similarity calculation model based on the question pair, and match the question to be matched with the candidate question according to the semantic similarity corresponding to each question pair.

[0087] It should be noted that the semantic similarity calculation model based on question pairs provided in the embodiment of the present application is a deep learning sentence pair model technology. With the continuous development of artificial intelligence and deep learning technology in recent years, more and more natural language processing tasks are processed using deep learning architectures, and the more common ones are neural network language models, recurrent neural network models, sentence pair models, etc. Specifically, the deep learning sentence pair model determines the relationship between sentences by identifying the semantic information encoded by the source sentence and the target sentence.

[0088] When using the deep learning sentence pair model technology, a set of training examples is first given, each example is a triple (source sentence, target sentence, and the relationship between the source sentence and the target sentence). By training a deep learning model (such as a recurrent neural network model), the probability of predicting the relationship between any two sentences is learned.

[0089] It should be particularly noted that the word information input into the input layer of the semantic similarity calculation model based on question pairs provided in the embodiment of the present application includes a word set generated after word segmentation of the question, a part-of-speech set generated after identifying the part of speech of each word in the word set, and a synonym set generated after synonym identification for each word in the word set.

[0090] After semantic analysis of the words inputted by the input layer, the encoding layer learns the context information of each word in the question and the structural feature information of the question from the multiple words generated after word segmentation of the question. Specifically, the information of the words on the left and right sides of each word in the question can be learned through a bidirectional recurrent neural network, and the semantic relevance of each word in the question is calculated with other words, so as to determine the importance of each word included in the question and learn the structural features of the question.

[0091] After the encoding layer determines the importance of each word included in the two questions in the question pair and the structural characteristics of the learning questions, the local interaction layer performs semantic relevance analysis on the two questions in the question pair on this basis. It can be understood that when the local interaction layer performs semantic relevance analysis, the more important the word is, the greater the impact on the semantic relevance analysis. Specifically, each word in question one can be weighted to calculate the semantic relevance with question two, with the weight being the importance of the word, so as to obtain the semantic relevance of each word in question one with question two. Through similar steps, the semantic relevance of each word in question two with question one can be obtained.

[0092] In order to reduce the amount of subsequent semantic similarity calculations, the aggregation layer uses the maximum / minimum pooling method to extract features from the output of the local interaction layer, and uses the splicing method to aggregate the extracted features. The output layer calculates the semantic similarity of the question pair based on the aggregated features.

[0093] In order to enable the encoding layer to achieve the above functions, one possible implementation method is that the encoding layer includes a bidirectional recurrent neural network layer, a first normalization layer and a stacked bidirectional self-attention layer.

[0094] It should be noted that the bidirectional recurrent neural network is a special type of recurrent neural network. The recurrent neural network is a recurrent neural network that takes sequence data as input, recursively in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain. The difference is that the recurrent neural network performs recursive calculations in one direction (usually the direction of sequence evolution), while the bidirectional recurrent neural network performs recursive calculations in two directions (usually the direction of sequence evolution and the opposite direction).

[0095] It can be understood that natural language, as a kind of sequence data, usually has strong correlation, and the bidirectional recurrent neural network can encode the data by combining the previous and next data in the sequence, thereby improving the rationality of data encoding.

[0096] The first normalization layer can normalize the data to facilitate subsequent data processing.

[0097] The stacked bidirectional self-attention layer determines the importance of each word in the question from the natural language through the self-attention mechanism, thereby providing weights for the subsequent calculation of the semantic relevance between the word and the question.

[0098] In order to enable the local interaction layer to achieve the above functions, a possible implementation method is that the local interaction layer includes a bidirectional multi-angle similarity analysis layer and a second normalization layer.

[0099] It should be noted that the bidirectional multi-angle similarity analysis layer provided in the embodiment of the present application first calculates the semantic relevance between words and questions from multiple angles, and after comprehensive processing of the calculation results, a comprehensive result of the semantic relevance calculation can be obtained.

[0100] A first possible perspective is to calculate the semantic relevance between the word and each word included in the question, and use the calculation result as the semantic relevance between the word and the question.

[0101] A second possible perspective is to calculate the semantic relevance between the word and the most important word in the question, and use the calculation result as the semantic relevance between the word and the question.

[0102] A third possible perspective is to first generate semantic features of the question based on the importance of multiple words included in the question, and then calculate the semantic relevance between the words and the semantic features of the question as the semantic relevance between the words and the question.

[0103] It can be understood that in the process of generating the comprehensive results of the semantic relevance calculation, the multiple angles used may include any one of the above-mentioned possible angles, and may also include other possible angles, which is not limited in the embodiments of the present application.

[0104] In summary, the problem matching method provided in the embodiment of the present application includes: obtaining the problem to be matched, and matching the problem to be matched using a dictionary tree. If the match fails, multiple candidate questions similar to the problem to be matched are retrieved from the preset question library, and the problem to be matched is matched with the candidate question using the trained semantic similarity calculation model based on the question pair. Among them, the semantic similarity calculation model based on the question pair includes an input layer, an encoding layer, a local interaction layer, an aggregation layer and an output layer, the input layer is used to input the word information of the question pair, the encoding layer is used to perform semantic analysis on the problem in the question pair, determine the importance of each word included in the question and the structural characteristics of the learning problem, the local interaction layer is used to perform semantic relevance analysis on the two problems in the question pair, the aggregation layer is used to extract features and aggregate the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair. Thus, the semantic similarity of the question pair is calculated using the trained semantic similarity calculation model based on the question pair, and semantics is used as a reference factor for finding similar candidate questions, thereby improving the accuracy of determining similar questions.

[0105] In addition, in order to make the problem matching method proposed in the embodiment of the present application applicable to different business scenarios and continuously updated over time, before obtaining the problem to be matched in step S101, the method also includes: updating the dictionary tree and the preset question library, as well as the corresponding index library.

[0106] Based on the foregoing description, it can be known that the dictionary tree and the preset question library in the embodiment of the present application are both a storage form of frequently asked questions. In different business scenarios, the required frequently asked questions are different, and the frequently asked questions will also change over time. Therefore, before obtaining the questions to be matched, the dictionary tree and the preset question library can be updated first to ensure the accuracy of the question matching method in the embodiment of the present application.

[0107] It should be noted that when the dictionary tree and the preset question library are updated, the corresponding index library also needs to be updated to realize the retrieval function of the dictionary tree and the preset question library. The embodiment of the present application uses full-text retrieval technology to search in the preset question library.

[0108] Full-text retrieval technology is essentially a search engine technology. Search engine technology refers to the technology of automatically collecting information from the Internet and providing it to users after sorting. Search engine technology includes steps such as web crawlers, web indexing, web retrieval, and search result sorting. Search engine technology supports dynamic index updates and near real-time search of web pages, and is currently used in a large number of retrieval scenarios.

[0109] It can be understood that since the information on the Internet is constantly updated, the search engine technology can achieve near-real-time retrieval of network information. When using full-text retrieval technology to retrieve the preset question library, it is possible to ensure near-real-time search of the preset question library through steps such as data crawling, data indexing, data retrieval, and search result sorting under the condition that the preset question library is constantly updated.

[0110] Based on the foregoing description, it can be known that in step S102, using a trie tree to match the question to be matched can quickly complete the matching of the question to be matched. When using a semantic similarity calculation model to match the question to be matched, the calculation amount is large and the efficiency is low.

[0111] In order to enable as many questions to be matched as possible to be completed through the trie tree, a possible implementation method is to perform homogenization processing on different expressions of the same question. Figure 4 This is a schematic flow chart of the homogenization processing proposed in the embodiment of the present application. As Figure 4 shown, based on Figure 3 the method flow shown, in step S102, using a trie tree to match the question to be matched includes:

[0112] Step S11, removing the modal particles in the question to be matched.

[0113] Among them, modal particles refer to function words indicating mood, which are commonly used at the end of a sentence or at the pause in a sentence to express various moods. Common modal particles include "ah", "ne", "ma", etc.

[0114] Step S12, unifying the punctuation marks in the question to be matched.

[0115] Specifically, the punctuation marks can be set to ",", that is to say, the function of the punctuation marks indicating mood is removed, and they are only regarded as symbols for pauses.

[0116] It should be noted that since multiple similar questions of the question to be matched need to be generated in step S13, considering that different users have different usage habits of modal particles and punctuation marks, and modal particles and punctuation marks have no impact on the essential content of the question, the modal particles can be removed first, and the punctuation marks are uniformly converted into symbols for pauses, so as to facilitate subsequent processing.

[0117] Step S13, performing synonym replacement on the question to be matched to generate multiple similar questions of the question to be matched.

[0118] Step S14, using a trie tree to match each similar question respectively.

[0119] It can be understood that in order to match the questions to be matched with the FAQs stored in the dictionary tree, the modal particles of the FAQs are removed and the punctuation marks are processed uniformly when the dictionary tree is generated. When using the dictionary tree, the same operation is performed on the questions to be matched to increase the possibility of successful matching between the questions to be matched and the FAQs.

[0120] In addition, considering that there are many ways to express matching questions in natural language, but the substantive content is the same, if any of these expressions is pre-stored in the dictionary tree, the dictionary tree can be used to complete the matching of the matching questions. Therefore, synonym replacement can be performed on the matching questions to generate multiple similar questions with different expressions from the matching questions. Use the dictionary tree to match multiple similar questions. If any similar question is successfully matched, it can be considered that the matching question is successfully matched.

[0121] In the question matching method provided in the embodiment of the present application, it is necessary to use the trained semantic similarity calculation model based on question pairs to match the question to be matched with the candidate model. In order to train the semantic similarity calculation model based on question pairs, one possible implementation method is: Figure 5 The following is a flow chart of the training steps of a semantic similarity calculation model based on question pairs provided in an embodiment of the present application. Figure 5 As shown, the semantic similarity calculation model based on question pairs is trained through the following steps:

[0122] Step S201, obtaining a reference question pair and a reference semantic similarity corresponding to the reference question pair.

[0123] The reference question pair includes a first reference question and a second reference question.

[0124] The reference question pair and the reference semantic similarity refer to training examples used to train a semantic similarity calculation model based on question pairs.

[0125] Based on the above description of the deep learning sentence pair model technology, it can be known that the reference question pair and the reference semantic similarity form a triple in the training sample. The first reference question included in the reference question pair can be used as the source sentence, and the second reference question can be used as the target sentence. Then the reference semantic similarity is the relationship between the source sentence and the target sentence.

[0126] It should be noted that the data in the FAQ list is used as a reference question pair and a reference semantic similarity in the embodiments of the present application. However, there may be an imbalance in the training samples. Specifically, if the answers to two questions are the same, the question pair formed by the two questions can be used as a positive training example, and the reference semantic similarity is set higher, such as to 1. If the answers to the two questions are different, the question pair formed by the two questions can be used as a negative training example, and the reference speech similarity is set lower, such as to 0. It can be understood that the number of positive training examples that can be formed in the FAQ list is much smaller than the number of negative training examples, so that the influence of positive training examples on model training is too small.

[0127] In order to balance the impact of training positive examples and training negative examples on model training, the present application embodiment proposes a possible implementation method: Figure 6 This is a schematic diagram of the process of generating positive training examples and negative training examples proposed in the embodiment of the present application, such as Figure 6 As shown, the method includes:

[0128] Step S21, performing data cleaning on the FAQ list.

[0129] Specifically, in the FAQ list, if there is data with similar questions and similar corresponding answers, the questions and corresponding answers can be merged, and if there is data with the same questions but different corresponding answers, the answers can be untied. In addition, English questions in the FAQ list can also be deleted.

[0130] Step S22, determining multiple questions corresponding to each answer.

[0131] Step S23, generating training positive examples according to the multiple questions corresponding to each answer.

[0132] Based on the above description, it can be known that the number of training positive examples that can be generated in the FAQ list is much smaller than the number of training negative examples, so it is necessary to give priority to generating training positive examples.

[0133] The characteristic of a training positive example is that the answers to two questions are the same, so a training positive example can be generated from multiple questions corresponding to the answers.

[0134] In addition, considering that different answers correspond to different numbers of questions, the number of training positive examples corresponding to each answer can be set to be similar in advance.

[0135] Step S24, generating training counterexamples according to questions corresponding to different answers.

[0136] It should be understood that in order to reduce the impact of counterexamples on model training, a more sophisticated strategy is adopted for the generation of training counterexamples.

[0137] Specifically, different answers may correspond to questions with similar wordings. In order to enable the trained model to distinguish between questions with similar wordings, it is necessary to use such situations as training counterexamples as much as possible.

[0138] Secondly, the number of training negative examples needs to be controlled to make it close to the number of training positive examples.

[0139] In order to avoid the same problem appearing in different training counterexamples, it is necessary to mark the selected problems during the selection of training counterexamples.

[0140] Through the above training example generation method, more practical reference question pairs and reference semantic similarities can be obtained from the FAQ list.

[0141] In actual use, part of the training samples generated above can be used as a development data set, and part can be used as a training data set. For example, 10% of the training samples can be used for development, and 90% of the training samples can be used for training, which is not limited in the embodiments of the present application.

[0142] Step S202 , performing word segmentation processing on the first reference question and the second reference question respectively, so as to generate a first reference word set corresponding to the first reference question, and a second reference word set corresponding to the second reference question.

[0143] Step S203 , determining the part of speech corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference part of speech set corresponding to the first reference word set and a second reference part of speech set corresponding to the second reference word set.

[0144] Step S204 , determining the synonyms corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference synonym set corresponding to the first reference word set and a second reference synonym set corresponding to the second reference word set.

[0145] Step S205 , inputting the first reference word set, the second reference word set, the first reference part-of-speech set, the second reference part-of-speech set, the first reference synonym set and the second reference synonym set into the encoding layer of the semantic similarity calculation model based on the question pair.

[0146] Based on the above description of the semantic similarity calculation model based on question pairs, it can be known that the word information input by the input layer includes a word set, a part-of-speech set, and a synonym set. Accordingly, during the model training process, the input layer needs to perform word segmentation, part-of-speech recognition, and synonym recognition processing on the first reference question and the second reference question in the input reference question pair.

[0147] It should be noted that, after being processed by the input layer, the input first reference question and the second reference question obtain a first reference word set, a first reference part-of-speech set and a first synonym set corresponding to the first reference question, and a second reference word set, a second reference part-of-speech set and a second reference synonym set corresponding to the second reference question.

[0148] Step S206 , training parameters of the semantic similarity calculation model based on question pairs according to the output of the semantic similarity calculation model based on question pairs and the reference semantic similarity.

[0149] It should be noted that, based on the above description, the function of the input layer is to convert the questions in the question pair into a word set, a part of speech set, and a synonym set suitable for model processing. Therefore, during the model training process, the parameters of the input layer remain unchanged.

[0150] The training of model parameters mainly focuses on the parameters in the encoding layer, local interaction layer, aggregation layer and output layer.

[0151] Step S207, when the accuracy of the semantic similarity calculation model based on the question pair is greater than a preset threshold, the training of the semantic similarity calculation model based on the question pair is completed.

[0152] When the accuracy of the semantic similarity calculation model is greater than a preset threshold, it means that the trained model can be used to meet actual needs, and the trained semantic similarity calculation model based on question pairs can be obtained.

[0153] Thus, the training of the semantic similarity calculation model based on question pairs is realized.

[0154] In order to more clearly illustrate the problem matching method provided in the embodiment of the present application, an example is given below.

[0155] Figure 7 A flowchart of a problem matching method provided in an embodiment of the present application. Figure 8 for Figure 7 Schematic diagram of initialization loading. Figure 7 and Figure 8 As shown, initial loading is first performed, and a dictionary tree based on FAQs is constructed offline according to a pre-stored FAQ list, a preset question library with full-text retrieval technology, and reference question pairs and reference semantic similarities for training a semantic similarity calculation model based on question pairs, thereby completing the training of a semantic similarity calculation model based on question pairs.

[0156] When question-answer pairs are added, deleted, or updated in the FAQ list, the dictionary tree, preset question library, and corresponding index library are updated based on the message event, mainly the index library of the dictionary tree and the inverted index library of the preset question library.

[0157] After obtaining the questions to be matched, the questions to be matched are preprocessed to remove modal particles, unify punctuation marks, and replace synonyms to generate multiple similar questions. The dictionary tree is used to match each similar question separately. If the match is successful, the matching of the questions to be matched is completed.

[0158] If the match fails, the full-text search technology of the preset question library is used to find multiple candidate questions similar to the question to be matched from the preset question library. The trained semantic similarity calculation model based on question pairs is used to match the question to be matched with the candidate questions, and the matching of the question to be matched is completed, thereby determining the matching FAQ.

[0159] Fig. 9 This is a schematic diagram of the structure of the semantic similarity calculation model based on question pairs provided in the embodiment of the present application. Fig. 9 As shown, the question to be matched and the candidate question are input into the model as a question pair. The input layer processes the question to be matched and the candidate question separately to generate corresponding word sets, part-of-speech sets and synonym sets. The information of the above multiple sets is input into the encoding layer. The bidirectional recurrent neural network layer in the encoding layer extracts the relationship between each word and the previous and next words. The bidirectional self-attention layer in the encoding layer determines the importance of each word in the question. In the local interaction layer, the semantic relevance calculation is performed between the words in the question to be matched and the candidate question, and the semantic relevance calculation is performed between the words in the candidate question and the question to be matched. After multi-angle analysis, the comprehensive result of the semantic relevance calculation is obtained.

[0160] After the aggregation layer extracts and aggregates the features of the output of the local interaction layer, the output layer outputs the semantic similarity between each candidate question and the question to be matched, and selects the candidate question with the highest semantic similarity as the matching result of the question to be matched.

[0161] Based on the above description of the question-answering system, it can be known that for multiple questions with similar semantics, the same answer can be used to reply. Therefore, the question matching method proposed in the embodiment of the present application can be used to determine frequently asked questions with the same semantics as the question to be matched. Therefore, the question matching method proposed in the embodiment of the present application can be used to reply to questions. Specifically, Fig.10 A flowchart of answering questions raised in the embodiments of the present application is shown in FIG. Fig.10 As shown, the method includes:

[0162] Step S301, obtaining questions to be answered.

[0163] Among them, the questions to be answered are the questions raised by users in the question-and-answer system through the human-computer interaction interface, which require the machine to answer.

[0164] Step S302: using the question matching method described in the above embodiment, determine candidate questions that match the question to be answered.

[0165] In a question-answering system based on a FAQ list, the aforementioned question matching method may be used to determine FAQs with the same semantics as the questions raised by the user from pre-stored FAQs as candidate questions.

[0166] Step S303: reply using the candidate answers corresponding to the candidate questions.

[0167] It can be understood that since the candidate question has the same semantics as the question raised by the user, the corresponding answer is also the same, and the candidate answer corresponding to the candidate question can be used to reply.

[0168] Thus, it is achieved that the answers to the pre-stored frequently asked questions are used as answers to the questions raised by the user through question matching.

[0169] In order to implement the above embodiment, the embodiment of the present application also proposes a matching device for the problem. Fig.11 A schematic diagram of the structure of a matching device for solving a problem proposed in an embodiment of the present application is shown in FIG. Fig.11 As shown, the device includes: a first acquisition module 410, a first matching module 420, a retrieval module 430, and a second matching module 440.

[0170] The first acquisition module 410 is used to acquire questions to be matched.

[0171] The questions to be matched are questions raised by users.

[0172] The first matching module 420 is used to match the questions to be matched using the dictionary tree.

[0173] Among them, the dictionary tree (trie tree), also known as the word search tree, is a tree structure. The root node does not contain any characters. Except for the root node, each node contains only one character. From the root node to a certain node, the characters passed on the path are connected to form the string corresponding to the node. All child nodes of each node contain different characters.

[0174] The retrieval module 430 is used to retrieve a plurality of candidate questions similar to the question to be matched from a preset question library when the first matching module fails to match.

[0175] When retrieving the preset question library, it is only necessary to retrieve the frequently asked questions with the same keywords as the question to be matched from it, without the need for the content forms to be exactly the same. It can be understood that the more the number of the same keywords between the frequently asked questions in the preset question library and the question to be matched, the higher the similarity of the frequently asked questions and the question to be matched in terms of content form. Therefore, in the embodiments of the present application, the retrieval results are sorted in reverse order according to the similarity, and then the first N frequently asked questions are taken as candidate questions, where N is a positive integer.

[0176] The second matching module 440 is configured to use the trained semantic similarity calculation model based on question pairs to match the question to be matched with the candidate questions.

[0177] Among them, the semantic similarity calculation model based on question pairs includes an input layer, an encoding layer, a local interaction layer, an aggregation layer, and an output layer. The input layer is used to input the word information of the question pair, the encoding layer is used to perform semantic analysis on the questions in the question pair to determine the importance of each word included in the questions and learn the structural features of the questions, the local interaction layer is used to perform semantic relevance analysis on the questions in the question pair, the aggregation layer is used to extract and aggregate the features of the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair.

[0178] Furthermore, in order to make the question matching method proposed in the embodiments of the present application applicable to different business scenarios and be continuously updated over time, a possible implementation is that the device further includes: an update module 450, configured to update the trie tree, the preset question library, and the corresponding index library.

[0179] It should be noted that when the trie tree and the preset question library are updated, the corresponding index library also needs to be updated to implement the retrieval function of the trie tree and the preset question library. In the embodiments of the present application, full-text retrieval technology is used to retrieve in the preset question library.

[0180] Furthermore, in order to enable as many questions to be matched as possible to be completed through the trie tree, a possible implementation is Fig.12 which is the structural schematic diagram of the first matching module proposed in the embodiments of the present application. As Fig.12 shown, based on Fig.11 the device structure shown, the first matching module 420 includes:

[0181] The removal sub-module 421 is configured to remove the modal particles in the question to be matched.

[0182] Among them, the modal particle refers to a function word indicating mood, which is commonly used at the end of a sentence or at the pause in a sentence to indicate various moods. Common modal particles include "ah", "ne", "ma", etc.

[0183] The unification submodule 422 is used to unify the punctuation marks in the question to be matched.

[0184] Specifically, the punctuation marks may be set to "," or " / ", that is, the function of the punctuation marks in expressing the tone of voice is removed, and the punctuation marks are only regarded as symbols of pause.

[0185] The replacement submodule 423 is used to perform synonym replacement on the question to be matched, so as to generate multiple similar questions to the question to be matched.

[0186] The first matching submodule 424 is used to use the dictionary tree to match each similar question separately.

[0187] Furthermore, in order to use the semantic similarity calculation model based on question pairs to match the questions to be matched with the candidate questions, a possible implementation method is: Fig.13 This is a schematic diagram of the structure of the second matching module proposed in the embodiment of the present application. Fig.13 As shown, based on Fig.11 The device structure shown, the second matching module 440, includes:

[0188] The pairing submodule 441 is used to form a question pair with each candidate question and the question to be matched.

[0189] The input submodule 442 is used to input the question pair into the semantic similarity calculation model based on the question pair.

[0190] The second matching submodule 443 is used to match the question to be matched with the candidate questions according to the semantic similarity corresponding to each question pair.

[0191] Furthermore, in order to train the semantic similarity calculation model based on question pairs, a possible implementation method is: Fig.14 Another structural schematic diagram of the matching device for solving the problem raised by the embodiment of the present application is as follows: Fig.14 As shown, based on Fig.11 The device structure shown in the figure further comprises:

[0192] The second acquisition module 510 is used to acquire reference question pairs and reference semantic similarities corresponding to the reference question pairs.

[0193] The reference question pair includes a first reference question and a second reference question.

[0194] The reference question pair and the reference semantic similarity refer to training examples used to train a semantic similarity calculation model based on question pairs.

[0195] The processing module 520 is used to perform word segmentation processing on the first reference question and the second reference question respectively to generate a first reference word set corresponding to the first reference question and a second reference word set corresponding to the second reference question.

[0196] The first determination module 530 is used to determine the part of speech corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference part of speech set corresponding to the first reference word set and a second reference part of speech set corresponding to the second reference word set.

[0197] The second determination module 540 is used to determine the synonyms corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference synonym set corresponding to the first reference word set and a second reference synonym set corresponding to the second reference word set.

[0198] Input module 550 is used to input the first reference word set, the second reference word set, the first reference part of speech set, the second reference part of speech set, the first reference synonym set and the second reference synonym set into the encoding layer of the semantic similarity calculation model based on question pairs.

[0199] The training module 560 is used to train the parameters of the semantic similarity calculation model based on question pairs according to the output of the output layer of the semantic similarity calculation model based on question pairs and the reference semantic similarity.

[0200] The training of model parameters mainly targets the parameters in the encoding layer, local interaction layer, aggregation layer and output layer, while the parameters of the input layer remain unchanged.

[0201] The completion module 570 is used to complete the training of the semantic similarity calculation model based on the question pair when the accuracy of the semantic similarity calculation model based on the question pair is greater than a preset threshold.

[0202] When the accuracy of the semantic similarity calculation model is greater than a preset threshold, it means that the trained model can be used to meet actual needs, and the trained semantic similarity calculation model based on question pairs can be obtained.

[0203] Furthermore, in order to enable the encoding layer to perform semantic analysis on the question in the question pair, determine the importance of each word included in the question and learn the structural characteristics of the question, a possible implementation method is that the encoding layer includes a bidirectional recurrent neural network layer, a first normalization layer and a stacked bidirectional self-attention layer.

[0204] Furthermore, in order to enable the local interaction layer to perform semantic relevance analysis on two questions in the question pair, the local interaction layer includes a bidirectional multi-angle similarity analysis layer and a second normalization layer.

[0205] It should be noted that the above description of the matching method embodiment of the problem is also applicable to the matching device of the problem in the embodiment of the present application, and will not be repeated here.

[0206] In summary, the problem matching device provided in the embodiment of the present application obtains the problem to be matched when matching the problem, and uses the dictionary tree to match the problem to be matched. If the match fails, multiple candidate questions similar to the problem to be matched are retrieved from the preset question library, and the trained semantic similarity calculation model based on the question pair is used to match the problem to be matched with the candidate question. Among them, the semantic similarity calculation model based on the question pair includes an input layer, an encoding layer, a local interaction layer, an aggregation layer and an output layer, the input layer is used to input the word information of the question pair, the encoding layer is used to perform semantic analysis on the problem in the question pair, determine the importance of each word included in the question and the structural characteristics of the learning problem, the local interaction layer is used to perform semantic relevance analysis on the two problems in the question pair, the aggregation layer is used to extract features and aggregate the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair. Thus, the semantic similarity calculation model based on the trained semantic similarity calculation model based on the question pair is used to calculate the semantic similarity of the question pair, and semantics is used as a reference factor for finding similar candidate problems, thereby improving the accuracy of determining similar problems.

[0207] In order to implement the above embodiment, the present application embodiment also proposes a question answering device, Fig.15 This is a schematic diagram of the structure of the device for answering the questions raised in the embodiment of the present application. Fig.15 As shown, the device includes: a third acquisition module 610, a third determination module 620, and a reply module 630.

[0208] The third acquisition module 610 is used to acquire questions to be answered.

[0209] Among them, the questions to be answered are the questions raised by users in the question-and-answer system through the human-computer interaction interface, which require the machine to answer.

[0210] The third determination module 620 is used to determine candidate questions that match the question to be answered using the question matching method in the aforementioned embodiment.

[0211] Specifically, frequently asked questions with the same semantics as the question raised by the user may be determined from pre-stored frequently asked questions as candidate questions.

[0212] The reply module 630 is used to reply using the candidate answers corresponding to the candidate questions.

[0213] It can be understood that since the candidate question has the same semantics as the question raised by the user, the corresponding answer is also the same, and the candidate answer corresponding to the candidate question can be used to reply.

[0214] It should be noted that the above description of the embodiment of the method for answering questions is also applicable to the device for answering questions in the embodiment of the present application, and will not be repeated here.

[0215] Thus, it is achieved that the answers to the pre-stored frequently asked questions are used as answers to the questions raised by the user through question matching.

[0216] In order to implement the above embodiment, the present application embodiment also proposes a question-answering system. Fig.16 This is a structural diagram of a question-answering system proposed in an embodiment of the present application, such as Fig.16 As shown, the question-answering system includes:

[0217] The question-and-answer interface 710 is used to receive user input content and display the generated reply content.

[0218] The distribution agent 720 is used to distribute the user's input content to the corresponding reply device according to the type of the user's input content. The reply device includes a question reply device 730, a natural language understanding device, and a natural language processing device.

[0219] The question answering device 730 is used to receive the questions to be answered sent by the distribution agent 720 and determine the corresponding answers from the preset question library 740.

[0220] In addition, the question-and-answer system also includes a data message middleware and an automatic program. The automatic program updates the preset question library at preset time intervals, specifically deleting and adding frequently asked questions in the preset question library and the corresponding index library. The data message middleware sends the message of updating the preset question library to the question reply device 730.

[0221] It should be noted that the above description of the embodiment of the device for answering questions is also applicable to the question-answering system of the embodiment of the present application and will not be repeated here.

[0222] Thus, it is achieved that the user's input content is received through the question and answer interface of the question and answer system, and the distribution agent transmits the question raised by the user to the aforementioned question answering device, and the question answering device generates the answer to the question and displays it through the question and answer interface.

[0223] In order to implement the above embodiment, the embodiment of the present application further proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are performed:

[0224] Step S101, obtaining questions to be matched.

[0225] Based on the above description of the question-answering system, it can be known that the questions to be matched are questions raised by users.

[0226] There are many ways to obtain questions raised by users, which can be achieved through a human-computer interaction interface. Users can ask questions through the human-computer interaction interface through various input methods such as keyboard input and voice input.

[0227] Step S102, using a dictionary tree to match the questions to be matched.

[0228] It should be noted that the embodiment of the present application matches questions from two aspects: characters and semantics. Specifically, a dictionary tree is used for character matching, and a semantic similarity calculation model based on question pairs is used for semantic matching.

[0229] Among them, the dictionary tree (trie tree), also known as the word search tree, is a tree structure. The root node does not contain any characters. Except for the root node, each node contains only one character. From the root node to a certain node, the characters passed on the path are connected to form the string corresponding to the node. All child nodes of each node contain different characters.

[0230] It can be understood that the embodiment of the present application stores multiple FAQs in advance, and each FAQ is filled into a dictionary tree as a string, and the nodes in the dictionary tree can be matched with the FAQ. The string corresponding to the question to be matched is split into multiple characters, and the multiple characters are matched in sequence using the dictionary tree. If a node matching the string corresponding to the question to be matched can be found in the dictionary tree, it can be determined that the question to be matched is matched with the FAQ corresponding to the node.

[0231] It should be noted that if the dictionary tree can complete the matching of the question to be matched, the character matching of the question to be matched is completed. In other words, if the question raised by the user is exactly the same as a candidate question in terms of content, then in the subsequent reply process, the answer corresponding to the candidate question can be directly used to answer the question raised by the user.

[0232] Step S103: if the matching fails, multiple candidate questions similar to the question to be matched are retrieved from a preset question library.

[0233] It should be noted that if the matching of the question to be matched is not completed in step S102, it means that there is no candidate question in the pre-stored FAQs that is exactly the same as the question raised by the user, and the question to be matched needs to be matched through semantic analysis.

[0234] It can be understood that there are a large number of frequently asked questions in the preset question library. If each frequently asked question is input into the semantic similarity calculation model together with the question to be matched, the amount of calculation will be too large and the efficiency will be very low.

[0235] In order to reduce the amount of calculation in the subsequent process, the frequently asked questions in the preset question library can be preliminarily screened. Specifically, multiple candidate questions similar to the question to be matched are retrieved from the preset question library, and then each candidate question is input into the semantic similarity calculation model together with the question to be matched, and the candidate question matching the question to be matched is determined.

[0236] Unlike the aforementioned use of a dictionary tree for matching, here when searching the preset question library, it is only necessary to retrieve the FAQs that have the same keywords as the questions to be matched, without having to be completely consistent in content form. It can be understood that the more common keywords the FAQs in the preset question library have with the questions to be matched, the higher the similarity in content form between the FAQs and the questions to be matched. Therefore, the embodiment of the present application sorts the search results in reverse order by similarity, and then takes the first N FAQs as candidate questions, where N is a positive integer.

[0237] Step S104: Use the trained semantic similarity calculation model based on question pairs to match the question to be matched with the candidate questions.

[0238] Among them, the semantic similarity calculation model based on question pairs includes an input layer, an encoding layer, a local interaction layer, an aggregation layer and an output layer. The input layer is used to input the word information of the question pair, the encoding layer is used to perform semantic analysis on the questions in the question pair, determine the importance of each word included in the question and learn the structural characteristics of the question, the local interaction layer is used to perform semantic relevance analysis on the two questions in the question pair, the aggregation layer is used to extract and aggregate features of the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair.

[0239] After completing the preliminary screening of candidate questions in terms of content form in step S103, it is necessary to match the questions to be matched in terms of semantics.

[0240] The embodiment of the present application adopts a semantic similarity calculation model based on question pairs to match the questions to be matched with the candidate questions.

[0241] One possible implementation method is to form a question pair with each candidate question and the question to be matched, input the question pair into a semantic similarity calculation model based on the question pair, and match the question to be matched with the candidate question according to the semantic similarity corresponding to each question pair.

[0242] It should be noted that the semantic similarity calculation model based on question pairs provided in the embodiment of the present application is a deep learning sentence pair model technology. With the continuous development of artificial intelligence and deep learning technology in recent years, more and more natural language processing tasks are processed using deep learning architectures, and the more common ones are neural network language models, recurrent neural network models, sentence pair models, etc. Specifically, the deep learning sentence pair model determines the relationship between sentences by identifying the semantic information encoded by the source sentence and the target sentence.

[0243] When using the deep learning sentence pair model technology, a set of training examples is first given, each example is a triple (source sentence, target sentence, and the relationship between the source sentence and the target sentence). By training a deep learning model (such as a recurrent neural network model), the probability of predicting the relationship between any two sentences is learned.

[0244] It should be particularly noted that the word information input into the input layer of the semantic similarity calculation model based on question pairs provided in the embodiment of the present application includes a word set generated after word segmentation of the question, a part-of-speech set generated after identifying the part of speech of each word in the word set, and a synonym set generated after synonym identification for each word in the word set.

[0245] After semantic analysis of the words inputted by the input layer, the encoding layer learns the context information of each word in the question and the structural feature information of the question from the multiple words generated after word segmentation of the question. Specifically, the information of the words on the left and right sides of each word in the question can be learned through a bidirectional recurrent neural network, and the semantic relevance of each word in the question is calculated with other words, so as to determine the importance of each word included in the question and learn the structural features of the question.

[0246] After the encoding layer determines the importance of each word included in the two questions in the question pair and the structural characteristics of the learning questions, the local interaction layer performs semantic relevance analysis on the two questions in the question pair on this basis. It can be understood that when the local interaction layer performs semantic relevance analysis, the more important the word is, the greater the impact on the semantic relevance analysis. Specifically, each word in question one can be weighted to calculate the semantic relevance with question two, with the weight being the importance of the word, so as to obtain the semantic relevance of each word in question one with question two. Through similar steps, the semantic relevance of each word in question two with question one can be obtained.

[0247] In order to reduce the amount of subsequent semantic similarity calculations, the aggregation layer uses the maximum / minimum pooling method to extract features from the output of the local interaction layer, and uses the splicing method to aggregate the extracted features. The output layer calculates the semantic similarity of the question pair based on the aggregated features.

[0248] In order to enable the encoding layer to achieve the above functions, one possible implementation method is that the encoding layer includes a bidirectional recurrent neural network layer, a first normalization layer and a stacked bidirectional self-attention layer.

[0249] It should be noted that the bidirectional recurrent neural network is a special type of recurrent neural network. The recurrent neural network is a recurrent neural network that takes sequence data as input, recursively in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain. The difference is that the recurrent neural network performs recursive calculations in one direction (usually the direction of sequence evolution), while the bidirectional recurrent neural network performs recursive calculations in two directions (usually the direction of sequence evolution and the opposite direction).

[0250] It can be understood that natural language, as a kind of sequence data, usually has strong correlation, and the bidirectional recurrent neural network can encode the data by combining the previous and next data in the sequence, thereby improving the rationality of data encoding.

[0251] The first normalization layer can normalize the data to facilitate subsequent data processing.

[0252] The stacked bidirectional self-attention layer determines the importance of each word in the question from the natural language through the self-attention mechanism, thereby providing weights for the subsequent calculation of the semantic relevance between the word and the question.

[0253] In order to enable the local interaction layer to achieve the above functions, a possible implementation method is that the local interaction layer includes a bidirectional multi-angle similarity analysis layer and a second normalization layer.

[0254] It should be noted that the bidirectional multi-angle similarity analysis layer provided in the embodiment of the present application first calculates the semantic relevance between words and questions from multiple angles, and after comprehensive processing of the calculation results, a comprehensive result of the semantic relevance calculation can be obtained.

[0255] A first possible perspective is to calculate the semantic relevance between the word and each word included in the question, and use the calculation result as the semantic relevance between the word and the question.

[0256] A second possible perspective is to calculate the semantic relevance between the word and the most important word in the question, and use the calculation result as the semantic relevance between the word and the question.

[0257] A third possible perspective is to first generate semantic features of the question based on the importance of multiple words included in the question, and then calculate the semantic relevance between the words and the semantic features of the question as the semantic relevance between the words and the question.

[0258] It can be understood that in the process of generating the comprehensive results of the semantic relevance calculation, the multiple angles used may include any one of the above-mentioned possible angles, and may also include other possible angles, which is not limited in the embodiments of the present application.

[0259] In summary, the electronic device provided in the embodiment of the present application obtains the problem to be matched when performing question matching, and uses the dictionary tree to match the problem to be matched. If the match fails, multiple candidate questions similar to the problem to be matched are retrieved from the preset question library, and the trained semantic similarity calculation model based on the question pair is used to match the problem to be matched with the candidate question. Among them, the semantic similarity calculation model based on the question pair includes an input layer, a coding layer, a local interaction layer, an aggregation layer and an output layer, the input layer is used to input the word information of the question pair, the coding layer is used to perform semantic analysis on the problem in the question pair, determine the importance of each word included in the question and the structural characteristics of the learning problem, the local interaction layer is used to perform semantic relevance analysis on the two problems in the question pair, the aggregation layer is used to extract features and aggregate the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair. Thus, the semantic similarity calculation model based on the trained semantic similarity calculation model based on the question pair is used to calculate the semantic similarity of the question pair, and semantics is used as a reference factor for finding similar candidate questions, which improves the accuracy of determining similar questions.

[0260] In addition, in order to make the electronic device proposed in the embodiment of the present application applicable to different business scenarios and continuously updated over time, before obtaining the question to be matched in step S101, the method also includes: updating the dictionary tree and the preset question library, and the corresponding index library.

[0261] Based on the foregoing description, it can be known that the dictionary tree and the preset question library in the embodiment of the present application are both a storage form of frequently asked questions. In different business scenarios, the required frequently asked questions are different, and the frequently asked questions will also change over time. Therefore, before obtaining the questions to be matched, the dictionary tree and the preset question library can be updated first to ensure the accuracy of the question matching method in the embodiment of the present application.

[0262] It should be noted that when the dictionary tree and the preset question library are updated, the corresponding index library also needs to be updated to realize the retrieval function of the dictionary tree and the preset question library. The embodiment of the present application uses full-text retrieval technology to search in the preset question library.

[0263] Full-text retrieval technology is essentially a search engine technology. Search engine technology refers to the technology of automatically collecting information from the Internet and providing queries to users after collation. Search engine technology includes steps such as web crawlers, web indexing, web retrieval, and search result ranking. Search engine technology supports dynamic updates of web indexing and near-real-time search, and is currently applied in a large number of retrieval scenarios.

[0264] It can be understood that since the information on the Internet is constantly being updated, search engine technology can achieve near-real-time retrieval of network information. When using full-text retrieval technology to retrieve a preset question library, it is possible to ensure near-real-time search of the preset question library through steps such as data crawlers, data indexing, data retrieval, and search result ranking under the condition that the preset question library is constantly updated.

[0265] Based on the foregoing description, it can be known that in step S102, using a trie to match the question to be matched can quickly complete the matching of the question to be matched. When using a semantic similarity calculation model to match the question to be matched, the calculation amount is large and the efficiency is low.

[0266] In order to enable as many questions to be matched as possible to be completed by the trie, a possible implementation method is to unify different expressions of the same question. Step S102, using a trie to match the question to be matched, includes:

[0267] Step S11, removing the modal particles in the question to be matched.

[0268] Among them, modal particles refer to function words that express mood, and are commonly used at the end of a sentence or at pauses in a sentence to express various moods. Common modal particles include "ah", "ne", "ma", etc.

[0269] Step S12, unifying the punctuation marks in the question to be matched.

[0270] Specifically, the punctuation marks can be set to ",", or " / ", that is, the function of the punctuation marks to express mood is removed, and they are only regarded as symbols for pauses.

[0271] It should be noted that since multiple similar questions of the question to be matched need to be generated in step S13, considering that different users have different usage habits of modal particles and punctuation marks, and modal particles and punctuation marks have no impact on the essential content of the question, the modal particles can be removed first, and the punctuation marks can be uniformly converted into symbols for pauses for subsequent processing.

[0272] Step S13, performing synonym replacement on the question to be matched to generate multiple similar questions of the question to be matched.

[0273] Step S14, using the dictionary tree to match each similar question separately.

[0274] It can be understood that in order to match the questions to be matched with the FAQs stored in the dictionary tree, the modal particles of the FAQs are removed and the punctuation marks are processed uniformly when the dictionary tree is generated. When using the dictionary tree, the same operation is performed on the questions to be matched to increase the possibility of successful matching between the questions to be matched and the FAQs.

[0275] In addition, considering that there are many ways to express matching questions in natural language, but the substantive content is the same, if any of these expressions is pre-stored in the dictionary tree, the dictionary tree can be used to complete the matching of the matching questions. Therefore, synonym replacement can be performed on the matching questions to generate multiple similar questions with different expressions from the matching questions. Use the dictionary tree to match multiple similar questions. If any similar question is successfully matched, it can be considered that the matching question is successfully matched.

[0276] In the electronic device provided in the embodiment of the present application, it is necessary to use the trained semantic similarity calculation model based on question pairs to match the questions to be matched with the candidate models. In order to train the semantic similarity calculation model based on question pairs, one possible implementation method is that when training the semantic similarity calculation model based on question pairs, the electronic device performs the following steps:

[0277] Step S201, obtaining a reference question pair and a reference semantic similarity corresponding to the reference question pair.

[0278] The reference question pair includes a first reference question and a second reference question.

[0279] The reference question pair and the reference semantic similarity refer to training examples used to train a semantic similarity calculation model based on question pairs.

[0280] Based on the above description of the deep learning sentence pair model technology, it can be known that the reference question pair and the reference semantic similarity form a triple in the training sample. The first reference question included in the reference question pair can be used as the source sentence, and the second reference question can be used as the target sentence. Then the reference semantic similarity is the relationship between the source sentence and the target sentence.

[0281] It should be noted that the data in the FAQ list is used as a reference question pair and a reference semantic similarity in the embodiments of the present application. However, there may be an imbalance in the training samples. Specifically, if the answers to two questions are the same, the question pair formed by the two questions can be used as a positive training example, and the reference semantic similarity is set higher, such as to 1. If the answers to the two questions are different, the question pair formed by the two questions can be used as a negative training example, and the reference speech similarity is set lower, such as to 0. It can be understood that the number of positive training examples that can be formed in the FAQ list is much smaller than the number of negative training examples, so that the influence of positive training examples on model training is too small.

[0282] In order to balance the impact of training positive examples and training negative examples on model training, the present application embodiment proposes a possible implementation method, which includes:

[0283] Step S21, performing data cleaning on the FAQ list.

[0284] Specifically, in the FAQ list, if there is data with similar questions and similar corresponding answers, the questions and corresponding answers can be merged, and if there is data with the same questions but different corresponding answers, the answers can be untied. In addition, English questions in the FAQ list can also be deleted.

[0285] Step S22, determining multiple questions corresponding to each answer.

[0286] Step S23, generating training positive examples according to the multiple questions corresponding to each answer.

[0287] Based on the above description, it can be known that the number of training positive examples that can be generated in the FAQ list is much smaller than the number of training negative examples, so it is necessary to give priority to generating training positive examples.

[0288] The characteristic of a training positive example is that the answers to two questions are the same, so a training positive example can be generated from multiple questions corresponding to the answers.

[0289] In addition, considering that different answers correspond to different numbers of questions, the number of training positive examples corresponding to each answer can be set to be similar in advance.

[0290] Step S24, generating training counterexamples according to questions corresponding to different answers.

[0291] It should be understood that in order to reduce the impact of counterexamples on model training, a more sophisticated strategy is adopted for the generation of training counterexamples.

[0292] Specifically, different answers may correspond to questions with similar wordings. In order to enable the trained model to distinguish between questions with similar wordings, it is necessary to use such situations as training counterexamples as much as possible.

[0293] Secondly, the number of training negative examples needs to be controlled to make it close to the number of training positive examples.

[0294] In order to avoid the same problem appearing in different training counterexamples, it is necessary to mark the selected problems during the selection of training counterexamples.

[0295] Through the above training example generation method, more practical reference question pairs and reference semantic similarities can be obtained from the FAQ list.

[0296] In actual use, part of the training samples generated above can be used as a development data set, and part can be used as a training data set. For example, 10% of the training samples can be used for development, and 90% of the training samples can be used for training, which is not limited in the embodiments of the present application.

[0297] Step S202 , performing word segmentation processing on the first reference question and the second reference question respectively, so as to generate a first reference word set corresponding to the first reference question, and a second reference word set corresponding to the second reference question.

[0298] Step S203 , determining the part of speech corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference part of speech set corresponding to the first reference word set and a second reference part of speech set corresponding to the second reference word set.

[0299] Step S204 , determining the synonyms corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference synonym set corresponding to the first reference word set and a second reference synonym set corresponding to the second reference word set.

[0300] Step S205 , inputting the first reference word set, the second reference word set, the first reference part-of-speech set, the second reference part-of-speech set, the first reference synonym set and the second reference synonym set into the encoding layer of the semantic similarity calculation model based on the question pair.

[0301] Based on the above description of the semantic similarity calculation model based on question pairs, it can be known that the word information input by the input layer includes a word set, a part-of-speech set, and a synonym set. Accordingly, during the model training process, the input layer needs to perform word segmentation, part-of-speech recognition, and synonym recognition processing on the first reference question and the second reference question in the input reference question pair.

[0302] It should be noted that, after being processed by the input layer, the input first reference question and the second reference question obtain a first reference word set, a first reference part-of-speech set and a first synonym set corresponding to the first reference question, and a second reference word set, a second reference part-of-speech set and a second reference synonym set corresponding to the second reference question.

[0303] Step S206 , training parameters of the semantic similarity calculation model based on question pairs according to the output of the semantic similarity calculation model based on question pairs and the reference semantic similarity.

[0304] It should be noted that, based on the above description, the function of the input layer is to convert the questions in the question pair into a word set, a part of speech set, and a synonym set suitable for model processing. Therefore, during the model training process, the parameters of the input layer remain unchanged.

[0305] The training of model parameters mainly focuses on the parameters in the encoding layer, local interaction layer, aggregation layer and output layer.

[0306] Step S207, when the accuracy of the semantic similarity calculation model based on the question pair is greater than a preset threshold, the training of the semantic similarity calculation model based on the question pair is completed.

[0307] When the accuracy of the semantic similarity calculation model is greater than a preset threshold, it means that the trained model can be used to meet actual needs, and the trained semantic similarity calculation model based on question pairs can be obtained.

[0308] Thus, the training of the semantic similarity calculation model based on question pairs is realized.

[0309] In order to implement the above embodiment, the present application also provides an electronic device, Fig.17 Schematic diagram of an electronic device proposed in an embodiment of the present application. Fig.17 As shown, it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the following steps are performed:

[0310] Step S301, obtaining questions to be answered.

[0311] Among them, the questions to be answered are the questions raised by users in the question-and-answer system through the human-computer interaction interface, which require the machine to answer.

[0312] Step S302: using the aforementioned question matching method, determine candidate questions that match the question to be answered.

[0313] In a question-answering system based on a FAQ list, the aforementioned question matching method may be used to determine FAQs with the same semantics as the questions raised by the user from pre-stored FAQs as candidate questions.

[0314] Step S303: reply using the candidate answers corresponding to the candidate questions.

[0315] It can be understood that since the candidate question has the same semantics as the question raised by the user, the corresponding answer is also the same, and the candidate answer corresponding to the candidate question can be used to reply.

[0316] Thus, it is achieved that the answers to the pre-stored frequently asked questions are used as answers to the questions raised by the user through question matching.

[0317] In order to implement the above embodiments, the embodiments of the present application also propose a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer executes the matching method of the problem in the above embodiments.

[0318] In order to implement the above embodiments, the embodiments of the present application also propose a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method for answering the questions in the above embodiments.

[0319] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0320] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0321] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0322] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0323] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A problem matching method, characterized in that: include: Get the questions to be matched; Use a dictionary tree to match the questions to be matched; If the matching fails, multiple candidate questions similar to the question to be matched are retrieved from a preset question library; Using the trained semantic similarity calculation model based on question pairs, matching the question to be matched with the candidate question; Among them, the semantic similarity calculation model based on question pairs includes an input layer, an encoding layer, a local interaction layer, an aggregation layer and an output layer. The input layer is used to input the word information of the question pair, the encoding layer is used to perform semantic analysis on the questions in the question pair, determine the importance of each word included in the question and learn the structural features of the question, the local interaction layer is used to perform semantic relevance analysis on the two questions in the question pair, the aggregation layer is used to extract and aggregate features of the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair.

2. The method according to claim 1, characterized in that Before obtaining the question to be matched, the method further includes: The dictionary tree, the preset question library, and the corresponding index library are updated.

3. The method according to claim 1, characterized in that The using a dictionary tree to match the to-be-matched questions includes: Removing modal particles from the question to be matched; Unifying the punctuation marks in the question to be matched; Performing synonym replacement on the question to be matched to generate multiple similar questions of the question to be matched; Each of the similar questions is matched respectively using the dictionary tree.

4. The method according to claim 1, characterized in that: The step of using the trained semantic similarity calculation model based on question pairs to match the question to be matched with the candidate question includes: Each of the candidate questions is combined with the question to be matched to form a question pair; Inputting the question pair into the semantic similarity calculation model based on the question pair; The question to be matched is matched with the candidate question according to the semantic similarity corresponding to each question pair.

5. The method according to any one of claims 1 to 4, characterized in that The semantic similarity calculation model based on question pairs is trained by the following steps: Obtaining a reference question pair and a reference semantic similarity corresponding to the reference question pair; wherein the reference question pair includes a first reference question and a second reference question; Performing word segmentation processing on the first reference question and the second reference question respectively to generate a first reference word set corresponding to the first reference question and a second reference word set corresponding to the second reference question; Determine the part of speech corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference part of speech set corresponding to the first reference word set and a second reference part of speech set corresponding to the second reference word set; Determine a synonym corresponding to each of the reference words in the first reference word set and the second reference word set to generate a first reference synonym set corresponding to the first reference word set and a second reference synonym set corresponding to the second reference word set; Inputting the first reference word set, the second reference word set, the first reference part-of-speech set, the second reference part-of-speech set, the first reference synonym set and the second reference synonym set into the encoding layer of the question pair-based semantic similarity calculation model; Training parameters of the semantic similarity calculation model based on question pairs according to the output of the output layer of the semantic similarity calculation model based on question pairs and the reference semantic similarity; When the accuracy of the semantic similarity calculation model based on the question pair is greater than a preset threshold, the training of the semantic similarity calculation model based on the question pair is completed.

6. The method according to any one of claims 1 to 4, characterized in that The encoding layer includes a bidirectional recurrent neural network layer, a first normalization layer, and a stacked bidirectional self-attention layer.

7. The method according to any one of claims 1 to 4, characterized in that The local interaction layer includes a bidirectional multi-angle similarity analysis layer and a second normalization layer.

8. A method for answering a question, characterized in that: include: Get questions waiting for answer; Using the question matching method according to any one of claims 1 to 7, determining a candidate question that matches the question to be answered; Reply using the candidate answer corresponding to the candidate question.

9. A matching device for a problem, characterized in that, include: A first acquisition module is used to acquire questions to be matched; A first matching module, used for matching the to-be-matched questions using a dictionary tree; A retrieval module, configured to retrieve a plurality of candidate questions similar to the question to be matched from a preset question library when the first matching module fails to match; A second matching module, used for matching the question to be matched with the candidate question using the trained semantic similarity calculation model based on question pairs; Among them, the semantic similarity calculation model based on question pairs includes an input layer, an encoding layer, a local interaction layer, an aggregation layer and an output layer. The input layer is used to input the word information of the question pair, the encoding layer is used to perform semantic analysis on the questions in the question pair, determine the importance of each word included in the question and learn the structural features of the question, the local interaction layer is used to perform semantic relevance analysis on the two questions in the question pair, the aggregation layer is used to extract and aggregate features of the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair.

10. The device according to claim 9, characterized in that The device also includes: An updating module is used to update the dictionary tree, the preset question library, and the corresponding index library.

11. The device according to claim 9, characterized in that The first matching module includes: A removal submodule, used for removing modal particles in the question to be matched; A unification submodule, used for unifying the punctuation marks in the question to be matched; A replacement submodule, used for performing synonym replacement on the question to be matched, so as to generate a plurality of similar questions to the question to be matched; The first matching submodule is used to use the dictionary tree to match each of the similar questions respectively.

12. The device according to claim 9, characterized in that The second matching module includes: A pairing submodule, used for forming a question pair with each candidate question and the question to be matched; An input submodule, used for inputting the question pair into the semantic similarity calculation model based on the question pair; The second matching submodule is used to match the question to be matched with the candidate question according to the semantic similarity corresponding to each question pair.

13. The device according to any one of claims 9 to 12, characterized in that: The device also includes: A second acquisition module is used to acquire a reference question pair and a reference semantic similarity corresponding to the reference question pair; wherein the reference question pair includes a first reference question and a second reference question; a processing module, configured to perform word segmentation processing on the first reference question and the second reference question respectively, so as to generate a first reference word set corresponding to the first reference question and a second reference word set corresponding to the second reference question; A first determination module, configured to determine the part of speech corresponding to each reference word in the first reference word set and the second reference word set, so as to generate a first reference part of speech set corresponding to the first reference word set and a second reference part of speech set corresponding to the second reference word set; A second determination module, configured to determine a synonym corresponding to each of the reference words in the first reference word set and the second reference word set, so as to generate a first reference synonym set corresponding to the first reference word set and a second reference synonym set corresponding to the second reference word set; An input module, configured to input the first reference word set, the second reference word set, the first reference part-of-speech set, the second reference part-of-speech set, the first reference synonym set, and the second reference synonym set into the encoding layer of the question pair-based semantic similarity calculation model; A training module, used for training parameters of the semantic similarity calculation model based on question pairs according to the output of the output layer of the semantic similarity calculation model based on question pairs and the reference semantic similarity; The completion module is used to complete the training of the semantic similarity calculation model based on the question pair when the accuracy of the semantic similarity calculation model based on the question pair is greater than a preset threshold.

14. The device according to any one of claims 9 to 12, characterized in that The encoding layer includes a bidirectional recurrent neural network layer, a first normalization layer, and a stacked bidirectional self-attention layer.

15. The device according to any one of claims 9 to 12, characterized in that The local interaction layer includes a bidirectional multi-angle similarity analysis layer and a second normalization layer.

16. A question answering device, characterized in that: The device comprises: The third acquisition module is used to obtain questions to be answered; A third determination module, configured to determine a candidate question matching the question to be answered using the question matching method according to any one of claims 1 to 7; The reply module is used to reply using the candidate answers corresponding to the candidate questions.

17. A question-answering system, characterized in that: The system comprises: The question-and-answer interface is used to receive user input and display the generated responses; A distribution agent, used for distributing the user's input content to a corresponding reply device according to the type of the user's input content; The question replying device as claimed in claim 16 is used to receive the questions to be replied sent by the distribution agent and determine the corresponding answers from a preset question library.

18. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are performed: Get the questions to be matched; Use a dictionary tree to match the questions to be matched; If the matching fails, multiple candidate questions similar to the question to be matched are retrieved from a preset question library; Using the trained semantic similarity calculation model based on question pairs, matching the question to be matched with the candidate question; Among them, the semantic similarity calculation model based on question pairs includes an input layer, an encoding layer, a local interaction layer, an aggregation layer and an output layer. The input layer is used to input the word information of the question pair, the encoding layer is used to perform semantic analysis on the questions in the question pair, determine the importance of each word included in the question and learn the structural features of the question, the local interaction layer is used to perform semantic relevance analysis on the two questions in the question pair, the aggregation layer is used to extract and aggregate features of the output of the local interaction layer, and the output layer is used to calculate the semantic similarity of the question pair.

19. The electronic device according to claim 18, characterized in that: Before the electronic device obtains the question to be matched, the following steps are further performed: The dictionary tree, the preset question library, and the corresponding index library are updated.

20. The electronic device according to claim 18, characterized in that The electronic device uses a dictionary tree to match the questions to be matched, specifically including the following steps: Removing modal particles from the question to be matched; Unifying the punctuation marks in the question to be matched; Performing synonym replacement on the question to be matched to generate multiple similar questions of the question to be matched; Each of the similar questions is matched respectively using the dictionary tree.

21. The electronic device according to claim 18, characterized in that The electronic device uses the trained semantic similarity calculation model based on question pairs to match the question to be matched with the candidate question, specifically including the following steps: Each of the candidate questions is combined with the question to be matched to form a question pair; Inputting the question pair into the semantic similarity calculation model based on the question pair; The question to be matched is matched with the candidate question according to the semantic similarity corresponding to each question pair.

22. The electronic device according to any one of claims 18 to 21, characterized in that: When training the semantic similarity calculation model based on question pairs, the electronic device performs the following steps: Obtaining a reference question pair and a reference semantic similarity corresponding to the reference question pair; wherein the reference question pair includes a first reference question and a second reference question; Performing word segmentation processing on the first reference question and the second reference question respectively to generate a first reference word set corresponding to the first reference question and a second reference word set corresponding to the second reference question; Determine the part of speech corresponding to each reference word in the first reference word set and the second reference word set to generate a first reference part of speech set corresponding to the first reference word set and a second reference part of speech set corresponding to the second reference word set; Determine a synonym corresponding to each of the reference words in the first reference word set and the second reference word set to generate a first reference synonym set corresponding to the first reference word set and a second reference synonym set corresponding to the second reference word set; Inputting the first reference word set, the second reference word set, the first reference part-of-speech set, the second reference part-of-speech set, the first reference synonym set and the second reference synonym set into the encoding layer of the question pair-based semantic similarity calculation model; Training parameters of the semantic similarity calculation model based on question pairs according to the output of the output layer of the semantic similarity calculation model based on question pairs and the reference semantic similarity; When the accuracy of the semantic similarity calculation model based on the question pair is greater than a preset threshold, the training of the semantic similarity calculation model based on the question pair is completed.

23. The electronic device according to any one of claims 18 to 21, characterized in that: The encoding layer includes a bidirectional recurrent neural network layer, a first normalization layer, and a stacked bidirectional self-attention layer.

24. The electronic device according to any one of claims 18 to 21, characterized in that: The local interaction layer includes a bidirectional multi-angle similarity analysis layer and a second normalization layer.

25. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are performed: Get questions waiting for answer; Using the question matching method according to any one of claims 1 to 7, determining a candidate question that matches the question to be answered; Reply using the candidate answer corresponding to the candidate question.

26. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the problem matching method according to any one of claims 1 to 7.

27. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method for answering questions as claimed in claim 8.

Images