Question and answer method, device and computer readable storage medium
By using word segmentation models in the question-answer system to obtain candidate questions and answers and filter target answers based on text similarity, the problem of low answer matching in the existing question-and-answer system is solved, improving the accuracy and matching of answers.
Patent Information
- Application Number
- CN202011261951.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-12
- Publication Date
- 2025-03-04
- Estimated Expiration
- 2040-11-12
AI Technical Summary
The question-and-answer mechanism in existing question-and-answer systems generally fuzzy searches from a very large number of question-and-answer pairs, resulting in a low degree of matching the recalled answers with the user's ideal answers.
By obtaining user questions, N candidate questions matching the first keyword of the user question and their corresponding candidate answers are obtained based on the preset word segmentation model, the text similarity between the user question and the candidate question is obtained, and M target answers are determined from the candidate answers based on the similarity.
Improve the accuracy of the recalled answers, making the answers more matched with user questions, and reducing recalls of irrelevant answers.
Smart Images

Figure CN114490957B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a question-answering method, device, and computer-readable storage medium. Background Art
[0002] The main function of the question-answering system (QA System) is to automatically answer questions raised by users. It is a type of dialogue system (also known as chatbot). It interacts with the system through language and text to get the answers people want. Since the question-answering system can meet people's needs for fast and accurate information acquisition, it has attracted more and more attention. However, the question-answering mechanism in the existing question-answering system generally performs fuzzy retrieval from a very large number of text segments or web page information, resulting in a low match between the recalled answers and the user's questions. Summary of the invention
[0003] Embodiments of the present invention provide a question-answering method, device, and computer-readable storage medium to solve the problem that the question-answering mechanism in the existing question-answering system generally performs defuzzy retrieval from a very large amount of question-answer pair data, resulting in a low match between the recalled answers and the user's ideal answers.
[0004] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:
[0005] In a first aspect, an embodiment of the present invention provides a question-answering method, including:
[0006] Get user questions;
[0007] Based on a preset word segmentation model, obtain N candidate questions matching the first keyword of the user question and candidate answers corresponding to the N candidate questions, where N is a positive integer;
[0008] Obtaining text similarity between the user question and the N candidate questions;
[0009] Based on the text similarity, M target answers are determined from the candidate answers corresponding to the N candidate questions, where M is a positive integer less than N.
[0010] In a second aspect, an embodiment of the present invention further provides a question-answering device, including:
[0011] The first acquisition module is used to obtain user questions;
[0012] A second acquisition module is used to acquire N candidate questions matching the first keyword of the user question and candidate answers corresponding to the N candidate questions based on a preset word segmentation model, wherein N is a positive integer;
[0013] A third acquisition module is used to obtain the text similarity between the user question and the N candidate questions;
[0014] A determination module is used to determine M target answers from the candidate answers corresponding to the N candidate questions based on the text similarity, wherein M is a positive integer less than N.
[0015] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the question-and-answer method of the first aspect are implemented.
[0016] In the embodiment of the present invention, the user question is obtained; based on the preset word segmentation model, N candidate questions matching the first keyword of the user question and candidate answers corresponding to the N candidate questions are obtained, wherein N is a positive integer; the text similarity between the user question and the N candidate questions is obtained; based on the text similarity, M target answers are determined from the candidate answers corresponding to the N candidate questions, wherein M is a positive integer less than N. In this way, the questions in the preset question-answer pair library can be initially sorted based on the keywords of the user question, N candidate questions are obtained, and then re-sorted based on the text similarity between the user question and the candidate questions, and M target answers are determined, so that the recalled answers are more accurate and more matched with the user question. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 One of the flow charts of the question-answering method provided in an embodiment of the present invention;
[0018] Figure 2 The second flowchart of the question-and-answer method provided in the embodiment of the present application;
[0019] Figure 3 Flow chart 3 of the question-and-answer method provided in the embodiment of the present application;
[0020] Figure 4 Flow chart 4 of the question-and-answer method provided in the embodiment of the present application;
[0021] Figure 5 A schematic diagram of the structure of a question-and-answer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0023] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0024] The question-and-answer method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.
[0025] See also Figure 1 , Figure 1 This is one of the flow charts of the question-answering method provided by the embodiment of the present invention. Figure 1 As shown, the question-answering method includes the following steps:
[0026] Step S100, obtaining user questions.
[0027] Specifically, the method of obtaining the user's question may be to obtain the user's voice input question through an audio input device, or to obtain the user's text input question through a user interface. If the question obtained is in voice form, the voice question can be converted into a text question through voice recognition technology, thereby facilitating the question-answering device to analyze and process the user's question.
[0028] Step S200: Based on a preset word segmentation model, N candidate questions matching the first keyword of the user's question and candidate answers corresponding to the N candidate questions are obtained, where N is a positive integer.
[0029] The above-mentioned preset word segmentation model refers to a model with word segmentation function preset by the user. The preset word segmentation model can be any one of models such as N-gram model, Hidden Markov Model (HMM for short), Conditional Random Field (CRF for short) model, etc., and the present invention does not make specific limitations. In an optional embodiment, a conditional random field model is used as the word segmentation model. The conditional random field model is a problem of calculating the conditional probability and the corresponding mathematical expectation given the input sequence x and the output sequence y of the conditional random field. Therefore, when training the conditional random field model, the learning method can include any one of methods such as maximum likelihood estimation, regularized maximum likelihood estimation, improved iterative scaling method IIS, gradient descent method, and quasi-Newton method. When training the conditional random field model, it is also necessary to segment the trained corpus and perform part-of-speech tagging on the segmented words. For example, the BEMS tagging method is used to tag the words, where B represents the beginning of the word, E represents the end of the word, M represents the middle of the word, and S represents a single-character word.
[0030] Based on the preset word segmentation model, the obtained user question can be segmented to obtain the word sequence of the user question, and the pause words in the word sequence, such as "de", "le", "shi", etc., are removed, thereby obtaining the first keywords in the user question. Then, the first keywords are matched with multiple first questions in the preset Q&A pair library to obtain the matching degrees between the first keywords and the multiple first questions in the preset Q&A pair library, and the N questions with the highest matching degrees are used as the candidate questions for the user question.
[0031] It should be noted that the first keyword here can be one keyword or multiple keywords, which can vary according to the different user questions obtained. The N candidate questions can be one or multiple, and the present application does not make specific limitations. In an optional embodiment, any number of candidate questions such as 5, 10, or 20 can be obtained. In this way, through the matching degree with the first keyword, multiple candidate Q&A pairs with the highest matching degree with the user question can be selected from the massive preset Q&A pair library, which is convenient for directly determining the target question with the highest similarity to the user question from the candidate questions subsequently, thereby reducing the similarity calculation range and accelerating the efficiency of answer recall.
[0032] Step S300, obtain the text similarity between the user question and the N candidate questions.
[0033] After determining the N candidate questions, the text similarity between the user question and the N candidate questions can be calculated respectively to determine the text similarity between the user question and the N candidate questions. The text similarity calculation here can be implemented by using any algorithm such as TF-IDF algorithm, BM25 algorithm, simhash algorithm, etc., and the present application does not make limitations.
[0034] Step S400: Determine M target answers from the candidate answers corresponding to N candidate questions based on text similarity, where M is a positive integer less than N.
[0035] Sort the text similarities between the user question and the N candidate questions, and use the candidate answers corresponding to the top M candidate questions in the sorting as the target answers and return them to the user. Here, M is a positive integer less than N.
[0036] In this embodiment, the questions in the preset Q&A pair library can be initially sorted based on the keywords of the user question to obtain N candidate questions, and then re-sorted based on the text similarity between the user question and the candidate questions to determine M target answers, so that the recalled answers are more accurate and have a higher matching degree with the user question.
[0037] Further, refer to Figure 2 , Figure 2 which is the second flowchart of the Q&A method provided by the embodiment of the present application. Based on Figure 1 the embodiment shown, the above step S200: Obtain N candidate questions that match the first keyword of the user question and the candidate answers corresponding to the N candidate questions based on a preset word segmentation model, specifically including the following steps:
[0038] Step S201: Segment the user question based on a preset word segmentation model to obtain the first keyword of the user question.
[0039] Based on a preset word segmentation model, the obtained user question can be segmented to obtain the word sequence of the user question, and the pause words in the word sequence, such as "de", "le", "shi", etc., are removed, thereby obtaining the first keyword in the user question. It should be noted that the first keyword here can be one keyword or multiple keywords, which can vary according to the different user questions obtained. For example, assume the obtained user question is "How to access the OneNET platform using the HTTP protocol", at this time, through the preset word segmentation model, the word sequence can be obtained: adopt / HTTP protocol / how / to access / OneNET platform, and the first keyword in the user question, such as "HTTP protocol", "OneNET platform", etc., can be determined.
[0040] Step S202: Calculate the matching degrees between the first keyword and multiple first questions in the preset Q&A pair library respectively to obtain multiple matching degrees.
[0041] Among them, the above first question is any question in the preset Q&A pair library. Multiple Q&A pairs are pre-stored in the preset Q&A pair library, and each Q&A pair includes a first question and the answer corresponding to the first question.
[0042] Similarly, based on the preset word segmentation library, multiple first questions in the preset question-answer pair library can be word segmented, and then the first keyword can be compared with the word sequences of the multiple first questions respectively, and the matching degree between the first keyword and the multiple first questions can be calculated to obtain multiple matching degrees. For example, assuming that there are 500 question-answer pairs in the preset question-answer pair library, it is necessary to calculate the matching degree between the user question and the first question of the 500 question-answer pairs respectively, thereby obtaining 500 matching degrees.
[0043] It should be noted that the method for calculating the degree of match can be implemented by TF-IDF algorithm, BM25 algorithm, simhash algorithm, or other methods, which is not limited in this application. The above-mentioned N candidate questions can be one or more, which is not specifically limited in this application. In an optional embodiment, any number of candidate questions such as 5, 10 or 20 can be obtained. Since the number of candidate questions is much smaller than the number of first questions in the preset question and answer library, when determining the target question later, it only needs to be determined from the candidate questions, thereby narrowing the scope of calculation and improving the efficiency of answer recall.
[0044] Step S203: Determine, based on multiple matching degrees, N candidate questions matching the first keyword and candidate answers corresponding to the N candidate questions from a preset question-answer pair library.
[0045] According to the matching degree between the first keyword in the user's question and multiple first questions in the preset question and answer pair library, the multiple matching degrees are sorted, and the top N first questions are used as candidate questions, wherein the multiple first questions include N candidate questions.
[0046] In this embodiment, the candidate question with the highest matching degree with the first keyword is determined by calculating the matching degree between the first keyword in the user's question and multiple first questions in the preset question-answer pair library. In this way, multiple candidate question-answer pairs with the highest matching degree with the user's question can be selected from the massive preset question-answer pair library based on the matching degree of the first keyword, which facilitates the subsequent direct determination of the target question with the highest similarity to the user's question from the candidate questions, thereby reducing the calculation scope and accelerating the efficiency of recalling the target answer.
[0047] Furthermore, the above step S202, respectively calculating the matching degree between the first keyword and the first question of the preset question-answer pair library, specifically includes the following steps:
[0048] Calculate similarity scores of the first keyword and the plurality of first questions respectively, wherein the similarity score is the product of a first score and a second score, the first score is determined based on the number of times the first keyword appears in the corresponding first question, and the second score is determined based on the proportion of the first questions containing the first keyword in the preset question-answer pair library;
[0049] Normalizing the similarity scores of the multiple first questions according to the lengths of the multiple first questions;
[0050] A plurality of matching degrees are determined according to the normalized similarity scores of the plurality of first questions.
[0051] Specifically, any one of the first questions in the preset question-answer pair library is selected, and the number of times the first keyword appears in the first question is calculated respectively, thereby determining the first score of the first keyword in the first question sentence. Among them, the higher the number of times the first keyword appears in the first question, the higher its first score. In addition, it is also necessary to calculate the proportion of the first questions containing the first keyword in the preset question-answer pair library in the preset question-answer pair library, thereby determining the second score of the first keyword in the first question sentence. Among them, the smaller the proportion of the first questions containing the first keyword in the preset question-answer pair library in the preset question-answer pair library, the higher its second score. After determining the first score and the second score, the product of the first score and the second score can be calculated, thereby obtaining the similarity score of the first keyword in the first question sentence. In this way, for all the first questions in the preset question-answer pair library, the above method can be used to obtain the similarity score of the first keyword and its corresponding similarity score.
[0052] Since the length of each first question is different, after the similarity score is calculated, the similarity score corresponding to each first question can be normalized according to the length of each first question, thereby obtaining the matching degree corresponding to each first question.
[0053] It should be noted that when the first keyword is multiple keywords, it is necessary to calculate the first score and the second score of the multiple keywords, and then calculate the product of the first score and the second score of each keyword to determine the similarity score of each keyword, and then sum the similarity scores of the multiple keywords. Finally, the summation result is normalized according to the length of each first question, thereby obtaining the matching degree of the first keyword and each first question.
[0054] In this embodiment, it is necessary to comprehensively consider the number of times the first keyword appears in the first question, the proportion of first questions containing the first keyword in the preset question and answer pair library, and the length of the first question to calculate the matching degree between the first keyword and the first question, so that the matching of the first keyword with the first question in the preset question and answer pair library is more accurate and more in line with the intention of the user's question itself.
[0055] Further, see Figure 3 , Figure 3 Flow chart 3 of the question-answering method provided in the embodiment of the present application. Figure 1 In the embodiment shown, the above step S300, obtaining the text similarity between the user question and N candidate questions, specifically includes the following steps:
[0056] Step S301, respectively calculating feature weight values of the first keyword and the second keywords of the N candidate questions.
[0057] The N candidate questions are the N first questions in the preset question-answer pair library that have the highest matching degree with the first keyword of the user question. The feature weight value can be the matching degree obtained in the previous embodiment, or the similarity of the keywords calculated according to the preset similarity algorithm.
[0058] In an optional embodiment, the TF-IDF algorithm can be used to calculate the feature weight values of the first keyword in the user question and the second keyword in the candidate question, that is, the TF-IDF value of the first keyword and the TF-IDF value of the second keyword. Among them, the word frequency TF of the first keyword refers to the number of times the first keyword appears in the user question. Specifically, in order to avoid changes in the total number of words in the user question, the word frequency TF of the first keyword can be divided by the total number of words in the user question to obtain the relative word frequency. The reverse document frequency IDF of the first keyword refers to the proportion of candidate questions containing the first keyword in the candidate questions of the preset question-answer pair library, which can be obtained by dividing the sum of the candidate questions by the number of candidate questions containing the first keyword, and then taking the logarithm of the quotient. The word frequency TF of the second keyword refers to the number of times the second keyword appears in the corresponding candidate question. Specifically, in order to avoid changes in the total number of words in the candidate question, the word frequency TF of the second keyword can be divided by the total number of words in the corresponding candidate question to obtain the relative word frequency. The inverse document frequency IDF of the second keyword refers to the proportion of candidate questions in the preset question-answer pair library that contain the second keyword, which can be obtained by dividing the sum of the candidate questions by the number of candidate questions containing the second keyword and then taking the logarithm of the quotient.
[0059] Step S302: Determine the feature vectors corresponding to the user question and the N candidate questions according to the feature weight values.
[0060] The TF-IDF value of the first keyword is used as the feature vector of the user question, and the TF-IDF value of the second keyword is used as the feature vector of the candidate question, thereby obtaining the feature vectors of the user question and N candidate questions. For example, assuming that the user question includes keyword 1, keyword 2, and keyword 3, and the feature weight values corresponding to keyword 1, keyword 2, and keyword 3 are 1, 2, and 3 respectively, then the feature vector of the user question can be obtained [1,2,3,0,0]; assuming that a candidate question in the preset question-answer pair library includes keyword 2, keyword 3, and keyword 4, and the feature weight values corresponding to keyword 2, keyword 3, and keyword 4 are 5, 6, and 7 respectively, then the feature vector of the first question is [0,5,6,7,0]; assuming that another candidate question in the preset question-answer pair library includes keyword 1, keyword 3, and keyword 5, and the feature weight values corresponding to keyword 1, keyword 3, and keyword 5 are 7, 8, and 9 respectively, then the feature vector of the first question is [7,0,8,0,9].
[0061] Step S303: Determine the text similarity between the user question and the N candidate questions based on the feature vector and a preset cosine distance algorithm.
[0062] After obtaining the feature vectors of the user question and each candidate question, the preset cosine distance algorithm can be used to calculate the text similarity between the user question and each candidate question. Continuing with the above example, the cosine distance method is used to calculate the text similarity between the user question and the two first questions as follows:
[0063]
[0064]
[0065] According to the above method, the text similarity between the user question and each candidate question can be obtained, and then the obtained text similarities are sorted to obtain the M target questions with the largest text similarities. The candidate answers corresponding to the M target questions are then used as the target answers and returned to the user.
[0066] In this embodiment, the text similarity between the user question and the candidate question is calculated respectively by a preset cosine similarity algorithm, and the candidate questions are reordered, so that the M candidate questions with the highest text similarity can be determined from the N candidate questions, thereby making the recalled answers more accurate.
[0067] Furthermore, the preset question-answer pair library includes an inverted index, which includes the third keyword of the first question and attribute information corresponding to the third keyword, wherein the attribute information of the third keyword includes at least one of an identifier of the first question to which the third keyword belongs, the number of times the third keyword appears in the first question to which it belongs, and the position where the third keyword appears in the first question to which it belongs.
[0068] In an optional embodiment, an inverted index can be established based on question-answer pairs in a preset question-answer pair library, so that when there are many question-answer pairs, the first keyword and information related to the first keyword can be quickly located directly based on the inverted index. Specifically, the inverted index includes the third keywords of all first questions and the attribute information corresponding to the third keywords, wherein the attribute information of the third keyword includes at least one of the identifier of the first question to which the third keyword belongs, the number of times the third keyword appears in the first question to which it belongs, and the position at which the third keyword appears in the first question to which it belongs. It should be noted that since the candidate question is the first question in the preset question-answer pair library, the third keyword and the second keyword in the candidate question are actually the same keyword.
[0069] When establishing an inverted index, the B+ tree and hash zipper method can be used to store the third keyword to form a dictionary, and the inverted list corresponding to each third keyword is associated with the third keyword, wherein the inverted list is the attribute information of the third keyword, such as the identifier of the first question to which the third keyword belongs, the number of times the third keyword appears in the first question to which it belongs, and the position of the third keyword in the first question to which it belongs, etc. In this way, an index is formed with the third keyword as the key and the attribute information corresponding to the third keyword as the value. When the user enters the user question, the inverted index can be used to quickly locate the attribute information containing the first keyword, thereby quickly locating the matching first question, thereby improving the search efficiency.
[0070] In an application embodiment, see Figure 4 , Figure 4 This is a fourth flowchart of the question-and-answer method provided in the embodiment of the present application. Figure 4 As shown, first, it is necessary to collect question answer data, establish question-answer pairs, and then establish an inverted index based on the question-answer pairs. Based on the inverted index and the question-answer pairs, a preset question-answer pair library is formed. After obtaining the user's question, the preset word segmentation model can be called to segment the user's question, and based on the data information provided by the preset question-answer pair library, the preset keyword matching algorithm can be called to perform a preliminary sorting of the questions in the preset question-answer pair library to obtain candidate question-answer pairs for the user's question. The preset keyword matching algorithm can use the above Figure 2 The embodiment shown in the figure can be implemented, and other matching algorithms can also be used to implement it. After completing the initial sorting, the preset text similarity algorithm can be called according to the data information provided by the preset question-answer pair library to re-sort the candidate questions in the preset question-answer pair library, thereby determining the target question from the candidate questions, and returning the target answer corresponding to the target question to the user.
[0071] like Figure 5 As shown, Figure 5The structure diagram of the question-answering device provided in an embodiment of the present invention is as follows. The question-answering device 500 includes:
[0072] A first acquisition module 510 is used to acquire user questions;
[0073] A second acquisition module 520 is used to acquire N candidate questions matching the first keyword of the user's question and candidate answers corresponding to the N candidate questions based on a preset word segmentation model, where N is a positive integer;
[0074] The third acquisition module 530 is used to obtain the text similarity between the user question and the N candidate questions;
[0075] The determination module 540 is used to determine M target answers from the candidate answers corresponding to the N candidate questions based on text similarity, where M is a positive integer less than N.
[0076] Optionally, the second acquisition module 520 includes:
[0077] The word segmentation submodule is used to segment the user's question based on a preset word segmentation model to obtain the first keyword of the user's question;
[0078] A first calculation submodule is used to calculate the matching degree between the first keyword and a plurality of first questions in the preset question-answer pair library respectively, to obtain a plurality of matching degrees;
[0079] The first determination submodule is used to determine N candidate questions matching the first keyword and candidate answers corresponding to the N candidate questions from a preset question-answer pair library according to multiple matching degrees, wherein the multiple first questions include the N candidate questions.
[0080] Optionally, the first calculation submodule includes:
[0081] a calculation unit, configured to respectively calculate similarity scores between the first keyword and the plurality of first questions, wherein the similarity score is the product of a first score and a second score, the first score being determined based on the number of times the first keyword appears in the corresponding first question, and the second score being determined based on the proportion of the first questions containing the first keyword in a preset question-answer pair library;
[0082] A normalization processing unit, used for normalizing the similarity scores of the plurality of first questions according to the lengths of the plurality of first questions;
[0083] The determination unit is used to determine a plurality of matching degrees according to the similarity scores of the plurality of first questions after normalization.
[0084] Optionally, the third acquisition module 530 includes:
[0085] A second calculation submodule is used to calculate the feature weight values of the first keyword and the second keywords of the N candidate questions respectively;
[0086] The second determination submodule is used to determine the feature vectors corresponding to the user question and the N candidate questions according to the feature weight values;
[0087] The third determination submodule is used to determine the text similarity between the user question and the N candidate questions based on the feature vector and a preset cosine distance algorithm.
[0088] Optionally, the preset question-answer pair library includes an inverted index, which includes the third keyword of the first question and attribute information corresponding to the third keyword, wherein the attribute information of the third keyword includes at least one of an identifier of the first question to which the third keyword belongs, the number of times the third keyword appears in the first question to which it belongs, and the position where the third keyword appears in the first question to which it belongs.
[0089] The question-and-answer device of the embodiment of the present invention is a device corresponding to the question-and-answer method described above. All implementation methods in the above method are applicable to the embodiment of the device and can achieve the same technical effect, which will not be described in detail here.
[0090] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned question-answering method are implemented. All implementation methods in the above-mentioned method are applicable to the embodiment of the computer-readable storage medium, and can achieve the same technical effect, and are not described one by one here.
[0091] The above are preferred embodiments of the present invention. It should be noted that ordinary persons skilled in the art can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications are also within the protection scope of the present invention.
Claims
1. A question-answering method, characterized in that: include: Get user questions; Based on a preset word segmentation model, obtain N candidate questions matching the first keyword of the user question and candidate answers corresponding to the N candidate questions, where N is a positive integer; Obtaining text similarity between the user question and the N candidate questions; Based on the text similarity, determine M target answers from the candidate answers corresponding to the N candidate questions, where M is a positive integer less than N; The obtaining, based on a preset word segmentation model, N candidate questions matching the first keyword of the user question and candidate answers corresponding to the N candidate questions includes: Based on a preset word segmentation model, the user question is segmented to obtain a first keyword of the user question; the matching degree between the first keyword and multiple first questions in a preset question-answer pair library is calculated respectively to obtain multiple matching degrees; according to the multiple matching degrees, N candidate questions matching the first keyword and candidate answers corresponding to the N candidate questions are determined from the preset question-answer pair library, wherein the multiple first questions include the N candidate questions; The respectively calculating the matching degree between the first keyword and a plurality of first questions in the preset question-answer pair library to obtain a plurality of matching degrees includes: Calculating similarity scores between the first keyword and the plurality of first questions respectively, wherein the similarity score is the product of a first score and a second score, the first score is determined based on the number of times the first keyword appears in the corresponding first question, and the second score is determined based on the proportion of the first questions containing the first keyword in the preset question-answer pair library; Normalizing the similarity scores of the multiple first questions according to the lengths of the multiple first questions; The multiple matching degrees are determined according to the normalized similarity scores of the multiple first questions.
2. The method according to claim 1, characterized in that The obtaining of the text similarity between the user question and the N candidate questions includes: Calculate feature weight values of the first keyword and the second keywords of the N candidate questions respectively; Determining feature vectors corresponding to the user question and the N candidate questions according to the feature weight values; Based on the feature vector and a preset cosine distance algorithm, the text similarity between the user question and the N candidate questions is determined.
3. The method according to claim 1, characterized in that The preset question-answer pair library includes an inverted index, which includes a third keyword of the first question and attribute information corresponding to the third keyword, wherein the attribute information of the third keyword includes at least one of an identifier of the first question to which the third keyword belongs, the number of times the third keyword appears in the first question to which it belongs, and the position where the third keyword appears in the first question to which it belongs.
4. A question-and-answer device, characterized in that: include: The first acquisition module is used to obtain user questions; A second acquisition module is used to acquire N candidate questions matching the first keyword of the user question and candidate answers corresponding to the N candidate questions based on a preset word segmentation model, wherein N is a positive integer; A third acquisition module is used to obtain the text similarity between the user question and the N candidate questions; A determination module, configured to determine M target answers from the candidate answers corresponding to the N candidate questions based on the text similarity, wherein M is a positive integer less than N; The second acquisition module includes: A word segmentation submodule, used to segment the user question based on a preset word segmentation model to obtain a first keyword of the user question; A first calculation submodule is used to calculate the matching degree between the first keyword and a plurality of first questions in a preset question-answer pair library respectively, to obtain a plurality of matching degrees; a first determination submodule, configured to determine, according to the multiple matching degrees, N candidate questions matching the first keyword and candidate answers corresponding to the N candidate questions from the preset question-answer pair library, wherein the multiple first questions include the N candidate questions; The first calculation submodule includes: a calculation unit, configured to respectively calculate similarity scores between the first keyword and the plurality of first questions, wherein the similarity score is the product of a first score and a second score, the first score being determined based on the number of times the first keyword appears in the corresponding first question, and the second score being determined based on the proportion of the first questions containing the first keyword in the preset question-answer pair library; a normalization processing unit, configured to perform normalization processing on the similarity scores of the plurality of first questions according to the lengths of the plurality of first questions; A determination unit is used to determine the multiple matching degrees according to the similarity scores of the multiple first questions after normalization.
5. The device according to claim 4, characterized in that The third acquisition module includes: A second calculation submodule, used to calculate feature weight values of the first keyword and the second keywords of the N candidate questions respectively; A second determination submodule, configured to determine feature vectors corresponding to the user question and the N candidate questions according to the feature weight values; The third determination submodule is used to determine the text similarity between the user question and the N candidate questions based on the feature vector and a preset cosine distance algorithm.
6. The device according to claim 4, characterized in that The preset question-answer pair library includes an inverted index, which includes a third keyword of the first question and attribute information corresponding to the third keyword, wherein the attribute information of the third keyword includes at least one of an identifier of the first question to which the third keyword belongs, the number of times the third keyword appears in the first question to which it belongs, and the position where the third keyword appears in the first question to which it belongs.
7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the question-answering method according to any one of claims 1 to 3 are implemented.
Citation Information
Patent Citations
Automatic question-answer processing method and automatic question-answer system
CN108536708A
A method for implementing a question answering system based on a question-answer pair
CN109271505A