Question and answer matching method and system based on question pair matching and question and answer pair matching
By fine-tuning the SimBERT model and performing multiple similarity calculations, combined with BM25 and cosine similarity algorithms, the problem of insufficient answer matching accuracy in question-answering systems was solved, achieving higher accuracy in question-answering matching.
Patent Information
- Application Number
- CN202211063932.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-31
- Publication Date
- 2026-02-06
- Estimated Expiration
- 2042-08-31
AI Technical Summary
Existing question-answering systems based on the SimBERT pre-trained model lack precision in calculating text similarity, making it difficult to accurately match the best answer.
The SimBERT model was fine-tuned using question-pair datasets and question-answer pair datasets respectively, and a question-question matching model and a question-answer matching model were trained. Question-answer pairs were filtered through two similarity calculations. The BM25 algorithm was used for preliminary filtering and the cosine similarity algorithm was used for precise matching. The sentence vector representation was processed by combining linear transformation and concatenation.
It improves the accuracy of question-and-answer matching, ensures matching effectiveness, and can more accurately obtain the best answer to a question.
Smart Images
Figure CN115391513B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of question and answer matching, and particularly relates to a question and answer matching method and system based on question pair matching and question and answer pair matching. BACKGROUND
[0002] A knowledge base-based question and answer system refers to searching for a correct answer in a constructed question and answer library for an input natural language question. A traditional method uses TF-IDF, BM25 and other algorithms to perform a shallow correlation calculation on a sentence of an input question and a question sentence in a question and answer library, wherein, TF-IDF (Term Frequency-Inverse Document Frequency) is a commonly used weighting technology for information retrieval and data mining, BM is the abbreviation of Best Match, and 25 refers to the 25th algorithm iteration.
[0003] The inventor finds that a SimBERT pre-training model for calculating text similarity can map a sentence into a high-dimensional vector space, and can better perform a deep similarity calculation on a sentence pair to a certain extent, but the calculation accuracy of the SimBERT pre-training model on the similarity still needs to be improved. SUMMARY
[0004] To solve the above problems, the application provides a question and answer matching method and system based on question pair matching and question and answer pair matching, which uses a question pair dataset and a question and answer pair dataset to respectively fine-tune two optimal question pair matching models and question and answer pair matching models based on a SimBERT pre-training model to calculate the similarity of an input question and a question in a question and answer library and the similarity of the input question and an answer in the question and answer library, and finally obtains the best answer according to the similarity between the question and the answer.
[0005] To achieve the above purpose, the application is implemented by the following technical solutions:
[0006] In a first aspect, the application provides a question and answer matching method based on question pair matching and question and answer pair matching, which comprises:
[0007] Obtaining a question sentence in a question;
[0008] Obtaining a question sentence in a question;
[0009] According to the vector representation of the question sentence in the question and the vector representation of the question sentence in the preset question and answer library, calculating the similarity between the question sentence in the question and the question sentence in the preset question and answer library, retaining a question and answer pair in the preset question and answer library that meets a preset requirement, and obtaining a screened question and answer library.
[0010] According to the obtained question and the preset question-answer matching model, the vector representation of the question in the question and the vector representation of the answer in the screened question-answer library are obtained.
[0011] According to the vector representation of the question in the question and the vector representation of the question in the preset question-answer library, the similarity between the question in the question and the answer in the screened question-answer library is calculated, and the question-answer pair with the highest similarity score is taken as the final matched question-answer pair of the obtained question.
[0012] Further, before the sentence pair data is input into the question-question matching model and the question-answer matching model, the words in each sentence in the sentence pair are converted into an indexed form, and corresponding index values are added at the beginning and end of each converted indexed sentence.
[0013] Further, the question-question matching model and the question-answer matching model are trained by a SimBERT model; in the SimBERT model, the vector representation of the last layer of each sentence in the sentence pair is obtained; then each sentence is first linearly transformed to map each sentence to a preset dimensional sentence vector space, and then the vector representations of the two sentences in the sentence pair are spliced in the last dimension to obtain a new vector representation; finally, a linear transformation is performed again to obtain the final vector representation.
[0014] Further, using the BM25 algorithm, the relevance of the obtained question and the questions in the question-answer library is calculated to obtain a plurality of question-answer pairs in the question-answer library related to the obtained question.
[0015] Further, after obtaining a plurality of question-answer pairs in the question-answer library related to the obtained question, the obtained sentence is paired with each question in the question-answer pair to obtain a question-question list;
[0016] According to the question-question matching model, the vector representation of the question in the question and the vector representation of the question in the question-answer library are obtained.
[0017] The cosine similarity algorithm is used to calculate the similarity score between the obtained question and the questions in the question-answer library, and the question-answer pairs with scores exceeding a preset value are retained.
[0018] Further, the obtained question is paired with the answer sentence in the question-answer pair with a score exceeding a preset value,
[0019] According to the question-answer matching model, the vector representation of the obtained question and the vector representation of the answer are obtained.
[0020] The cosine similarity algorithm is used to calculate the similarity score between the obtained sentence and each answer, and the answer with the highest score in the question-answer library is taken as the answer to the obtained question.
[0021] Further, the question-answer pair comprises a question sentence and an answer sentence.
[0022] In a second aspect, the present application further provides a question-answer matching system based on question pair matching and question-answer pair matching, comprising:
[0023] a data acquisition module configured to acquire a question sentence in a question;
[0024] a question-question matching module configured to obtain a vector representation of the question sentence in the question and a vector representation of a question sentence in a preset question-answer library according to the acquired question sentence and a preset question-question matching model;
[0025] a question-answer library screening module configured to calculate a similarity between the question sentence in the question and the question sentence in the preset question-answer library according to the vector representation of the question sentence in the question and the vector representation of the question sentence in the preset question-answer library, retain a question-answer pair in the preset question-answer library that meets a preset requirement, and obtain a screened question-answer library;
[0026] a question-answer matching module configured to obtain a vector representation of the question sentence in the question and a vector representation of an answer in the screened question-answer library according to the acquired question sentence and a preset question-answer matching model;
[0027] a question-answer library matching module configured to calculate a similarity between the question sentence in the question and the answer in the screened question-answer library according to the vector representation of the question sentence in the question and the vector representation of the question sentence in the preset question-answer library, and take a question-answer pair with a highest similarity score as a final matching question-answer pair of the question.
[0028] In a third aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the question-answer matching method based on question pair matching and question-answer pair matching according to the first aspect.
[0029] In a fourth aspect, the present application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the question-answer matching method based on question pair matching and question-answer pair matching according to the first aspect when executing the program.
[0030] Compared with the prior art, the present application has the following beneficial effects:
[0031] 1. In this invention, after obtaining the question, a preset question-to-question matching model is used to obtain the vector representation of the question in the question and the vector representation of the question in the preset question-answer database. Based on this, the similarity between the question in the question and the question in the preset question-answer database is calculated to perform preliminary screening of the question-answer database. Then, a preset question-to-answer matching model is used to obtain the vector representation of the question in the question and the vector representation of the answer in the filtered question-answer database. Based on this, the similarity between the question in the question and the answer in the filtered question-answer database is calculated, and the question-answer pair with the highest similarity score is taken as the final question-answer pair for obtaining the question. In the question-answer pair matching process, the two similarity calculations realize the preliminary screening and final determination of the question-answer pairs, improve the accuracy of similarity calculation, and ensure the matching effect.
[0032] 2. In this invention, the question-to-question matching model and the question-to-answer matching model are trained using the SimBERT model. Within the SimBERT model, fine-tuning is performed. Specifically, the vector representation of the last layer of each sentence in a sentence pair is first obtained. Then, a linear transformation is performed on each sentence to map it to a sentence vector space of a preset dimension. Next, the vector representations of the two sentences in the sentence pair are concatenated on the last dimension to obtain a new vector representation. Finally, another linear transformation is performed to obtain the final vector representation. By fine-tuning the model, better vector representations of the question and answer sentences are obtained. Simultaneously, combined with question-to-question matching, question-to-answer matching, and two similarity calculations, the goal of obtaining the optimal answer is achieved. Attached Figure Description
[0033] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.
[0034] Figure 1 This is a flowchart of Embodiment 1 of the present invention. Detailed Implementation
[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0036] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0037] Sentence matching refers to question-question matching or question-answer matching;
[0038] Question-answer pairing refers to a question-answer pair or a question-answer sentence pair.
[0039] The question pair refers to a question-question pair, which refers to the question in the question and the question in the question-answer library pair;
[0040] JSON (JavaScript Object Notation) is a lightweight data exchange format.
[0041] BERT, which stands for Bidirectional Enoceder Representations from Transformers, is a language representation model.
[0042] Embodiment 1:
[0043] The embodiment provides a question-answer matching method based on question pair matching and question-answer pair matching, which comprises:
[0044] Obtaining the question in the question;
[0045] According to the obtained question, and the preset question-question matching model, the vector representation of the question in the question and the vector representation of the question in the preset question-answer library are obtained;
[0046] According to the vector representation of the question in the question and the vector representation of the question in the preset question-answer library, the similarity between the question in the question and the question in the preset question-answer library is calculated, the question-answer pair in the preset question-answer library that meets the preset requirement is reserved, and the screened question-answer library is obtained;
[0047] According to the obtained question, and the preset question-answer matching model, the vector representation of the question in the question and the vector representation of the answer in the screened question-answer library are obtained;
[0048] According to the vector representation of the question in the question and the vector representation of the question in the preset question-answer library, the similarity between the question in the question and the answer in the screened question-answer library is calculated, and the question-answer pair with the highest similarity score is obtained as the final matched question-answer pair of the obtained question.
[0049] The question-question matching model and the question-answer matching model are trained by the SimBERT model.
[0050] In this embodiment, first, the question vector representation in the question and the vector representation of the question in the preset question and answer library are obtained through the preset question-question matching model, and on this basis, the similarity between the question in the question and the question in the preset question and answer library is calculated, and the question and answer library is preliminarily screened; then, the question vector representation in the question and the vector representation of the answer in the screened question and answer library are obtained through the preset question-answer matching model, and on this basis, the similarity between the question in the question and the answer in the screened question and answer library is calculated, and the question and answer pair with the highest similarity score is taken as the final matched question and answer pair of the question; in the question and answer pair matching process, the two similarity calculations realize the preliminary screening and final determination of the question and answer pair, improve the accuracy of the similarity calculation, and ensure the matching effect; the specific steps of this embodiment are:
[0051] S1, question-question matching model fine-tuning training:
[0052] S1.1, preprocessing:
[0053] In the question-question matching model fine-tuning training stage, the question-question sentence pair data set used in this embodiment contains two questions and a label 0 or 1 indicating whether the two questions are similar questions in each JSON data. Before inputting the sentence pair data into the model, the tokenizer of the BERT model is used to convert the words in each sentence in the sentence pair into an indexed form, and an index value corresponding to [CLS] is added at the beginning of each converted indexed sentence, and an index value corresponding to [SEP] is added at the end, the two index values are used to separate the tokens of the two questions, and finally the unified maximum length is filled with "0" at the end.
[0054] S1.2, input the preprocessed data into the model for fine-tuning training based on the BERT model structure, the pre-training model used in this embodiment is an open source SimBERT model trained by using 22 million similar sentences on Baidu Knows by Zhiyi Technology, and on the basis of the pre-training model, the model in this embodiment uses a double-tower model design to fine-tune the sentence pair, in the fine-tuning process, the last layer shape of each sentence in the sentence pair input into the model is obtained [sentence length, hidden layer dimension] vector representation, then each sentence is first linearly transformed to map them to a specific dimension sentence vector space, such as a 768-dimensional sentence vector space, the vector representation of each sentence becomes [1, new hidden layer dimension], then the vector representations of the two sentences in the sentence pair are spliced in the last dimension to obtain a vector representation of [1, new hidden layer dimension*2], and finally a linear transformation is performed to obtain a vector representation of [1, class number], the final model returns the vector representation of each sentence before splicing and the final vector representation of the sentence pair obtained after splicing. The result of whether the two sentences are similar can be obtained by applying a softmax activation function to the final sentence pair vector representation during training.
[0055] S1.3, after training, the trained model is evaluated on the validation set, and the model with the best F1 index on the validation set is finally saved as the question-question matching model in the final question answering system.
[0056] S2, question-answer matching model fine-tuning training:
[0057] S2.1, the question-answer matching model fine-tuning training method is the same as the question-question matching model fine-tuning training method in step S1, the form of the data set is changed to question sentences and answer sentences and the labels of whether the question and answer match; the question-answer matching model fine-tuning training process is not described in detail here.
[0058] S2.2, similarly, after training, the trained model is evaluated on the validation set, and the model with the best F1 index on the validation set is finally saved as the question-answer matching model in the final question answering system.
[0059] S3, coarse-grained similar question query:
[0060] After the fine-tuning model training in steps S1 and S2 is completed to obtain the optimal model, the two models can be loaded into the intelligent question answering system model, and the question sentence is input into the question answering model. The question answering model first uses the BM25 algorithm to calculate the relevance of the input question and the question in the question-answer pair in the preset question-answer library in a coarse-grained manner, and finally obtains the topN question-answer pairs in the question-answer library that are most relevant to the input question.
[0061] S4, fine-grained question similarity calculation
[0062] For the topN question-answer pairs obtained in step S3, first, the input question sentence is paired with each question sentence in the question-answer pairs obtained in step S3 to obtain a question-question list, which is input into the model fine-tuned in step S1 to obtain the vector representation of the input question sentence and the vector representation of the question in the question-answer pair, and then the cosine similarity algorithm is used to calculate the similarity score between the two, and the question-answer pair with a score greater than 0.5 is retained.
[0063] S5, fine-grained question-answer pair similarity calculation
[0064] The input question sentence is paired with the answer sentence in the question-answer pair obtained in step S4, and then input into the question-answer matching model fine-tuned in step S2 to obtain the vector representation of the input question and the vector representation of each answer, and then use the cosine similarity algorithm to calculate the similarity score between the question sentence and each answer; the answer in the question-answer pair with the highest final score is the answer to the input question.
[0065] In this embodiment, the pre-trained model is fine-tuned using the question pair dataset and the question-answer pair dataset. The model will have better improvement in the matching effect of question-question pairs and question-answer pairs. In a question-answer system based on sentence pair similarity matching, the question-question matching model is first applied to extract the candidate question-answer pairs similar to the input question from the question-answer library, and then the question-answer matching model is applied to determine the best answer in the candidate question-answer pair, which can further improve the accuracy of extracting answers using only the question-question matching model.
[0066] Embodiment 2:
[0067] The embodiment provides a question-answer matching system based on question pair matching and question-answer pair matching, comprising:
[0068] The data acquisition module is configured to acquire a question in a question;
[0069] The question-question matching module is configured to obtain the vector representation of the question in the question and the vector representation of the question in the preset question-answer library according to the acquired question and the preset question-question matching model;
[0070] The question-answer library screening module is configured to calculate the similarity between the question in the question and the question in the preset question-answer library according to the vector representation of the question in the question and the vector representation of the question in the preset question-answer library, retain the question-answer pairs in the preset question-answer library that meet the preset requirements, and obtain the screened question-answer library;
[0071] The question-answer matching module is configured to obtain vector representations of the question in the question and vector representations of the answers in the screened question-answer database according to the obtained question and a preset question-answer matching model.
[0072] The question-answer database matching module is configured to calculate similarity between the question in the question and the answers in the screened question-answer database according to the vector representations of the question in the question and the vector representations of the questions in the preset question-answer database, and take a question-answer pair with the highest similarity score as a final matched question-answer pair of the question.
[0073] The working method of the system is the same as the question-answer matching method based on question pair matching and question-answer pair matching in Embodiment 1, and thus is not described here.
[0074] Embodiment 3
[0075] The embodiment provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement steps of the question-answer matching method based on question pair matching and question-answer pair matching in Embodiment 1.
[0076] Embodiment 4
[0077] The embodiment provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements steps of the question-answer matching method based on question pair matching and question-answer pair matching in Embodiment 1 when executing the program.
[0078] The above merely describes preferred embodiments of the present embodiment and is not intended to limit the present embodiment. The present embodiment can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.
Claims
1. A question answering matching method based on question pair matching and answer pair matching, characterized in that, The method comprises the following steps: obtaining a question in a question; obtaining a question vector representation in the question and a question vector representation in a preset question and answer library according to the obtained question and a preset question-question matching model; calculating the similarity between the question in the question and the answer in the filtered question and answer library according to the question vector representation in the question and the question vector representation in the preset question and answer library, and retaining the question and answer pair with the highest similarity score as the final matched question and answer pair of the obtained question; using the BM25 algorithm to calculate the relevance of the obtained question and the question in the question and answer library, and obtaining a plurality of question and answer pairs in the question and answer library related to the obtained question; after obtaining a plurality of question and answer pairs in the question and answer library related to the obtained question, pairing the obtained question with each question in the question and answer pair to obtain a question-question list; obtaining a question vector representation in the question and a question vector representation in the question and answer library according to the question-question matching model; using the cosine similarity algorithm to calculate the similarity score between the obtained question and the question in the question and answer library, and retaining the question and answer pair with a score higher than a preset value; pairing the obtained question with the answer sentence in the question and answer pair with a score higher than the preset value, obtaining a question vector representation in the question and an answer vector representation according to the question-answer matching model; using the cosine similarity algorithm to calculate the similarity score between the obtained question and each answer, and the answer with the highest score in the question and answer library is the answer to the obtained question. Before the sentence pair data is input into the question-question matching model and the question-answer matching model, the words in each sentence in the sentence pair are converted into an indexed form, and corresponding index values are added at the beginning and end of each converted indexed sentence. The question-question matching model and the question-answer matching model are trained by a SimBERT model; in the SimBERT model, the vector representation of the last layer of each sentence in the sentence pair is obtained; then each sentence is linearly transformed to map each sentence to a preset dimensional sentence vector space, and the vector representations of the two sentences in the sentence pair are spliced in the last dimension to obtain a new vector representation; finally, a linear transformation is performed again to obtain the final vector representation.
2. The question-answer matching method based on matching of question pairs and matching of answer pairs according to claim 1, wherein, The question and answer pair comprises a question and an answer sentence.
3. The question-answer matching method based on matching of question pairs and matching of answer pairs according to claim 1, wherein, The method comprises the following steps:
4. The method of claim 1, wherein the matching of the question pair and the matching of the answer pair are based on a question-answer matching method, wherein the question-answer matching method comprises: a data acquisition module configured to obtain a question in a question; 5. A question answering matching system based on question pair matching and answer pair matching, characterized in that, a question-question matching module configured to obtain a question vector representation in the question and a question vector representation in a preset question and answer library according to the obtained question and a preset question-question matching model; The question-answer bank screening module is configured to: calculate the similarity between the question in the question and the question in the preset question-answer bank according to the vector representation of the question in the question and the vector representation of the question in the preset question-answer bank, retain the question-answer pair in the preset question-answer bank that meets the preset requirement, and obtain the screened question-answer bank; The question-answer bank matching module is configured to: calculate the similarity between the question in the question and the answer in the screened question-answer bank according to the vector representation of the question in the question and the vector representation of the question in the preset question-answer bank, and the question-answer pair with the highest similarity score is used as the final matched question-answer pair of the question. The question-answer bank matching module is configured to: calculate the similarity between the question in the question and the answer in the screened question-answer bank according to the vector representation of the question in the question and the vector representation of the question in the preset question-answer bank, and the question-answer pair with the highest similarity score is used as the final matched question-answer pair of the question. The question-question matching model and the question-answer matching model are obtained by training a SimBERT model. The BM25 algorithm is used to calculate the relevance between the obtained question and the questions in the question-answer bank, and a plurality of question-answer pairs in the question-answer bank related to the obtained question are obtained. After obtaining the plurality of question-answer pairs in the question-answer bank related to the obtained question, the obtained question is paired with each question in the question-answer pair to obtain a question-question list. The question-question matching model is used to obtain the vector representation of the question in the question and the vector representation of the question in the question-answer bank. The cosine similarity algorithm is used to calculate the similarity score between the obtained question and the questions in the question-answer bank, and the question-answer pair with a score higher than a preset value is retained. The obtained question is paired with the answer sentence in the question-answer pair with a score higher than the preset value, The question-answer matching model is used to obtain the vector representation of the obtained question and the vector representation of the answer. The cosine similarity algorithm is used to calculate the similarity score between the obtained question and each answer, and the answer with the highest score in the question-answer bank is used as the answer to the question.
6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the question-answer matching method based on question pair matching and question-answer pair matching according to any one of claims 1-4.
7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the question-answer matching method based on question pair matching and question-answer pair matching according to any one of claims 1-4.
Citation Information
Patent Citations
FAQ conversation method and device and electronic equipment
CN110096580A