Question-answer pair matching technology based on semantic similarity
A technology of semantic similarity and question answering, applied in semantic analysis, natural language data processing, special data processing applications, etc., can solve problems such as inaccurate understanding of user question intentions, lack of pertinence, etc.
Inactive Publication Date: 2021-03-26
四川智仟科技有限公司
View PDF2 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
The traditional retrieval system conducts answer query based entirely on keywords, cannot accurately understand the user's question intention, and the retrieval results contain a large amount of irrelevant information, which is not targeted
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0013] The present invention will be further described in detail below in conjunction with the embodiments, so that those skilled in the art can implement it with reference to the description.
[0014] The question-answer pair matching technology based on semantic similarity of the present embodiment comprises the following steps:
[0015] 1) Collect the corpus of question-answer pairs, and use the ElasticSearch tool to build an inverted index for all question-answer pairs.
[0016] 2) Using the collected question-answer pairs as data, fine-tune the BERT model so that it can judge the semantic similarity between questions and answers.
[0017] 3) Use the IK tokenizer to segment the questions entered by the user.
[0018] 4) For the question after word segmentation, remove the stop words in the question, so as to perform more precise matching and obtain more accurate results.
[0019] 5) After removing the stop words, use ElasticSearch to retrieve similar questions, and use t...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a question and answer pair matching technology based on semantic similarity, and the technology comprises the following steps: establishing an inverted index for a question andanswer pair data set by utilizing ElasticSearch, and then finely adjusting a BERT model by utilizing the question and answer pair data set, so that the semantic similarity between a question and an answer can be judged; for questions input by a user, firstly employing ElasticSearch retrieval to obtain a preliminary retrieval result, employing TF-IDF to rank the results, answers ranked in the first five are combined with the input questions respectively, inputting the input questions into a BERT model to obtain semantic similarity scores, and finally, weighting and summing the two scores to obtain a final score, re-ranking the final score, and returning an answer with the highest score. According to the question and answer pair matching technology based on semantic similarity, the recall rate of answers is high.
Description
technical field [0001] The invention relates to the field of intelligent question answering, in particular to a question answer pair matching technology based on semantic similarity. Background technique [0002] With the rapid development of artificial intelligence, intelligent question answering technology has been applied to various industries. The traditional retrieval system searches for answers based entirely on keywords, which cannot accurately understand the user's question intention, and the retrieval results contain a large amount of irrelevant information, which is not targeted. Pre-trained models such as BERT have recently received attention, and have achieved state-of-the-art results in many natural language processing tasks, such as machine translation, automatic summarization, and intelligent question answering. The present invention proposes a question-and-answer pair matching technology based on semantic similarity. First, a preliminary search is performed ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/332G06F16/338G06F40/284G06F40/289G06F40/30
CPCG06F16/3329G06F16/338G06F40/30G06F40/289G06F40/284
Inventor 银大伟
Owner 四川智仟科技有限公司