Document retrieval method and device and computer readable storage medium

Abstract

The embodiment of the invention relates to the field of natural language processing, and discloses a document retrieval method and device and a computer readable storage medium. The document retrievalmethod comprises the steps of acquiring candidate documents, wherein the candidate documents are determined by query statements input by a user, judging whether the text word number of the candidatedocument is smaller than or equal to a preset word number or not, if not, segmenting the candidate document into a plurality of sentences, calculating the similarity between each sentence in the plurality of sentences and the query sentence, deleting a part of sentences in the plurality of sentences according to the similarity until the total word number of the remaining sentences is smaller thanor equal to the preset word number, and inputting the remaining sentences and the query statement into a preset machine reading model to obtain an answer of the query statement. According to the document retrieval method and device and the computer readable storage medium provided by the invention, the reading difficulty of the machine reading model can be reduced, and meanwhile, the document retrieval accuracy is improved.

Description

technical field [0001] Embodiments of the present invention relate to the field of natural language processing, and in particular to a document retrieval method, device, and computer-readable storage medium. Background technique [0002] Document retrieval refers to retrieving the first few documents most relevant to the query from the retrieval database as document candidate sets; document reading refers to obtaining the answer to the query by machine reading the query and documents. Among them, the document candidate set is mainly obtained by calculating the similarity between the query and each document in the text library, and then sorting according to the similarity. [0003] The inventors have found that there are at least the following problems in the prior art: documents are usually composed of multiple sentences or multi-paragraph texts. If the retrieved candidate documents are directly input into the reading comprehension model, due to the limitation of the reading...

AI Technical Summary

Problems solved by technology

[0003] The inventors have found that there are at least the following problems in the prior art: documents are usually composed of multiple sentences or multi-paragraph texts. If the retrieved candidate documents are directly input into the reading comprehension model, due to the limitation of the reading length of the reading comprehension model (that is, only Read and comprehend documents within the preset word count), so that candidate documents with more text words will not only increase the difficulty of machine reading, but also may lose some semantic information due to the limitation of reading length, thus directly affecting the overall performance of reading comprehension. resulting in inaccurate document retrieval

Images