Document Q&A Method, Apparatus, System, Electronic Device and Storage Medium

Abstract

The present invention relates to the technical field of natural language processing, and provides a document question-answering method, device, system, electronic device and storage medium. The method uses a rough ranking model to obtain multiple candidate documents in a target document library, uses a fine ranking model to obtain the similarity between each candidate document and a user question, and determines a target document, and then determines a target answer corresponding to the user question. The samples used in the training processes of the rough ranking model and the fine ranking model include content document pairs and a set of question-document pairs. Through the content document pairs, the pre-trained rough ranking model and the pre-trained fine ranking model can learn more sufficient knowledge without introducing manual annotation, which can not only save the model training cost, improve the training efficiency, but also solve the problem of insufficient high-quality labeled data. Through the set of question-document pairs, it can be ensured that accurate target answers can be provided for users through the rough ranking model and the fine ranking model.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and particularly to a document question-answering method, device, system, electronic device and storage medium. Background Art

[0002] With the increasing maturity of artificial intelligence-related technologies, voice interaction requirements play an increasingly important role in work and life. The question-answering module is an important module in the voice interaction system, mainly used to answer various questions of users.

[0003] The existing question-answering methods applied by the question-answering module are mainly open-domain question-answering methods based on documents. This method requires collecting a large-scale document library in advance, and after obtaining the user's question, using a question-answering model to retrieve relevant documents of the user's question from the document library, and then extracting or generating answers from the documents.

[0004] Although the existing question-answering models can achieve question-answering, they need to rely on a large amount of labeled data during the training process, facing the problems of difficult acquisition of high-quality labeled data and low efficiency. Summary of the Invention

[0005] The present invention provides a document question-answering method, device, system, electronic device and storage medium to solve the defects existing in the prior art.

[0006] The present invention provides a document question-answering method, including:

[0007] Obtain a user question;

[0008] Input the user question into a rough ranking model to obtain multiple candidate documents retrieved by the rough ranking model from a document library;

[0009] Input the user question and the multiple candidate documents into a fine ranking model to obtain the similarity between each candidate document output by the fine ranking model and the user question, and determine a target document whose similarity to the user question is higher than a preset threshold;

[0010] Based on the user question, the target document, and the similarity between the target document and the user question, determine a target answer corresponding to the user question;

[0011] Wherein, the training steps of the rough ranking model and the fine ranking model include:

[0012] Pre-train an initial rough ranking model and an initial fine ranking model based on the content document pairs corresponding to the document library, respectively obtaining a pre-trained rough ranking model and a pre-trained fine ranking model; the content document pairs are determined based on the target content of any document in the document library and the documents in the document library;

[0013] Based on a set of question-document pairs, jointly train the pre-trained rough ranking model and the pre-trained fine ranking model to obtain the rough ranking model and the fine ranking model.

[0014] According to a document question answering method provided by the present invention, the step of jointly training the pre-trained rough ranking model and the pre-trained fine ranking model based on the set of question-document pairs to obtain the rough ranking model and the fine ranking model includes:

[0015] Based on the first type of question-document pairs in the set of question-document pairs, jointly train the pre-trained rough ranking model and the pre-trained fine ranking model to respectively obtain a baseline rough ranking model and a baseline fine ranking model;

[0016] Based on the first type of question-document pairs, determine the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model, and based on the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model, iteratively train the baseline rough ranking model and the baseline fine ranking model to obtain the rough ranking model and an alternative fine ranking model;

[0017] Based on the alternative fine ranking model, determine the fine ranking model.

[0018] According to a document question answering method provided by the present invention, the step of determining the target answer corresponding to the user question based on the user question, the target document, and the similarity between the target document and the user question includes:

[0019] Input the user question and the target document into an understanding model to obtain answer fragments within the target document output by the understanding model;

[0020] Based on the user question, the answer fragments, and the similarity between the target document and the user question, determine the target answer;

[0021] Wherein, the fine ranking model and the understanding model are determined based on the following steps:

[0022] Based on the second type of question-document pairs with answer fragment labels in the set of question-document pairs, jointly train the alternative fine ranking model and an initial understanding model to obtain the fine ranking model and the understanding model.

[0023] According to a document question answering method provided by the present invention, the encoder parameters of the alternative fine ranking model and the initial understanding model are shared.

[0024] According to a document question answering method provided by the present invention, the step of determining the target answer based on the user question, the answer fragments, and the similarity between the target document and the user question includes:

[0025] Input the similarity between the user question, the answer snippet, and the target document to the generation model to obtain the target answer output by the generation model;

[0026] Among them, the generation model is trained based on question-answer snippet pairs and the similarity between the document samples where the answer snippet samples in the question-answer snippet pairs are located and the question samples in the question-answer snippet pairs.

[0027] According to a document Q&A method provided by the present invention, determining the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine-ranking model based on the first type of question-document pairs includes:

[0028] Input the question samples in the first type of question-document pairs to the baseline coarse-ranking model to obtain multiple associated documents retrieved by the baseline coarse-ranking model from the document library;

[0029] Input the multiple associated documents to the baseline fine-ranking model to obtain the similarity between each associated document output by the baseline fine-ranking model and the question sample;

[0030] Based on the similarity between each associated document and the question sample, determine the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine-ranking model.

[0031] According to a document Q&A method provided by the present invention, inputting the user question to the coarse-ranking model to obtain multiple candidate documents retrieved by the coarse-ranking model from the document library includes:

[0032] Input the user question to the question encoder of the coarse-ranking model to obtain the question encoding vector output by the question encoder;

[0033] Input the question encoding vector and the document encoding vectors corresponding to each document in the document library obtained based on the document encoder of the coarse-ranking model to the similarity calculation layer of the coarse-ranking model to obtain the similarity between the user question and each document output by the similarity calculation layer;

[0034] Input the similarity between the user question and each document to the output layer to obtain the multiple candidate documents with high similarity among the documents output by the output layer.

[0035] The present invention also provides a document Q&A device, including:

[0036] An acquisition module for acquiring a user question;

[0037] The rough ranking module is used to input the user question into a rough ranking model, and obtain multiple alternative documents retrieved by the rough ranking model from a document library;

[0038] The fine ranking module is used to input the user question and the multiple alternative documents into a fine ranking model, obtain the similarity between each alternative document output by the fine ranking model and the user question, and determine a target document whose similarity to the user question is higher than a preset threshold;

[0039] The determination module is used to determine a target answer corresponding to the user question based on the user question, the target document, and the similarity between the target document and the user question;

[0040] The training module is used for:

[0041] Pre-train an initial rough ranking model and an initial fine ranking model based on content document pairs corresponding to the document library, and respectively obtain a pre-trained rough ranking model and a pre-trained fine ranking model; the content document pairs are determined based on the target content of any document in the document library and the documents in the document library;

[0042] Jointly train the pre-trained rough ranking model and the pre-trained fine ranking model based on a set of question-document pairs to obtain the rough ranking model and the fine ranking model.

[0043] The present invention also provides a document question-answering system, including: a voice module and the above-mentioned document question-answering device;

[0044] The voice module is connected to the document question-answering device;

[0045] The voice module is used to collect a user question and transmit the user question to the document question-answering device.

[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the document question-answering method as described in any one of the above.

[0047] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the document question-answering method as described in any one of the above.

[0048] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the document question-answering method as described in any one of the above.

[0049] The document question-answering method, device, system, electronic device and storage medium provided by the present invention first obtain a user question; then retrieve multiple candidate documents from a document library using a rough ranking model, obtain the similarity between each candidate document and the user question using a fine ranking model, and determine a target document whose similarity to the user question is higher than a preset threshold, and further determine a target answer corresponding to the user question. The samples used in the training processes of the rough ranking model and the fine ranking model may include content document pairs corresponding to the document library and a set of question-document pairs. Through the content document pairs, the pre-trained rough ranking model and the pre-trained fine ranking model can learn more sufficient knowledge without introducing manual annotations, which can not only save the model training cost and improve the training efficiency, but also solve the problem of insufficient high-quality labeled data. Through the set of question-document pairs, it can be ensured that the pre-trained rough ranking model and the pre-trained fine ranking model can be trained into a rough ranking model and a fine ranking model that can accurately screen out the target document with less labeled data, and thus can provide an accurate target answer for the user. In addition, since the document library used in this method is not restricted by the field, open-ended question answering can be realized, and it has strong scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings, for those of ordinary skill in the art, can obtain other drawings without creative efforts.

[0051] Figure 1 is a schematic flowchart of a document question-answering method provided by the present invention;

[0052] Figure 2 is a schematic flowchart of a document question-answering method implemented based on a document question-answering model provided by the present invention;

[0053] Figure 3 is a schematic structural diagram of a rough ranking model in a document question-answering model provided by the present invention;

[0054] Figure 4 is a schematic structural diagram of a fine ranking model in a document question-answering model provided by the present invention;

[0055] Figure 5 is a schematic structural diagram of a document question-answering device provided by the present invention;

[0056] Figure 6 is a schematic structural diagram of a document question-answering system provided by the present invention;

[0057] Figure 7 is a schematic structural diagram of an electronic device provided by the present invention. Detailed implementation mode

[0058] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0059] Currently, the mainstream question-and-answer methods may include the question-and-answer method based on question-and-answer library matching, the question-and-answer method based on knowledge graph, and the open-domain question-and-answer method based on documents.

[0060] 1) Question-and-answer method based on question-and-answer library matching: First, a high-frequency question-and-answer library is constructed according to user habits through log analysis or a domain question-and-answer library is constructed by artificial experts in a certain field. After the user asks a question, the most similar question is found from the question-and-answer library through a similarity prediction module, and the answer corresponding to the question is returned to the user. The construction of the question-and-answer library of this method requires a large amount of manpower, and the coverage of question-and-answer pairs is relatively narrow, only suitable for high-frequency questions, and the coverage for long-tail questions is far from enough.

[0061] 2) Question-and-answer method based on knowledge graph: It is necessary to construct a large-scale knowledge graph in advance. After the user asks a question, entity linking is first performed to associate the entities in the question with the nodes of the knowledge graph, and then the attributes of the nodes are predicted by the question to find the answer to the question. Since the triples of the knowledge graph are a kind of structured knowledge and need to be manually sorted out, the construction of a large-scale knowledge graph is time-consuming and laborious. At the same time, it is difficult to abstract the long-tail questions of users into triple representations, and it is also difficult to solve such questions through the knowledge graph.

[0062] 3) Open-domain question-and-answer method based on documents: It is necessary to collect a large-scale document library in advance, and after obtaining the user's question, use a question-and-answer model to retrieve relevant documents of the user's question from the document library, and then extract or generate answers from the documents. When retrieving relevant documents from the document library, the question-and-answer model usually implements sparse retrieval schemes based on technologies such as TF-IDF and BM25 and dense vector retrieval schemes based on semantic similarity. Although sparse retrieval is efficient, its accuracy is low; dense vector retrieval encodes the question and the document into semantic vectors, and then calculates the similarity between the question and the document according to the semantic vectors to select the document most relevant to the question. Although dense vector retrieval has higher accuracy, it needs to rely on a large amount of labeled data during the training process, facing the problems of difficult acquisition of high-quality labeled data and low efficiency.

[0063] Based on this, a document question-and-answer method is provided in the embodiments of the present invention.

[0064] Figure 1 The flowchart of a document question-answering method provided in an embodiment of the present invention is shown as Figure 1 follows. The method includes:

[0065] S1. Obtain a user question;

[0066] S2. Input the user question into a rough ranking model to obtain multiple candidate documents retrieved by the rough ranking model from a document library;

[0067] S3. Input the user question and the multiple candidate documents into a fine ranking model to obtain the similarity between each candidate document output by the fine ranking model and the user question, and determine a target document whose similarity to the user question is higher than a preset threshold;

[0068] S4. Based on the user question, the target document, and the similarity between the target document and the user question, determine a target answer corresponding to the user question;

[0069] Among them, the training steps of the rough ranking model and the fine ranking model include:

[0070] Pre-train an initial rough ranking model and an initial fine ranking model based on content document pairs corresponding to the document library to obtain a pre-trained rough ranking model and a pre-trained fine ranking model respectively; the content document pairs are determined based on the target content of any document in the document library and the documents in the document library;

[0071] Jointly train the pre-trained rough ranking model and the pre-trained fine ranking model based on a set of question-document pairs to obtain the rough ranking model and the fine ranking model.

[0072] Specifically, for the document question-answering method provided in an embodiment of the present invention, the execution subject is a document question-answering device, which can be configured in a computer. The computer can be a local computer or a cloud computer. The local computer can be a computer, a tablet, etc., and no specific limitation is made here.

[0073] First, step S1 is executed to obtain a user question. The user question can be collected through a voice module or directly obtained through an external interface. That is, the user question can be in the form of voice or text, and no specific limitation is made here. It can be understood that in the embodiment of the present invention, document question-answering is performed. Therefore, if the user question is in the form of voice, it needs to be converted into text form, and the conversion method can adopt a conventional conversion method.

[0074] Then, step S2 is executed. A document question-answering model can be introduced. The document question-answering model can include a rough ranking model and a fine ranking model. The user's question can be input into the rough ranking model, and the rough ranking model calculates the similarity between the user's question and each document in the target document library, and retrieves multiple candidate documents from the document library. That is, the input of the rough ranking model can include the user's question and each document in the target document library, and the output is multiple candidate documents.

[0075] The document library can be a large-scale document library, for example, it can reach a scale of millions or even tens of millions. The document library can contain documents in various fields, such as, but not limited to, fields of encyclopedic knowledge, news, social conversations, Internet Q&A, etc., covering as comprehensively as possible. Therefore, this document question-answering method can achieve open-ended question answering.

[0076] Since encoding a large-scale document library requires a large amount of resources and time, each document in the document library can be pre-input into the rough ranking model, and the feature information of each document can be obtained through offline encoding. Tools such as faiss can also be used to build indexes for each document. Thus, the rough ranking model can be a two-tower structure, that is, it can include two encoders: a question encoder q-encoder and a document encoder p-encoder. The question encoder encodes online, and the document encoder encodes offline. Both of these encoders can be bidirectional language models such as BERT. Through the rough ranking model with a two-tower structure, the recall problem can be taken into account and the efficiency can be improved.

[0077] The rough ranking model can first encode the user's question to obtain the feature information of the user's question, and then calculate the similarity between the user's question and each document through the feature information of the user's question and the feature information of each document. Furthermore, multiple candidate documents can be selected from the document library according to the high and low of this similarity. Here, the top several documents with high similarity can be directly selected from the target document library as candidate documents, such as the top-K with high similarity. When selecting candidate documents, tools such as faiss can be used to retrieve from the pre-established index.

[0078] After that, step S3 is executed. The user's question and multiple candidate documents are input into the fine ranking model. The fine ranking model further calculates the similarity between each candidate document and the user's question, and then can sort each candidate document in descending order of the similarity to the user's question, and select the candidate documents with a similarity greater than a preset threshold from the multiple candidate documents as the target document. That is, the input of the fine ranking model can be the concatenation result of the user's question and each candidate document, and the output is the similarity between each candidate document and the user's question.

[0079] It is understandable that there can be one or more target documents. When there is one target document, it means that there is only one document in the target document library whose similarity to the user's question is higher than the preset threshold. When there are multiple target documents, it means that there are multiple documents in the target document library whose similarity to the user's question is higher than the preset threshold, and the multiple target documents can be sorted in descending order of similarity to the user's question.

[0080] Here, the fine-ranking model can only include one encoder, which can be a cross-encoder, perform deep interaction encoding on the concatenation result, and obtain the similarity between each candidate document and the user's question based on the encoding result, and output the similarity through a fully connected layer. The fine-ranking model can also be a two-way language model such as BERT, and its loss function can be a classification loss.

[0081] The similarity between each candidate document passage and the user's question query can be calculated through the following formula:

[0082] score(query,passage)=BERT(query;passage).

[0083] Among them, score(query,passage) represents the similarity between the candidate document passage and the user's question query, and query;passage represents the concatenation result of the user's question query and the candidate document passage.

[0084] The rough-ranking model and the fine-ranking model can be trained through the following steps:

[0085] First, pre-train the initial rough-ranking model and the initial fine-ranking model according to the content document pairs corresponding to the document library, and obtain the pre-trained rough-ranking model and the pre-trained fine-ranking model respectively.

[0086] The content document pairs can include positive sample pairs and negative sample pairs. The positive sample pairs can be question-answer pairs, and the negative sample pairs can be question-non-answer pairs. The positive sample pairs can be determined by the target content of any document and any document in the document library, and the negative sample pairs can be determined by the target content of any document and the remaining documents in the document library except this any document.

[0087] It is understandable that any document in the document library can be a randomly sampled document. The target content can be the title of this any document or a sentence in this any document.

[0088] In the positive sample pairs, the positive sample pair can be directly formed by the target content of any document p i and any document p i That is, the positive sample pair is formed by any document p iThe target content is used as the question q, and any one of the documents p i is used as the answer. At this time, the positive sample pair can be expressed as (q, p i ).

[0089] In the negative sample pairs, any one of the documents p i 's target content and any one of the remaining documents p j constitute a negative sample pair, that is, the target content of any one of the documents p i is used as the question q, and any one of the remaining documents p j is used as the non-answer. At this time, the negative sample pair can be expressed as (q, p j ). It is also possible to recall documents from the document library according to the question q through the Term Frequency–Inverse Document Frequency (TF-IDF) algorithm or the BM25 algorithm, and screen out M remaining documents p1, p m , …, p M with scores less than the first threshold. Then, the target content of any one of the documents p i and any one of the M remaining documents p m constitute a negative sample pair, that is, any one of the M remaining documents p m is used as the non-answer. At this time, the negative sample pair can be expressed as (q, p m ). The specific value of this first threshold can be set as needed and is not specifically limited here.

[0090] Since the target content is not a real question but an equivalent question, it can be called a pseudo-question. Therefore, the content-document pairs can be used as weakly supervised training samples to achieve weakly supervised pre-training of the document question-answering model. In this way, the document question-answering model can learn more sufficient knowledge without introducing manual annotation, which can save the model training cost. It can be understood that the process of this weakly supervised pre-training can be a contrastive learning process, that is, the loss function used in the weakly supervised pre-training is a loss function in the form of contrastive learning.

[0091] The question-document pair set refers to a set containing several question-document pairs. The number of question-document pairs contained in the question-document pair set can be set as needed and is not specifically limited here, as long as it meets the requirement of fine-tuning the model after weakly supervised pre-training. A question-document pair consists of a question sample and a document sample corresponding to the question sample. The question sample refers to a real question and can be obtained through manual annotation.

[0092] Finally, step S4 is executed. First, the target document can be understood through the user's question to determine the answer fragment in the target document. Here, the specific value of this preset threshold can be set as needed and is not specifically limited here.

[0093] After determining the answer fragments in the target document, it is possible to judge whether the answer fragments in the target document can answer the user's question by combining the similarity between the target document and the user's question. The answer fragments in the target document that can answer the user's question are output as the target answer corresponding to the user's question.

[0094] In the document question-answering method provided in the embodiments of the present invention, first, the user's question is obtained; then, multiple candidate documents retrieved from the document library by using the rough ranking model are used, and the similarity between each candidate document and the user's question is obtained by using the fine ranking model, and the target document with a similarity higher than a preset threshold to the user's question is determined, and then the target answer corresponding to the user's question is determined. The samples used in the training processes of the rough ranking model and the fine ranking model may include the content document pairs corresponding to the document library and the set of question-document pairs. Through the content document pairs, the pre-trained rough ranking model and the pre-trained fine ranking model can learn more sufficient knowledge without introducing manual annotation, which can not only save the model training cost and improve the training efficiency, but also solve the problem of insufficient high-quality labeled data. Through the set of question-document pairs, it can be ensured that the pre-trained rough ranking model and the pre-trained fine ranking model can be trained into a rough ranking model and a fine ranking model that can accurately screen out the target document with less labeled data, and then accurate target answers can be provided for users. In addition, since the document library used in this method is not restricted by the field, open-ended question answering can be realized, and it has strong scalability.

[0095] Based on the above embodiments, in the document question-answering method provided in the embodiments of the present invention, the joint training of the pre-trained rough ranking model and the pre-trained fine ranking model based on the set of question-document pairs to obtain the rough ranking model and the fine ranking model includes:

[0096] Based on the first type of question-document pairs in the set of question-document pairs, the pre-trained rough ranking model and the pre-trained fine ranking model are jointly trained to obtain a baseline rough ranking model and a baseline fine ranking model respectively;

[0097] Based on the first type of question-document pairs, the difficult negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model are determined, and based on the difficult negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model, the baseline rough ranking model and the baseline fine ranking model are iteratively trained to obtain the rough ranking model and an alternative fine ranking model;

[0098] Based on the alternative fine ranking model, the fine ranking model is determined.

[0099] Specifically, in the embodiments of the present invention, through the problem-document pair set, the coarse ranking model and the fine ranking model can be jointly trained. That is, first, according to the first type of problem-document pairs in the problem-document pair set, the pre-trained coarse ranking model and the pre-trained fine ranking model can be jointly trained to obtain a baseline coarse ranking model and a baseline fine ranking model respectively. This joint training process can be understood as a process of fine-tuning the pre-trained coarse ranking model and the pre-trained fine ranking model obtained by weakly supervised pre-training according to the first type of problem-document pairs.

[0100] It can be understood that the problem-document pair set can include the first type of problem-document pairs and the second type of problem-document pairs. The first type of problem-document pairs and the second type of problem-document pairs can be the same or different, and are only used here to distinguish different training processes. In addition, the second type of problem-document pairs also carry answer segment labels.

[0101] According to the first type of problem-document pairs, the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model can be determined first. For example, the problem samples in the first type of problem-document pairs can be input into the baseline coarse ranking model, and the output result of the baseline coarse ranking model can be input into the baseline fine ranking model, and the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model can be determined according to the output result of the baseline fine ranking model. Hard negative samples refer to negative samples that are easily judged as positive samples, that is, pseudo-positive samples.

[0102] After that, according to the first type of problem-document pairs, and the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model, the baseline coarse ranking model and the baseline fine ranking model can be iteratively trained to obtain a coarse ranking model and an alternative fine ranking model. The process of this iterative training means that in each iteration round, the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model will be determined and will be used for the training of the baseline coarse ranking model and the baseline fine ranking model in the next iteration round. In the first iteration round, the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model can be obtained by processing the first type of problem-document pairs through techniques such as TF-IDF and BM25.

[0103] After that, the fine ranking model can be determined according to the alternative fine ranking model. For example, the alternative fine ranking model can be directly used as the fine ranking model, or the alternative fine ranking model and the initial understanding model can be jointly trained to determine the fine ranking model and the understanding model respectively, which is not specifically limited here.

[0104] In the embodiments of the present invention, by jointly training the initial coarse ranking model and the initial fine ranking model, not only can the training efficiency be improved and the training cycle be shortened, but also the recall effect of the coarse ranking model can be effectively improved; by introducing the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model, the sample quality can be effectively improved, and thus the accuracy of the obtained coarse ranking model and alternative fine ranking model can be higher.

[0105] Based on the above embodiments, in the document question-answering method provided in the embodiments of the present invention, determining the target answer corresponding to the user question based on the user question, the target document, and the similarity between the target document and the user question includes:

[0106] Input the user question and the target document into an understanding model, and obtain answer fragments within the target document output by the understanding model;

[0107] Based on the user question, the answer fragments, and the similarity between the target document and the user question, determine the target answer;

[0108] Among them, the refined ranking model and the understanding model are determined based on the following steps:

[0109] Based on the second type of question-document pairs with answer fragment tags in the question-document pair set, jointly train the alternative refined ranking model and the initial understanding model to obtain the refined ranking model and the understanding model.

[0110] Specifically, in the embodiments of the present invention, the document question-answering model may further include an understanding model and a generation model.

[0111] The understanding model is used to determine answer fragments within each target document. The input of this understanding model is the same as that of the refined ranking model, which is the concatenation result of the user question and each target document, and the output is the answer fragments within each target document. This understanding model may also only include an encoder, which is used to encode the concatenation result of the user question and each target document, and output the answer fragments obtained according to the encoding result through a fully connected layer. The answer fragments can be represented by the start position and end position within the target document.

[0112] Among them, based on the second type of question-document pairs with answer fragment tags in the question-document pair set, the alternative refined ranking model and the initial understanding model can be jointly trained to obtain the refined ranking model and the understanding model.

[0113] The second type of question-document pairs may include: 1) Processing an open-source Chinese reading comprehension dataset into the form of questions and answers, with the target being the fragments of the answers in the document; 2) Questions collected from the Internet and question data of a voice interaction system, and retrieving the corresponding documents from a document library, and obtaining answer fragments after manual annotation.

[0114] For example, the question samples in the second type of question-document pairs can be input into the alternative fine-ranking model to obtain the first result output by the alternative fine-ranking model, and the first result can be input into the initial understanding model to obtain the second result output by the initial understanding model. Then, the first loss is calculated using the document samples in the second type of question-document pairs and the first result, and the second loss is calculated using the answer fragment tags and the second result. The parameters of the alternative fine-ranking model are iterated according to the first loss, and the parameters of the initial understanding model are iterated according to the second loss until the first loss and the second loss converge, and the training is ended.

[0115] In the embodiments of the present invention, by jointly training the alternative fine-ranking model and the initial understanding model, the accuracy of the target answer can be effectively improved.

[0116] Based on the above embodiments, in the document question-answering method provided in the embodiments of the present invention, the encoder parameters of the alternative fine-ranking model and the initial understanding model are shared.

[0117] Specifically, in the embodiments of the present invention, both the alternative fine-ranking model and the initial understanding model only include one encoder. Therefore, the encoder parameters of the two can be shared, that is, the two share the same encoder, and the corresponding functions are respectively implemented by the fully connected layers connected after the encoder.

[0118] Therefore, when jointly training the alternative fine-ranking model and the initial understanding model, multi-task learning can be performed on the alternative fine-ranking model and the initial understanding model according to the second type of question-document pairs in the question-document pair set, and then the fine-ranking model and the understanding model can be obtained. On the one hand, the online understanding duration can be shortened, and on the other hand, the training data of each task can be used mutually to improve the overall effect of the fine-ranking model and the understanding model.

[0119] Based on the above embodiments, in the document question-answering method provided in the embodiments of the present invention, determining the target answer based on the user question, the answer fragment, and the similarity between the target document and the user question includes:

[0120] Inputting the user question, the answer fragment, and the similarity between the target document and the user question into a generation model to obtain the target answer output by the generation model;

[0121] Wherein, the generation model is trained based on question-answer fragment pairs and the similarity between the document samples where the answer fragment samples in the question-answer fragment pairs are located and the question samples in the question-answer fragment pairs.

[0122] Specifically, the generation model is used to determine whether the answer segments in each target document can answer the user's question, and output the answer segments that can answer the user's question, that is, the target answer corresponding to the user's question. The input of the generation model includes the user's question, the answer segments in each target document, and the similarity between each target document and the user's question, and the output is the target answer corresponding to the user's question.

[0123] The generation model may also include only one encoder, which may also be a cross-encoder, to perform in-depth interactive encoding on the concatenated result of the input. The generation model can sort and output the target answers according to the similarity between each target answer and the user's question from high to low, or may not output all sorted answers, but only select several target answers with high similarity to the user's question or select target answers with similarity higher than the second threshold to the user's question by setting the second threshold for output. For example, the target answer with the highest similarity to the user's question can be directly selected for output and provided to the user.

[0124] Among them, the generation model can be trained through question-answer passage pairs and the similarity between the document samples where the answer passage samples in the question-answer passage pairs are located and the question samples in the question-answer passage pairs. Its training process may include: inputting the question samples in the question-answer passage pairs, the document samples where the answer passage samples in the question-answer passage pairs are located, and the similarity between the document samples and the question samples in the question-answer passage pairs into the initial generation model to obtain the output result of the initial generation model, then calculating the loss function value according to the output result and the answer passage samples in the question-answer passage pairs, and finally updating the model parameters of the initial generation model according to the loss function value; iteratively execute the above input process and calculation process until the loss function converges or reaches the preset number of iterations to obtain the generation model. It can be understood that the initial generation model can be the basic generation model in the basic question-answering model or other initial neural network models, which is not specifically limited here.

[0125] When training the initial generation model, the question-answer segment pairs used may include reading comprehension data sets and question-answer pairs collected from Internet community Q&A data. After the Internet community Q&A data is cleaned, the remaining question-answer pairs can be used as positive samples; for negative samples, one is to randomly select answers for each question, and the other is to select negative samples according to the literal similarity between the question and the answer.

[0126] The initial generation model may have the same structure as the fine-ranking model, that is, both include an encoder, and the encoder may be a cross-encoder.

[0127] In the embodiments of the present invention, rich question-answer segment pairs can ensure the accuracy of the generation model.

[0128] Based on the above embodiments, in the document question-answering method provided in the embodiments of the present invention, determining the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine-ranking model based on the first type of question-document pairs includes:

[0129] Input the question samples in the first type of question-document pairs into the baseline coarse-ranking model to obtain multiple associated documents retrieved by the baseline coarse-ranking model from the document library;

[0130] Input the multiple associated documents into the baseline fine-ranking model to obtain the similarity between each associated document output by the baseline fine-ranking model and the question sample;

[0131] Based on the similarity between each associated document and the question sample, determine the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine-ranking model.

[0132] Specifically, in the embodiments of the present invention, when determining the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine-ranking model, the question samples in the first type of question-document pairs can be first input into the baseline coarse-ranking model to obtain multiple associated documents retrieved by the baseline coarse-ranking model from the document library. For example, the top N documents with high similarity to the question samples in the document library can be used as the associated documents, which are respectively represented as p1, p2,..., p N After that, the top N associated documents can be input into the baseline fine-ranking model to obtain the similarity between each associated document output by the baseline fine-ranking model and the question sample. For example, the similarities can be respectively represented as s1, s2,..., s N

[0133] Finally, according to the similarity between each associated document and the question sample, the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine-ranking model can be determined. Here, a third threshold λ - and a fourth threshold λ + can be introduced, and the third threshold λ - is less than the fourth threshold λ + . The specific values of the third threshold λ - and the fourth threshold λ + can be set as needed and are not specifically limited here.

[0134] For any associated document, if the similarity between the any associated document and the question sample is less than the third threshold, it can be determined that the any associated document and the question sample constitute the hard negative samples corresponding to the baseline fine-ranking model; if the similarity between the any associated document and the question sample is greater than the fourth threshold, it can be determined that the any associated document and the question sample constitute the pseudo-negative samples corresponding to the baseline fine-ranking model.

[0135] ​In the embodiments of the present invention, by means of a baseline rough ranking model and a baseline fine ranking model, difficult negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model are determined, adding samples for the iterative training of the baseline rough ranking model and the baseline fine ranking model, and ensuring the accuracy of the rough ranking model and the fine ranking model.

[0136] Based on the above embodiments, in the document question answering method provided in the embodiments of the present invention, the inputting the user question into the rough ranking model to obtain a plurality of candidate documents retrieved by the rough ranking model from the document library includes:

[0137] Inputting the user question into the question encoder of the rough ranking model to obtain a question encoding vector output by the question encoder;

[0138] Inputting the question encoding vector and the document encoding vectors corresponding to each document in the document library obtained based on the document encoder of the rough ranking model into the similarity calculation layer of the rough ranking model to obtain the similarity between the user question and each document output by the similarity calculation layer;

[0139] Inputting the similarity between the user question and each document into the output layer to obtain the plurality of candidate documents with high similarity in each document output by the output layer.

[0140] Specifically, in the embodiments of the present invention, the rough ranking model may be a two-tower structure, that is, it includes a question encoder q-encoder and a document encoder p-encoder, and also includes a similarity calculation layer and an output layer. The question encoder q-encoder and the document encoder p-encoder are respectively connected to the similarity calculation layer, and the similarity calculation layer is connected to the output layer.

[0141] After inputting the user question query into the rough ranking model, the question encoder q-encoder can perform semantic encoding on the user question query to obtain a question encoding vector E q (query), and then calculate the product of the question encoding vector E q (query) and the document encoding vector E p (passage0) corresponding to each document passage0 in the document library through the similarity calculation layer. This product represents the similarity sim(query, passage0) between the user question query and each document passage0.

[0142] Wherein,

[0143] E q (query) = BERT q (query),

[0144] Ep (passage0) = BERT p (passage0),

[0145] sim(query, passage0) = E q (query) · E p (passage0).

[0146] Figure 2 This is a schematic flow chart of a document question - answering method implemented based on a document question - answering model provided in an embodiment of the present invention. As Figure 2 shown, the method includes:

[0147] Input the user question query and the document library into the document question - answering model;

[0148] Retrieve the top K target documents from the document library through the rough - ranking model in the document question - answering model;

[0149] Through the fine - ranking model and the understanding model in the document question - answering model, simultaneously encode the user question and the top K target documents, extract answer fragments from each target document according to the user question and sort them;

[0150] Determine the target answer corresponding to the user question through the generation model in the document question - answering model and output it.

[0151] Figure 3 This is a schematic structural diagram of the rough - ranking model in the document question - answering model provided in an embodiment of the present invention. As Figure 3 shown, the rough - ranking model is a two - tower structure, including a question encoder q - encoder and a document encoder p - encoder, and also including a similarity calculation layer 31 and an output layer 32. The question encoder q - encoder and the document encoder p - encoder are respectively connected to the similarity calculation layer 31, and the similarity calculation layer 31 is connected to the output layer 32.

[0152] Figure 4 This is a schematic structural diagram of the fine - ranking model in the document question - answering model provided in an embodiment of the present invention. As Figure 4 shown, the inputs of the fine - ranking model and the understanding model are both the concatenation results of the user question query and each alternative document passage. After passing through the cross - encoder cross - encoder, they respectively output the similarity score score between the user question query and each target document, the start (start) position and the end (end) position of the answer fragment corresponding to the user question query in each target document.

[0153] As Figure 5As shown, based on the above embodiments, an embodiment of the present invention provides a document question-answering device, including:

[0154] An acquisition module 51, configured to acquire a user question;

[0155] A rough ranking module 52, configured to input the user question into a rough ranking model to obtain multiple candidate documents retrieved by the rough ranking model from a document library;

[0156] A fine ranking module 53, configured to input the user question and the multiple candidate documents into a fine ranking model to obtain the similarity between each candidate document output by the fine ranking model and the user question, and determine a target document whose similarity to the user question is higher than a preset threshold;

[0157] A determination module 54, configured to determine a target answer corresponding to the user question based on the user question, the target document, and the similarity between the target document and the user question;

[0158] A training module 55, configured to:

[0159] Pre-train an initial rough ranking model and an initial fine ranking model based on content document pairs corresponding to the document library to obtain a pre-trained rough ranking model and a pre-trained fine ranking model respectively; the content document pairs are determined based on the target content of any document in the document library and the documents in the document library;

[0160] Jointly train the pre-trained rough ranking model and the pre-trained fine ranking model based on a set of question-document pairs to obtain the rough ranking model and the fine ranking model.

[0161] Based on the above embodiments, for the document question-answering device provided in an embodiment of the present invention, the training module is configured to:

[0162] Jointly train the pre-trained rough ranking model and the pre-trained fine ranking model based on the first type of question-document pairs in the set of question-document pairs to obtain a baseline rough ranking model and a baseline fine ranking model respectively;

[0163] Determine difficult negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model based on the first type of question-document pairs, and iteratively train the baseline rough ranking model and the baseline fine ranking model based on the difficult negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model to obtain the rough ranking model and an alternative fine ranking model;

[0164] Determine the fine ranking model based on the alternative fine ranking model.

[0165] Based on the above embodiments, for the document question-answering device provided in an embodiment of the present invention, the training module is configured to:

[0166] Input the user question and the target document into the understanding model to obtain the answer fragment within the target document output by the understanding model;

[0167] Determine the target answer based on the user question, the answer fragment, and the similarity between the target document and the user question;

[0168] Among them, the refined ranking model and the understanding model are determined based on the following steps:

[0169] Based on the second type of question-document pairs in the question-document pair set carrying answer fragment tags, jointly train the alternative refined ranking model and the initial understanding model to obtain the refined ranking model and the understanding model.

[0170] Based on the above embodiments, in the document question-answering device provided in the embodiments of the present invention, the encoder parameters of the alternative refined ranking model and the initial understanding model are shared.

[0171] Based on the above embodiments, in the document question-answering device provided in the embodiments of the present invention, the determining module is specifically configured to:

[0172] Input the user question, the answer fragment, and the similarity between the target document and the user question into the generation model to obtain the target answer output by the generation model;

[0173] Among them, the generation model is trained based on the question-answer fragment pairs and the similarity between the document sample where the answer fragment sample in the question-answer fragment pair is located and the question sample in the question-answer fragment pair.

[0174] Based on the above embodiments, in the document question-answering device provided in the embodiments of the present invention, the training module is used for:

[0175] Input the question samples in the first type of question-document pairs into the baseline rough ranking model to obtain multiple associated documents retrieved by the baseline rough ranking model from the document library;

[0176] Input the multiple associated documents into the baseline refined ranking model to obtain the similarity between each associated document output by the baseline refined ranking model and the question sample;

[0177] Based on the similarity between each associated document and the question sample, determine the difficult negative samples and / or pseudo-negative samples corresponding to the baseline refined ranking model.

[0178] Based on the above embodiments, in the document question-answering device provided in the embodiments of the present invention, the rough ranking module is specifically configured to:

[0179] Input the user question into the question encoder of the rough ranking model to obtain the question encoding vector output by the question encoder;

[0180] Input the question encoding vector and the document encoding vectors corresponding to each document in the document library obtained based on the document encoder of the rough ranking model into the similarity calculation layer of the rough ranking model to obtain the similarity between the user question and each document output by the similarity calculation layer;

[0181] Input the similarity between the user question and each document into the output layer to obtain multiple candidate documents with high similarity among the documents output by the output layer.

[0182] Specifically, in the document Q&A device provided in the embodiments of the present invention, the functions of each module correspond one-to-one to the operation processes of each step in the above method embodiments, and the achieved effects are also the same. For details, refer to the above embodiments, and the embodiments of the present invention will not be elaborated herein.

[0183] As Figure 6 shown, on the basis of the above embodiments, the embodiments of the present invention provide a document Q&A system, including: a voice module 61 and the document Q&A device 62 provided in each of the above embodiments, and the voice module 61 is connected to the document Q&A device 62; the voice module 61 is used to collect user questions and transmit the user questions to the document Q&A device 62.

[0184] Specifically, in the embodiments of the present invention, the user can output the user question by voice, and the voice signal of the user question can be collected by the voice module 61, and then the voice signal can be sent to the document Q&A device 62. The document Q&A device 62 then converts the voice signal into text and executes the document Q&A method to provide the target answer corresponding to the user question to the user.

[0185] Here, the voice module 61 can also directly convert the voice signal into text and send the user question in text form to the document Q&A device 62.

[0186] The document Q&A system provided in the embodiments of the present invention combines the voice module with the document Q&A device, enabling the document Q&A system to implement the functions of a voice Q&A system for voice Q&A and improving the user experience.

[0187] Figure 7 Illustrates a schematic physical structure diagram of an electronic device, as Figure 7As shown in the figure, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communications interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logical instructions in the memory 730 to execute the document Q&A method provided in each of the above embodiments. The method includes: obtaining a user question; inputting the user question into a rough ranking model to obtain multiple candidate documents retrieved by the rough ranking model from a document library; inputting the user question and the multiple candidate documents into a fine ranking model to obtain the similarity between each candidate document output by the fine ranking model and the user question, and determining a target document whose similarity to the user question is higher than a preset threshold; based on the user question, the target document, and the similarity between the target document and the user question, determining a target answer corresponding to the user question; where the training steps of the rough ranking model and the fine ranking model include: pre-training an initial rough ranking model and an initial fine ranking model based on the content document pairs corresponding to the document library to obtain a pre-trained rough ranking model and a pre-trained fine ranking model respectively; the content document pairs are determined based on the target content of any document in the document library and the documents in the document library; based on the set of question-document pairs, jointly training the pre-trained rough ranking model and the pre-trained fine ranking model to obtain the rough ranking model and the fine ranking model.

[0188] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0189] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the document Q&A method provided in the above embodiments. The method includes: obtaining a user question; inputting the user question into a rough ranking model to obtain multiple candidate documents retrieved by the rough ranking model from a document library; inputting the user question and the multiple candidate documents into a fine ranking model to obtain the similarity between each candidate document output by the fine ranking model and the user question, and determining a target document whose similarity to the user question is higher than a preset threshold; determining a target answer corresponding to the user question based on the user question, the target document, and the similarity between the target document and the user question. Wherein, the training steps of the rough ranking model and the fine ranking model include: pre-training an initial rough ranking model and an initial fine ranking model based on the content document pairs corresponding to the document library to obtain a pre-trained rough ranking model and a pre-trained fine ranking model respectively; the content document pairs are determined based on the target content of any document in the document library and the documents in the document library; jointly training the pre-trained rough ranking model and the pre-trained fine ranking model based on the set of question-document pairs to obtain the rough ranking model and the fine ranking model.

[0190] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the document Q&A method provided in the above embodiments. The method includes: obtaining a user question; inputting the user question into a rough ranking model to obtain multiple candidate documents retrieved by the rough ranking model from a document library; inputting the user question and the multiple candidate documents into a fine ranking model to obtain the similarity between each candidate document output by the fine ranking model and the user question, and determining a target document whose similarity to the user question is higher than a preset threshold; determining a target answer corresponding to the user question based on the user question, the target document, and the similarity between the target document and the user question. Wherein, the training steps of the rough ranking model and the fine ranking model include: pre-training an initial rough ranking model and an initial fine ranking model based on the content document pairs corresponding to the document library to obtain a pre-trained rough ranking model and a pre-trained fine ranking model respectively; the content document pairs are determined based on the target content of any document in the document library and the documents in the document library; jointly training the pre-trained rough ranking model and the pre-trained fine ranking model based on the set of question-document pairs to obtain the rough ranking model and the fine ranking model.

[0191] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative effort.

[0192] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A document question answering method, characterized in that, Including: Obtain the user's question; Input the user's question into the rough ranking model to obtain multiple candidate documents retrieved by the rough ranking model from the document library; Input the user's question and the multiple candidate documents into the fine ranking model to obtain the similarity between each candidate document output by the fine ranking model and the user's question, and determine the target document whose similarity to the user's question is higher than the preset threshold; Based on the user's question, the target document, and the similarity between the target document and the user's question, determine the target answer corresponding to the user's question; Among them, the training steps of the rough ranking model and the fine ranking model include: Based on the content document pairs corresponding to the document library, pre-train the initial rough ranking model and the initial fine ranking model respectively to obtain the pre-trained rough ranking model and the pre-trained fine ranking model; the content document pairs are determined based on the target content of any document in the document library and the documents in the document library; Based on the set of question-document pairs, jointly train the pre-trained rough ranking model and the pre-trained fine ranking model to obtain the rough ranking model and the fine ranking model.

2. The document Q&A method according to claim 1, wherein The step of jointly training the pre-trained rough ranking model and the pre-trained fine ranking model based on the set of question-document pairs to obtain the rough ranking model and the fine ranking model includes: Based on the first type of question-document pairs in the set of question-document pairs, jointly train the pre-trained rough ranking model and the pre-trained fine ranking model to obtain a baseline rough ranking model and a baseline fine ranking model respectively; Based on the first type of question-document pairs, determine the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model, and based on the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine ranking model, iteratively train the baseline rough ranking model and the baseline fine ranking model to obtain the rough ranking model and an alternative fine ranking model; Based on the alternative fine ranking model, determine the fine ranking model.

3. The document Q&A method according to claim 2, wherein The step of determining the target answer corresponding to the user's question based on the user's question, the target document, and the similarity between the target document and the user's question includes: Input the user's question and the target document into the understanding model to obtain the answer fragment in the target document output by the understanding model; Based on the user's question, the answer fragment, and the similarity between the target document and the user's question, determine the target answer; Among them, the fine ranking model and the understanding model are determined based on the following steps: Based on the second type of question-document pairs with answer fragment labels in the set of question-document pairs, jointly train the alternative fine ranking model and the initial understanding model to obtain the fine ranking model and the understanding model.

4. The document Q&A method according to claim 3, wherein The encoder parameters of the alternative fine ranking model and the initial understanding model are shared.

5. The document Q&A method according to claim 3, wherein The step of determining the target answer based on the user's question, the answer fragment, and the similarity between the target document and the user's question includes: Input the user's question, the answer fragment, and the similarity between the target document and the user's question into the generation model to obtain the target answer output by the generation model; Among them, the generation model is trained based on the question-answer snippet pairs and the similarity between the document samples where the answer snippet samples in the question-answer snippet pairs are located and the question samples in the question-answer snippet pairs.

6. The document Q&A method according to claim 2, wherein Determining the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine-ranking model based on the first type of question-document pairs includes: Inputting the question samples in the first type of question-document pairs into the baseline coarse-ranking model to obtain multiple associated documents retrieved by the baseline coarse-ranking model from the document library; Inputting the multiple associated documents into the baseline fine-ranking model to obtain the similarity between each associated document output by the baseline fine-ranking model and the question sample; Based on the similarity between each associated document and the question sample, determining the hard negative samples and / or pseudo-negative samples corresponding to the baseline fine-ranking model.

7. The document Q&A method according to any one of claims 1-6, characterized in that, The step of inputting the user question into the coarse-ranking model to obtain multiple alternative documents retrieved by the coarse-ranking model from the document library includes: Inputting the user question into the question encoder of the coarse-ranking model to obtain the question encoding vector output by the question encoder; Inputting the question encoding vector and the document encoding vectors corresponding to each document in the document library obtained based on the document encoder of the coarse-ranking model into the similarity calculation layer of the coarse-ranking model to obtain the similarity between the user question and each document output by the similarity calculation layer; Inputting the similarity between the user question and each document into the output layer to obtain the multiple alternative documents with high similarity in each document output by the output layer.

8. A document question answering device, characterized in that, It includes: An acquisition module for acquiring a user question; A coarse-ranking module for inputting the user question into a coarse-ranking model to obtain multiple alternative documents retrieved by the coarse-ranking model from the document library; A fine-ranking module for inputting the user question and the multiple alternative documents into a fine-ranking model to obtain the similarity between each alternative document output by the fine-ranking model and the user question, and determining a target document whose similarity to the user question is higher than a preset threshold; A determination module for determining a target answer corresponding to the user question based on the user question, the target document, and the similarity between the target document and the user question; A training module for: Pre-training an initial coarse-ranking model and an initial fine-ranking model based on the content-document pairs corresponding to the document library to obtain a pre-trained coarse-ranking model and a pre-trained fine-ranking model respectively; the content-document pairs are determined based on the target content of any document in the document library and the documents in the document library; Jointly training the pre-trained coarse-ranking model and the pre-trained fine-ranking model based on a set of question-document pairs to obtain the coarse-ranking model and the fine-ranking model.

9. A document question answering system, characterized in that, It includes: A voice module and the document question-answering device as claimed in claim 8; The voice module is connected to the document question-answering device; The voice module is used to collect a user question and transmit the user question to the document question-answering device.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the document question-answering method as described in any one of claims 1-7.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the document Q&A method according to any one of claims 1-7.

Images