Multi-hop reading understanding method and system based on paragraph pair loss and replication mechanism

Abstract

The invention discloses a multi-hop reading understanding method based on a paragraph pair loss and replication mechanism, and the method comprises the steps: respectively splicing a preset question with a plurality of paragraphs, and obtaining a spliced paragraph; screening the plurality of spliced paragraphs to obtain high-score paragraphs; splicing the high-score paragraph with a preset question again to obtain a high-score spliced paragraph; generating word list distribution and replication distribution based on cross attention scores for the high-score spliced paragraphs through a preset decoder, and dynamically fusing the two types of distribution to generate a prediction distribution probability; and determining a maximum value in the prediction distribution probability, and determining a corresponding prediction token in a preset word list according to an index corresponding to the maximum value so as to obtain a prediction text.

Description

Technical Field

[0001] The present invention relates to the technical field of reading comprehension in natural language processing, and in particular to a multi-hop reading comprehension method and system based on a paragraph pair loss and duplication mechanism. Background Art

[0002] The multi-hop reading comprehension task constructs complex multi-hop questions based on multiple paragraphs of text, requiring the machine to find clues about the questions from the context and answer the multi-hop questions through semantic-level reasoning. Most existing multi-hop reading comprehension methods adopt a two-stage framework consisting of paragraph screening and reading reasoning. The goal of paragraph screening is to filter irrelevant information from multiple paragraphs so that reading reasoning can focus on key information. Reading reasoning is the core module of multi-hop reading comprehension technology. It simulates human reasoning behavior through the interaction between questions and context, and predicts answers and evidence sentences based on the semantic features of the interaction. In particular, multi-hop reading comprehension is extractive, and its answers come from continuous text or "yes / no" in the context.

[0003] First, for the paragraph screening module, the basic approach is to concatenate the question and individual paragraphs, perform classification, and then select the highest-scoring paragraphs. However, this selection method only considers the relationship between the question and the paragraphs, ignoring the relationships between paragraphs—that is, the reasoning paths that exist between paragraphs. This can cause the screening process to miss relevant paragraphs. Tu et al., in their paper "Select, answer, and explain: Interpretable multi-hop reading comprehension over multiple documents," proposed an optimization objective for ranking paragraph scores, hoping that correct paragraphs would be ranked higher and thus more likely to be selected. However, this method still does not directly exploit the dependencies between paragraphs. To directly model the semantic connections between paragraphs, several two-stage screening methods have been proposed. Wu et al., in their paper "Graph-free multi-hop reading comprehension: A select-to-guide strategy," initially screen out high-scoring paragraphs, then concatenate them with the question to select other candidate paragraphs. Yin et al., in their paper "Rethinking Label Smoothing on Multi-hop Question Answering," applied a label smoothing strategy to the loss function of this multi-stage screening process, improving the generalization performance of the filter. However, the above method is designed for the HotpotQA (HotpotQA: A dataset for diverse, explainable multi-hop question answering) dataset, which has only two relevant paragraphs, and is difficult to extend to scenarios with more relevant paragraphs.

[0004] After obtaining the selected high-scoring paragraphs, the second stage requires reasoning based on the question and paragraph information, ultimately providing an answer to the question and providing evidence sentences to enhance interpretability. A common approach is to use a discriminative model to transform the prediction task into a classification task. This involves using a pre-trained model with an encoder architecture, such as BERT ("BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"), to acquire question and paragraph features. This model then allows for full interaction between the text features, and finally classifies the answer and evidence based on the interaction features. Specifically, the answer prediction task requires extracting a continuous segment from the context. This task is typically modeled as a classification task for each context token (symbol, word, etc.): determining whether each token represents the start and end position of the answer. The segment with the highest sum of the start and end position scores is the predicted answer. Evidence prediction is a binary classification task for each sentence, determining whether each sentence is an evidence sentence. Sentences with a classification score above a threshold are considered the set of predicted evidence sentences. Common approaches for text interaction include graph neural networks (GNNs) and attention mechanisms. Because graph structures can model complex relationships, GNNs are becoming increasingly popular for multi-hop reading comprehension tasks.

[0005] To accurately locate answer entities, Qiu et al. proposed constructing a dynamic entity graph in "Dynamically fused graph network for multi-hop reasoning" to propagate information from text to graph to text, achieving entity-level reasoning during the graph network iteration process. Tu et al. ("Select, answer and explain: Interpretable multi-hop reading comprehension over multiple documents") and Huang et al. ("Breadthfirst reasoning graph for multi-hop question answering") used sentence vectors as nodes and relationships between sentences as edges, achieving sentence-level reasoning through graph network updates, optimizing sentence features to improve the performance of evidence sentence selection. Fang et al. ("Hierarchical Graph Network for Multi-hop Question Answering") and Li et al. ("Asynchronous Multi-grained Graph Network for Interpretable Multi-hop Reading Comprehension") considered the connections between information at different granularities and constructed a more complex multi-granularity graph to enable cross-granular information exchange. However, graph neural networks inherently suffer from the problem of oversmoothing, which to some extent limits the performance of these graph-based methods. Furthermore, because constructing graph structures requires prior knowledge, which introduces noise and additional complexity, Shao et al. studied the necessity of graph attention in "Is graph structure necessary for multi-hop question answering?", arguing that graph neural networks are a special type of self-attention mechanism. Wu et al. further utilized and designed various attention mechanisms for multi-hop reading comprehension tasks in "Graph-free multi-hop reading comprehension: A select-to-guide strategy" to improve the model's ability to focus on important information. These discriminative model-based approaches require specific processing of inputs and outputs, specifically converting answers and evidence into corresponding category labels. Generative pre-trained models, on the other hand, can model different tasks as text-to-text generation tasks, avoiding the complex processing of specific downstream tasks. They are widely used in other areas of natural language processing, such as machine translation, text summarization, and free question answering.In "Modeling Multi-hop Question Answering as Single Sequence Prediction," Yavuz et al. modeled the multi-hop reading comprehension task as a sequence prediction task, leveraging the power of generative models to simultaneously predict both the answer and the evidence. However, when applied to multi-hop reading comprehension, while discriminative models can extract continuous segments from the context, generative models offer more flexibility in sequence prediction. However, this freedom is also unrestricted by the context, making it prone to producing context-independent or inconsistent results.

[0006] The Chinese invention patent with application number 202110810163.6 discloses "A keyword-driven dynamic graph neural network multi-hop reading comprehension method". The method is driven by keywords to extract keywords in the question and context respectively; the extracted keywords assist the model in selecting paragraphs related to the question, constructing a static reasoning graph between sentences, and guiding the model to accurately locate the answer span and evidence sentences; then the boundary-aware answer span loss is calculated to strengthen the boundary of the answer; in order to solve the problem of obstructed information transmission across paragraphs, sentences in the context are dynamically retrieved, and the retrieved sentences are used to dynamically update the reasoning graph and use the graph neural network model to transmit information in the reasoning graph; the answer span and evidence sentences are jointly trained, and finally the answer and evidence sentences are extracted and verified. Summary of the Invention

[0007] To address the problems of paragraph filters and generative readers in the background art, the present invention provides a multi-hop reading comprehension method and system based on a paragraph pair loss and duplication mechanism. The technical solutions adopted by the present invention are as follows: A first aspect of the present invention provides a multi-hop reading comprehension method based on a paragraph pair loss and duplication mechanism, the method comprising: Splice the preset questions with several paragraphs respectively to obtain spliced ​​paragraphs; Screening the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs; Splicing the high-scoring paragraphs with the preset questions again to obtain high-scoring spliced ​​paragraphs; The high-scoring spliced ​​paragraph is passed through a preset decoder to generate a vocabulary distribution and a copy distribution based on the cross-attention score, and the two types of distributions are dynamically integrated to generate a predicted distribution probability; The maximum value in the predicted distribution probability is determined, and the corresponding prediction token is determined in the preset vocabulary according to the index corresponding to the maximum value, so as to obtain the predicted text.

[0008] As a preferred solution, the method of screening the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs includes: Extract problem features separately through pre-training model and paragraph features ; The paragraph features are updated using the cross attention operation to obtain ; Pool the updated paragraph features and generate the interaction features between paragraphs through the self-attention mechanism ; based on Calculating binary classification loss and paragraph pair loss , where the paragraph pair labels are obtained by adding the paragraph binary labels, and the loss function is the weighted sum of the two .

[0009] As a preferred solution, the label value of the paragraph pair loss is 0, 1 or 2, and the paragraph feature pairs are concatenated and input into a multi-layer perception mechanism for ternary classification prediction. The paragraph pair loss is a cross entropy loss.

[0010] As a preferred solution, the method of screening the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs further includes: Perform softmax normalization on the binary classification output distribution to obtain the probability that each paragraph is a related paragraph; The three paragraphs with the highest scores are selected as output according to the probability ranking.

[0011] As a preferred solution, the method of generating a predicted distribution probability by dynamically fusing the vocabulary distribution and the copy distribution based on the cross attention score generated by the high-scoring spliced ​​paragraph through a preset decoder includes: Concatenate the question and the selected high-scoring paragraphs into a sequence and input it into the encoder to obtain features; During the decoding process, the word list distribution is generated through the decoder output , and the replication distribution calculated based on the cross-attention score ;in Represents the word embedding vector of the t-th token, It is the output vector of the token after passing through multiple layers of decoder. is the weight of the linear layer, represents the cross attention score between the tth token and the context feature, L is the total length of the context, is the one-hot vector of the context; The weights of the vocabulary distribution and the copy distribution based on the cross-attention score are learned by the following network:

[0012]

[0013] Dynamically fuse the two distributions to generate the final predicted distribution .

[0014] As a preferred solution, the replication distribution calculated based on the cross-attention score is generated by accumulating the cross-attention scores of the same token in the context, and the cross-attention scores come from the attention weights of the last layer of the decoder.

[0015] As a preferred solution, the pre-training model is a RoBERTa model.

[0016] A second aspect of the present invention provides a multi-hop reading comprehension system based on a paragraph pair loss and duplication mechanism, the system comprising a first splicing module, a filter module, a second splicing module, a reading reasoning module, and a word segmenter module; The first splicing module is used to splice the preset questions with several paragraphs to obtain spliced ​​paragraphs; The filter module is used to filter the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs; The second splicing module is used to splice the high-scoring paragraph with the preset question again to obtain a high-scoring spliced ​​paragraph; The reading reasoning module is used to generate a vocabulary distribution and a copy distribution based on the cross attention score for the high-scoring spliced ​​paragraph through a preset decoder, and dynamically fuse the two types of distributions to generate a predicted distribution probability; The word segmenter module is used to determine the maximum value in the predicted distribution probability, and determine the corresponding prediction token in the preset vocabulary according to the index corresponding to the maximum value, so as to obtain the predicted text.

[0017] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned multi-hop reading comprehension method based on paragraph pair loss and duplication mechanism.

[0018] The fourth aspect of the present invention provides a computer device, including a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor. When the computer program is executed by the processor, the steps of the aforementioned multi-hop reading comprehension method based on paragraph loss and copy mechanism are implemented.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention makes full use of the attention mechanism in single-step screening to make paragraphs perceive problems and other paragraphs, and proposes optimization goals at the paragraph level. It uses the interactive relationship between paragraphs to guide the paragraph screening process and improve the accuracy of screening evidence paragraphs.

[0020] The present invention applies a cross-attention-based replication mechanism to improve the reference degree of the output content of the generative model to the context, thereby improving the performance of generating answers and evidence. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flow chart of the multi-hop reading comprehension method based on the paragraph pair loss and duplication mechanism provided in this embodiment; Figure 2 A diagram showing the filter module framework provided in this embodiment; Figure 3 A diagram showing the framework of the reading reasoning module provided in this embodiment; Figure 4 A diagram of the decoding process during prediction provided by this embodiment; Figure 5 This is a diagram of the decoding process during training provided by this embodiment. DETAILED DESCRIPTION

[0022] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present invention; It should be clear that the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the embodiments of the present application.

[0023] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the embodiments of the present application. The singular forms "a," "the," and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0024] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0025] In addition, in the description of this application, unless otherwise specified, "plurality" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship. The present invention is further described below with reference to the accompanying drawings and examples.

[0026] The present invention is further described below with reference to the accompanying drawings and embodiments.

[0027] Example 1 Please refer to Figure 1 This embodiment provides a multi-hop reading comprehension method based on a paragraph pair loss and duplication mechanism, the method comprising: S1: Splice the preset questions with several paragraphs to obtain spliced ​​paragraphs; S2: screening the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs; In a specific embodiment, the method of screening the plurality of concatenated paragraphs to obtain high-scoring paragraphs includes: Extract problem features separately through pre-training model and paragraph features ; The paragraph features are updated using the cross attention operation to obtain ; Pool the updated paragraph features and generate the interaction features between paragraphs through the self-attention mechanism ; based on Calculating binary classification loss and paragraph pair loss , where the paragraph pair labels are obtained by adding the paragraph binary labels, and the loss function is the weighted sum of the two .

[0028] In a specific embodiment, the pre-trained model is a RoBERTa model.

[0029] In a specific embodiment, the label value of the paragraph pair loss is 0, 1 or 2, and the paragraph feature pairs are concatenated and input into a multi-layer perception mechanism for ternary classification prediction, and the paragraph pair loss is a cross entropy loss.

[0030] In a specific embodiment, the method of screening the plurality of concatenated paragraphs to obtain high-scoring paragraphs further includes: Perform softmax normalization on the binary classification output distribution to obtain the probability that each paragraph is a related paragraph; The three paragraphs with the highest scores are selected as output according to the probability ranking.

[0031] S3: splicing the high-scoring paragraph with the preset question again to obtain a high-scoring spliced ​​paragraph; S4: The high-scoring spliced ​​paragraph is passed through a preset decoder to generate a vocabulary distribution and a copy distribution based on the cross-attention score, and the two types of distributions are dynamically integrated to generate a predicted distribution probability; In a specific embodiment, the method of dynamically fusing the vocabulary distribution and the copy distribution based on the cross attention score generated by the high-scoring spliced ​​paragraph through a preset decoder to generate a predicted distribution probability includes: Concatenate the question and the selected high-scoring paragraphs into a sequence and input it into the encoder to obtain features; During the decoding process, the word list distribution is generated through the decoder output , and the replication distribution calculated based on the cross-attention score ;in Represents the word embedding vector of the t-th token, It is the output vector of the token after passing through multiple layers of decoder. is the weight of the linear layer, represents the cross attention score between the tth token and the context feature, L is the total length of the context, is the one-hot vector of the context; The weights of the vocabulary distribution and the copy distribution based on the cross-attention score are learned by the following network:

[0032]

[0033] Dynamically fuse the two distributions to generate the final predicted distribution .

[0034] In a specific embodiment, the replication distribution calculated based on the cross-attention score is generated by accumulating the cross-attention scores of the same token in the context, and the cross-attention scores come from the attention weights of the last layer of the decoder.

[0035] S5: Determine the maximum value in the predicted distribution probability, and determine the corresponding prediction token in the preset vocabulary according to the index corresponding to the maximum value, so as to obtain the predicted text.

[0036] As shown in Table 1, compared to existing techniques, the present invention improves the performance of single-stage paragraph screening. Compared to single-stage screening methods, the present invention simultaneously considers the connections between questions and paragraphs, and between paragraphs. Therefore, it can more completely and rationally identify the reasoning chains existing in questions and paragraphs, helping to improve the accuracy of paragraph screening. Compared to multi-stage methods designed for specific data, the present invention is easily scalable to scenarios with more relevant paragraphs.

[0037] Table 1 Paragraph screening results

[0038] As shown in Table 2, reading comprehension performance is improved in both answer and evidence prediction. All metrics are improved compared to the official baseline model, and as a generative approach, it significantly outperforms the graph neural network-based method DFGN. PathFiD (T5-base) replicates the results using the T5-base model. This example uses it as a baseline and achieves improvements in both answer and evidence performance. This is due to the replication mechanism, which increases the reference to contextual information during the decoding process.

[0039] Table 2 Answer and evidence prediction results of the proposed method on the HotpotQA dataset

[0040] Example 2 This embodiment provides a multi-hop reading comprehension system based on a paragraph pair loss and duplication mechanism, the system comprising a first splicing module, a filter module, a second splicing module, a reading inference module, and a word segmenter module; The first splicing module is used to splice the preset questions with several paragraphs to obtain spliced ​​paragraphs; The filter module is used to filter the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs; The second splicing module is used to splice the high-scoring paragraph with the preset question again to obtain a high-scoring spliced ​​paragraph; The reading reasoning module is used to generate a vocabulary distribution and a copy distribution based on the cross-attention score for the high-scoring spliced ​​paragraph through a preset decoder, and dynamically fuse the two types of distributions to generate a predicted distribution probability; The word segmenter module is used to determine the maximum value in the predicted distribution probability, and determine the corresponding prediction token in the preset vocabulary according to the index corresponding to the maximum value, so as to obtain the predicted text.

[0041] Specifically, please refer to Figure 2 , assuming that 10 candidate paragraphs are provided and there are 2 relevant paragraphs and , and its corresponding binary classification label is 1 for paragraph tags and 0 for other paragraph tags.

[0042] First, concatenate the question with the 10 paragraphs and input them into the pre-trained model BoBERTa to obtain the concatenated features. And through the cross attention mechanism, each paragraph can perceive the question information :

[0043] Next, we first analyze the paragraph features Perform pooling and dimensionality reduction operations to reduce its dimension from Transformed into , l is the maximum length of a paragraph in a batch, and then a self-attention operation is performed to enable interaction between paragraphs:

[0044] The paragraph features will pass through two classifiers. On the one hand, as shown in Figure 2 above, the paragraph features Directly output the predicted distribution through a multi-layer perception mechanism consisting of linear layers and activation layers , then the corresponding binary classification cross entropy loss can be directly calculated:

[0045] On the other hand, as shown in the bottom of Figure 2, paragraph features are concatenated in pairs and passed through another multi-layer perception mechanism to predict the paragraph pair association distribution z. It is obtained by adding the corresponding binary labels, that is, In the implementation, the binary labels are first added in pairs through the broadcast mechanism to obtain the label matrix as shown in Figure 3, and then the lower triangle and diagonal parts are set to -100. The ignore_index parameter of the cross entropy loss function will ignore the loss calculation process corresponding to the -100 position. Then the loss function actually only calculates the distribution z corresponding to the upper triangle part and its corresponding label. Therefore, the paragraph pair loss will be calculated by the following cross entropy function:

[0046] Finally, the above binary classification and ternary paragraph pair loss weighting and optimized paragraph filter are used. The following formula has the super parameter :

[0047] After training the paragraph filter, you can select the top-k high-scoring paragraphs as needed during testing or actual application. Here, we select the top 3 paragraphs.

[0048] Specifically, please refer to Figure 3 In order to distinguish different parts of the input and output sequences, we use the tags from the paper "Modeling Multi-hop Question Answering as Single Sequence Prediction" to separate the text. We add "question: " and "title: " before the question and title respectively, and add a special tag to each sentence in the paragraph to form a sentence number " <fn>", we get the input sequence shown in the upper part of Figure 3. The decoder receives the concatenated question paragraph pair features and outputs the predicted distribution. The target output sequence format is set in the format shown in the figure, where " <title-n> ”，" <fact-n>"and" <answer>" is a special marker to separate the title, sentence number and answer, making it easier to parse the evidence and answer from the output sequence.

[0049] The features of each question paragraph pair are obtained through the T5 encoder and concatenated into a sequence, that is, the feature shape is transformed from Transformed into ， The maximum length of a question and a single paragraph, The feature is input into the decoder to generate the answer and evidence sentence sequence.

[0050] The decoder has different decoding processes during training and testing, such as Figure 5 , during the training process, the target sequence is shifted right by one As input, the prediction is as follows Figure 4 Figure, the output of the previous step is input. represents the word embedding vector of the t-th input token of the decoder, is the output vector of the token after passing through multiple layers of decoders. The generated distribution can be directly obtained from this feature:

[0051] is the weight of the linear layer, which is used to map the d-dimensional features output by the last layer of the decoder to a dimension of In the vocabulary space, we get the distribution of predicted tokens .

[0052] The probability of "copying" depends on the correlation between the current token and the context. The T5 model itself considers the relationship between the decoder input and the context. At each layer of the decoder, cross-attention operations are performed on the input token and the output features of the last layer of the encoder. The corresponding cross-attention score can represent the correlation, so the attention score of the last layer is directly used. Compute the "replication" distribution:

[0053] in represents the cross attention score between the tth token and the context feature, is the onehot vector of the context. The above formula obtains the replication distribution by accumulating the cross attention scores of the same token in the context. .

[0054] The weights of the generated distribution The input vector from the decoder and the output vector After concatenation, it is learned through a linear layer and an activation layer, and the weights of the distribution are generated and the weights of the replica distribution The sum is 1:

[0055]

[0056] The final prediction token can be generated by the decoder of the generative model or "copied" from the context. The final prediction distribution probability is defined as:

[0057] During training, the above network generates a sequence of predicted distributions y and true labels Optimize the cross entropy between:

[0058] During testing, we obtain text by generating the predicted distribution y of the sequence:

[0059]

[0060] Example 3 This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the multi-hop reading comprehension method based on the paragraph pair loss and duplication mechanism described in Example 1 are implemented.

[0061] Example 4 This embodiment provides a computer device, including a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor. When the computer program is executed by the processor, the steps of the multi-hop reading comprehension method based on the paragraph pair loss and duplication mechanism described in Example 1 are implemented. Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.< / answer> < / title-n> < / fn>

Claims

1. A multi-hop reading comprehension method based on paragraph pair loss and replication mechanism, characterized by: The method comprises: Splice the preset questions with several paragraphs respectively to obtain spliced ​​paragraphs; Screening the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs; Splicing the high-scoring paragraphs with the preset questions again to obtain high-scoring spliced ​​paragraphs; The high-scoring spliced ​​paragraph is passed through a preset decoder to generate a vocabulary distribution and a copy distribution based on the cross-attention score, and the two types of distributions are dynamically integrated to generate a predicted distribution probability; The maximum value in the predicted distribution probability is determined, and the corresponding prediction token is determined in the preset vocabulary according to the index corresponding to the maximum value, so as to obtain the predicted text.

2. The multi-hop reading comprehension method based on paragraph pair loss and duplication mechanism according to claim 1, characterized in that: The method of screening the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs includes: Extract problem features separately through pre-training model and paragraph features ; The paragraph features are updated using the cross attention operation to obtain ; Pool the updated paragraph features and generate the interaction features between paragraphs through the self-attention mechanism ; based on Calculating binary classification loss and paragraph pair loss , where the paragraph pair labels are obtained by adding the paragraph binary labels, and the loss function is the weighted sum of the two .

3. The multi-hop reading comprehension method based on paragraph pair loss and duplication mechanism according to claim 2, characterized in that: The label value of the paragraph pair loss is 0, 1 or 2. The paragraph feature pairs are concatenated and input into a multi-layer perception mechanism for ternary classification prediction. The paragraph pair loss is a cross entropy loss.

4. The multi-hop reading comprehension method based on paragraph pair loss and duplication mechanism according to claim 1, characterized in that: The method of screening the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs further includes: Perform softmax normalization on the binary classification output distribution to obtain the probability that each paragraph is a related paragraph; The three paragraphs with the highest scores are selected as output according to the probability ranking.

5. The multi-hop reading comprehension method based on paragraph pair loss and duplication mechanism according to claim 1, characterized in that: The method of generating a vocabulary distribution and a copy distribution based on a cross-attention score by passing the high-scoring spliced ​​paragraph through a preset decoder, and dynamically fusing the two types of distributions to generate a predicted distribution probability includes: Concatenate the question and the selected high-scoring paragraphs into a sequence and input it into the encoder to obtain features; During the decoding process, the word list distribution is generated through the decoder output , and the replication distribution calculated based on the cross-attention score ;in Represents the word embedding vector of the t-th token, It is the output vector of the token after passing through multiple layers of decoder. is the weight of the linear layer, represents the cross attention score between the tth token and the context feature, L is the total length of the context, is the one-hot vector of the context; The weights of the vocabulary distribution and the copy distribution based on the cross-attention score are learned by the following network: Dynamically fuse the two distributions to generate the final predicted distribution .

6. The multi-hop reading comprehension method based on paragraph pair loss and duplication mechanism according to claim 5, characterized in that: The replicated distribution calculated based on the cross-attention score is generated by accumulating the cross-attention scores of the same token in the context, where the cross-attention scores come from the attention weights of the last layer of the decoder.

7. The multi-hop reading comprehension method based on paragraph pair loss and duplication mechanism according to claim 2, characterized in that: The pre-trained model is the RoBERTa model.

8. A multi-hop reading comprehension system based on paragraph pair loss and replication mechanism, characterized by: The system includes a first splicing module, a filter module, a second splicing module, a reading inference module, and a word segmenter module; The first splicing module is used to splice the preset questions with several paragraphs to obtain spliced ​​paragraphs; The filter module is used to filter the plurality of spliced ​​paragraphs to obtain high-scoring paragraphs; The second splicing module is used to splice the high-scoring paragraph with the preset question again to obtain a high-scoring spliced ​​paragraph; The reading reasoning module is used to generate a vocabulary distribution and a copy distribution based on the cross attention score for the high-scoring spliced ​​paragraph through a preset decoder, and dynamically fuse the two types of distributions to generate a predicted distribution probability; The word segmenter module is used to determine the maximum value in the predicted distribution probability, and determine the corresponding prediction token in the preset vocabulary according to the index corresponding to the maximum value, so as to obtain the predicted text.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-hop reading comprehension method based on paragraph pair loss and duplication mechanism are implemented as described in any one of claims 1 to 7.

10. A computer device, characterized in that: The method comprises a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein when the computer program is executed by the processor, the steps of the multi-hop reading comprehension method based on the paragraph pair loss and duplication mechanism as described in any one of claims 1 to 7 are implemented.

Images