Method, apparatus and device for generating an answer to a multi-hop question

Abstract

The application relates to the field of natural language processing, and discloses a multi-hop question answer generation method, device and equipment. The method comprises the following steps: receiving a user input original question, and decomposing the original question into a subtask list; for any subtask, inputting a query statement into an anti-interference generation model to perform a query operation to obtain a query answer; wherein the anti-interference generation model is obtained by training an initial generation model by using a training sample and a first reward function for optimizing the correct matching degree of a model output answer and a related text block in a first training stage, a second reward function for optimizing the coincidence degree of the model output answer and a keyword in a second training stage, and a third reward function for optimizing the correct attention focusing degree of the model to the related text block in a third training stage; and the query answers of any subtask are summarized to obtain a target answer of the original question. Thus, the accuracy and robustness of multi-hop question answering are significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and in particular to a multi-hop question answer generation method, device and equipment. BACKGROUND

[0002] With the rapid development of technology, large language models in related technologies begin to implicitly store knowledge through massive parameters, but this approach lacks transparency and is limited in capacity, and updating requires repeated training with high cost. At the same time, the model is difficult to handle long context information association, leading to attention dilution, increasing the risk of hallucinations and errors, and reducing accuracy.

[0003] To alleviate the above problems, Retrieval-Augmented Generation (RAG) technology has emerged, which allows the model to first retrieve external knowledge bases in real time when answering, and then integrates the retrieval results into the generation process, thereby significantly improving accuracy. However, in multi-hop question answering scenarios that require progressive progression, RAG technology still has limitations, and a new solution is needed that takes into account anti-noise accurate positioning and coherent reasoning planning. SUMMARY

[0004] The present application aims to at least partially solve one of the technical problems in the related art. To this end, the present application proposes a multi-hop question answer generation method, device and equipment. The main technical solution adopted by the present application includes:

[0005] In a first aspect, the present application provides a multi-hop question answer generation method, which includes receiving a user input original question and decomposing the original question into a subtask list; wherein the subtask list includes at least two subtasks arranged in a predetermined order; for any subtask, input the query statement contained in any subtask into an anti-interference generation model to perform a query operation, and obtain the query answer of any subtask; wherein the anti-interference generation model is obtained by: determining an initial generation model and a training sample; training the initial generation model using the training sample and a first reward function, a second reward function, and a third reward function to obtain the anti-interference generation model; wherein the training sample includes a relevant text block and a keyword sentence; the first reward function is used to optimize the correct matching degree of the model output answer and the relevant text block in the first training stage; the second reward function is used to optimize the coincidence degree of the model output answer and the keyword sentence in the second training stage; the third reward function is used to optimize the correct attention focusing degree of the model on the relevant text block in the third training stage; and based on the query answer of any subtask, the target answer of the original question is obtained by summarizing.

[0006] Optionally, the first reward function includes an answer correctness reward, a retrieval utility reward, and a noise penalty utility reward.

[0007] Optionally, the second reward function comprises a golden document attention reward, a key sentence attention reward, and a keyword attention reward.

[0008] Optionally, the third reward function comprises a document focus calibration reward and a sentence focus calibration reward.

[0009] Optionally, if two sequentially adjacent sub-tasks in the sub-task list are connected by a placeholder, the two sequentially adjacent sub-tasks are respectively denoted as a first sub-task and a second sub-task; the method further comprises: determining a query answer of the first sub-task; and constructing a query sentence of the second sub-task based on the query answer of the first sub-task, the placeholder, and an initial sentence of the second sub-task, so as to perform a query operation of the second sub-task by using the interference-resistant generation model.

[0010] Optionally, any sub-task further comprises a task type and a related text block; the task type comprises an inference task; and the query operation is performed by: in a case where the task type indicates that any sub-task is an inference task, inputting the query sentence of any sub-task into the interference-resistant generation model, and performing inference in the related text block according to a preset inference prompt word, so as to perform the query operation of the inference task.

[0011] Optionally, the task type of any sub-task comprises a retrieval task; and the query operation is performed by: in a case where the task type indicates that any sub-task is a retrieval task, performing retrieval in the vector database by using the query sentence of any sub-task to obtain an additional text block; inputting the query sentence of any sub-task into the interference-resistant generation model, and performing filtering and integration in the additional text block, so as to perform the query operation of the retrieval task.

[0012] Optionally, the performing retrieval in the vector database by using the query sentence of any sub-task to obtain an additional text block comprises: performing retrieval in the vector database by using the query sentence of any sub-task to obtain a basic text block, and dividing the basic text block into at least one sentence; performing similarity calculation on the query sentence and each sentence respectively to obtain similarity data of each sentence; determining a target sentence in each sentence based on the similarity data, and performing reorganization processing on the target sentence to obtain the additional text block.

[0013] In a second aspect, the embodiments of the present application provide a multi-hop question answer generation device, which comprises: a task processing module configured to receive an original question input by a user and decompose the original question into a subtask list; wherein the subtask list comprises at least two subtasks arranged in a preset order; a query execution module configured to, for any subtask, input a query statement contained in the subtask into an anti-interference generation model to perform a query operation, and obtain a query answer of the subtask; wherein the anti-interference generation model is obtained by: determining an initial generation model and a training sample; training the initial generation model by using the training sample and a first reward function, a second reward function and a third reward function, to obtain the anti-interference generation model; wherein the training sample comprises a relevant text block and a keyword sentence; the first reward function is used to optimize the correct matching degree of the model output answer in a first training stage and the relevant text block; the second reward function is used to optimize the coincidence degree of the model output answer in a second training stage and the keyword sentence; and the third reward function is used to optimize the correct attention focusing degree of the model on the relevant text block in a third training stage; and an answer generation module configured to aggregate the query answers of any subtasks to obtain a target answer of the original question.

[0014] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of any one of the above aspects when executing the computer program.

[0015] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the processor implements the steps of the method of any one of the above aspects when executing the computer program.

[0016] In a fifth aspect, the present application provides a computer program product comprising a computer program, and the processor implements the steps of the method of any one of the above aspects when executing the computer program.

[0017] In the above embodiments, the three-stage reward function is used to optimize the generation process in layers, which significantly improves the accuracy of key information extraction in a noisy environment. In addition, the problem decomposition and dynamic programming technology are combined to solve the logical fault problem, which significantly improves the accuracy and robustness of multi-hop question answering, and can be widely applied to complex multi-hop question answering scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the description of the embodiments or the prior art will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings without creative labor based on these drawings.

[0019] Figure 1 a flowchart of a method for generating an answer to a multi-hop question according to an embodiment of the present application;

[0020] Figure 2 a flowchart of a method for searching task query execution according to an embodiment of the present application;

[0021] Figure 3 a structural block diagram of an answer generation device for a multi-hop question according to an embodiment of the present application;

[0022] Figure 4 an internal structure diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings for the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0024] With the rapid development of technology, a large language model in the related art begins to implicitly store knowledge through a large number of parameters, but this way lacks transparency and is limited in capacity, and the update needs to be repeatedly trained at a high cost. At the same time, the model is difficult to process long context information association, leading to attention dilution, increasing the risk of hallucination and errors, and reducing accuracy.

[0025] To alleviate the above problems, retrieval augmented generation (RAG) emerges as the times require, which combines an external knowledge base to improve answer accuracy, but the traditional “retrieval-then-reasoning” paradigm has defects: retrieval easily introduces redundant documents, increasing the reasoning burden; multi-hop task information retrieval may exceed the context window limit, leading to information truncation or logical discontinuity.

[0026] The related RAG technology solutions for multi-hop question answering include: 1) query reformulation, decomposing complex questions into atomic sub-queries; 2) iterative retrieval generation, approximating the optimal solution through multiple rounds of interaction; 3) hybrid retrieval architecture, combining sparse and dense retrieval to improve recall effect. These technical solutions have achieved certain results in improving the basic retrieval capability.

[0027] However, the related technical solutions have technical bottlenecks: 1) query reformulation lacks modeling of sub-query semantic association, leading to logical discontinuity; 2) iterative retrieval produces noise accumulation, increasing the computational burden and causing information overload; 3) hybrid retrieval lacks semantic focus, introducing low-relevance documents to interfere with model reasoning. These defects limit the application effect of RAG in multi-hop question answering.

[0028] Based on this, according to the embodiment of the present application, a multi-hop question answer generation method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from here.

[0029] In this embodiment, a multi-hop question answer generation method is provided, Figure 1 is a flowchart of the multi-hop question answer generation method according to the embodiment of the present application, as Figure 1 shown, the flow includes the following steps:

[0030] S110, receiving a user input original question, and decomposing the original question into a subtask list.

[0031] Wherein, the original question can refer to the multi-hop query content input by the user through the system interaction interface, which needs to give a complete answer. Specifically, the original question can be understood as a problem with a certain complexity, which cannot be directly answered by a single step, and needs to rely on multi-step logical deduction or cross-information integration to get the final answer. Since the original question cannot be directly answered by a single step, it needs to be first decomposed into multiple sub-steps with logical association for step-by-step processing. These sub-steps need to meet the characteristics of being able to be executed independently by single-hop question answering. Each sub-step corresponds to a specific target that can be processed independently, so each subtask can be regarded as a subtask executable by single-hop question answering. Illustratively, a large language model can decompose and analyze the relationship between subtasks under the guidance of instruction prompts, and sort these subtasks to form an ordered set, which is the subtask list. Specifically, the subtask list includes at least two subtasks arranged in a predetermined order. Wherein, the predetermined order can refer to the logical relationship order that the result of the previous subtask supports the execution of the subsequent subtask, that is, the answer of the previous subtask can be used as the input basis of the subsequent subtask to avoid the subsequent task from being unable to execute due to the lack of pre-information.

[0032] S120, for any subtask, input the query statement contained in any subtask into the anti-interference generation model to perform a query operation, and obtain the query answer of any subtask.

[0033] It should be noted that each subtask is stored in the subtask list in the form of a triple like , wherein, represents the number of subtasks; represents the task type of the th subtask; represents the query statement of the th subtask; represents the a task label of the subtask.

[0034] The task type can be divided into an inference task that does not require external retrieval and directly generates an answer, and a retrieval that generates an answer after obtaining information from an external vector database. The query statement corresponds to the specific processing target of the subtask, that is, the single-hop sub-problem to be solved. The task label is a special identifier of the subtask, which is used for subsequent storage and calling of the subtask answer, and realizes the result transmission between subtasks.

[0035] Optionally, if two adjacent subtasks in the subtask list are connected by a placeholder, the two adjacent subtasks can be respectively denoted as a first subtask and a second subtask.

[0036] The first subtask can be a subtask with an execution sequence in front, and the second subtask can be a subtask with an execution sequence behind and dependent on the result of the first subtask. In order to realize the result storage and calling between the two subtasks, a mapping dictionary is pre-set and maintained by the system, which is used to store the one-to-one mapping relationship between the subtask label and the corresponding query answer. Exemplarily, the mapping dictionary stores the <placeholder-answer> key-value pair of each subtask, that is, the task label of each subtask and the query answer corresponding to the subtask are in a one-to-one mapping relationship, so that the answer of the previous subtask can be quickly called by replacing the placeholder, the repeated calling of the model is reduced, and the accuracy and efficiency of the result transmission between the subtasks are ensured. The placeholder can be a symbol used to reserve a filling position for the previous subtask in the query statement of any subtask.

[0037] Therefore, for any subtask, the query answer of any subtask is obtained by inputting the query statement contained in any subtask into the anti-interference generation model to perform a query operation. The query answer of the first subtask needs to be determined first.

[0038] Specifically, the query statement of the first subtask can be extracted and input into the anti-interference generation model. Then, the model will process the query statement of the subtask according to the demand of the subtask, to generate a query answer that meets the target of the subtask. After generating the query answer of the first subtask, the task label of the first subtask and the query answer can be bound to form a mapping relationship of : > in the mapping dictionary, wherein, is the task label of the first subtask; is the query answer of the first subtask.

[0039] Subsequently, a query sentence of the second subtask is constructed based on the query answer of the first subtask, the placeholder, and the initial sentence of the second subtask, so as to execute the query operation of the second subtask through the anti-interference generation model.

[0040] The initial sentence of the second subtask refers to an original query sentence without the first subtask answer, which contains a placeholder corresponding to the first subtask label. Because the specific information of the first subtask is missing, the model cannot directly execute the query. Specifically, the initial sentence of the second subtask can be extracted first, the placeholder contained in the sentence is identified, and then the first subtask query answer corresponding to the placeholder is retrieved from the mapping dictionary. Subsequently, the first subtask query answer is used to replace the placeholder in the initial sentence of the second subtask, so that the initial sentence containing the reserved position is converted into the second subtask query sentence containing complete information and can be directly executed.

[0041] Optionally, the number of subtasks contained in the subtask list is not limited to two, that is, the original problem can be decomposed into three or more ordered subtasks according to the complexity of the original problem. The mapping dictionary and the placeholder mechanism are also applicable to all adjacent subtask pairs that have the dependency relationship, so as to ensure the coherent transmission of the results among the multiple subtasks. Therefore, in the execution process of the subsequent subtask j, all placeholders in the initial sentence of the subtask j can be replaced by to generate the final executable version of the query sentence of the subsequent subtask j, and k represents any subtask in the sequence of the subtask j.

[0042] Subsequently, the constructed second subtask query sentence is input into the anti-interference generation model, so that the model can integrate the required information, analyze and process the query sentence, and generate the query answer of the second subtask.

[0043] Optionally, after obtaining the query answer of the second subtask, the task label of the second subtask can be bound with the query answer, and then stored in the mapping dictionary again, so as to provide answer support for the subsequent subtasks that may depend on the result. At this point, through the placeholder and the mapping dictionary, the answer generated by the previous subtask can be bound with the unique label and stored, so as to eliminate the information gap problem in the multi-hop reasoning. Moreover, the subsequent subtask only needs to replace the placeholder to immediately obtain the complete context, without the need to repeatedly perform the retrieval or generation operation, thereby greatly reducing the computational overhead.

[0044] It should be noted that the anti-interference generation model is an answer generation model with the ability to accurately filter irrelevant interference information and focus on the core content, which can stably generate an output meeting the task target in an input environment containing noise or redundant information.

[0045] Specifically, it is necessary to first determine the initial generation model and the training sample.

[0046] The initial generation model can refer to a pre-trained model with basic content generation capability. In theory, any pre-trained language model can be used as an initial model. For example, the initial generation model, i.e., the base model, can be initialized using Llama38B-Instruct. The training sample can refer to a data set used for model training. Specifically, the training sample includes relevant text blocks and key sentences. The relevant text block can refer to a text segment or document that is semantically associated with the question in the training sample. It includes a core text block that directly matches the question and contains an accurate answer, which can also be referred to as a gold document. It also includes irrelevant text blocks that are semantically associated with the question but do not contain accurate answers, which can also be referred to as interference documents. The key sentence can refer to the core information identifier extracted from the gold text block that connects the question and the answer. It can be a single keyword or a complete sentence, used to guide the model to focus on the key content in the gold text block. It is extracted from the gold text block by a large language model combined with a prompt word.

[0047] Specifically, taking Wikipedia as an example, the original document can be first divided into blocks. Each block is divided by 100 tokens, and long texts can be truncated at the period or comma. The document title is kept at the beginning of each block to record the topic information of the block. Then, all blocks are converted into 768-dimensional high-dimensional semantic vectors through the vectorization processing of the e5-base-v2 encoder, stored in the vector database and indexed to form a searchable text block resource library.

[0048] It should be noted that the essence of the anti-interference generation model is a noise-robust model that can accurately locate key information from noise documents. Since the key information required to solve complex multi-hop problems is often scattered in multiple locations of the document, retrieval and generation are difficult, the complex problem is first decomposed into relatively simple subtasks, and the problem handled by the default generation model is a general single-hop problem, that is, the problem and the relevant document paragraph are basically in a one-to-one correspondence. At this time, the training target of the model is to locate a single gold document and gold text block from a long document and to stably identify the key sentences and keywords in the gold text block.

[0049] Based on this, the Natural Questions and SQuAD two single-hop question answering data sets can be selected as the original data source of the training sample, 5000 samples are extracted from the two training sets respectively, and each sample has <question, answer, gold document> metadata, wherein the gold document is the initial source of the gold text block in the relevant text block. Subsequently, for each original sample, the question in the sample is converted into a high-dimensional semantic vector by using the embedding model vectorization method, that is, the e5-base-v2 embedding model, so as to obtain the query vector q. The semantic correlation degree between the query vector q and all text block vectors in the vector database is calculated by using the cosine similarity formula, so as to obtain the similar relevant document list. Exemplarily, according to the cosine similarity, the query vector q can be retrieved from the vector database by using the following formula:

[0050]

[0051] In the formula, q represents the query vector of the current original sample, is any vector in the vector database , that is, each text block after blocking; n represents the dimension of the vector and the vector ; and represents the cosine similarity between the query vector and each document segment vector in the vector database; the dot product is represented as the cumulative sum of components; and the norm of the vector and is also represented as the square root of the cumulative sum of the squares of the components.

[0052] Through the cosine similarity result of the above formula, the top k text block vectors with the highest correlation degree with the sample query vector q can be screened out, the corresponding index is extracted, and the TOP-K index set is formed, as follows:

[0053]

[0054] In the formula, q represents the query vector of the current original sample, is any vector in the vector database ; and represents the index corresponding to the k vectors most adjacent to the query vector q in the vector database in terms of semantics; is the set of all text block vectors in the vector database; and represents the k vectors with the highest similarity.

[0055] Subsequently, according to the TOP-K index, the index-matched text block is retrieved from the original document library corresponding to the vector database, to form the original text block set . denotes the text block corresponding to the ith index. And each text block in the set keeps the topic title recorded when the text is segmented. Then, based on the pre-stored topic identifiers, the text is aggregated, and the text blocks with the same title are de-duplicated and spliced to generate a set of topic blocks with coherent topics . The construction rule of a single topic block satisfies , denotes the mth unique title, so that the text blocks with the same title are spliced into a topic block, ensuring the integrity of the same topic information. Then, each topic block is cut into sentences with original titles to form a sentence set , and the sentences keep the original titles, so that the sentences belonging to the same document are connected with the same title. Then, the cosine similarity between each sentence and the query vector q is calculated to obtain a list of similarity scores . Then, the is sorted in reverse order from high to low, and the low semantic association sentences are deleted according to the preset proportion (50%), and the top 50% high association sentences are kept to obtain , wherein represents the ceiling. Finally, according to the filtered sentence list and its index, the original sentence set is mapped and sorted to obtain the noise filtered , in which each text block title is unique, and the text content is related to the query statement, which can be the required relevant text block for training samples.

[0056] After determining the relevant text block, the gold document can be determined using a content similarity measurement method. Exemplarily, a content similarity scoring function Similarity can be used, which integrates text overlap degree (Recall-Oriented Understudy for Gisting Evaluation, ROUGE-L) and embedding cosine similarity (semantic association degree) to quantify and evaluate the similarity between the relevant text block and the gold document in the original sample. Specifically, for The Similarity value of each relevant text block in the golden document is calculated, and if the Similarity value is greater than a preset threshold, it is judged that the relevant document block contains the core information of the golden document, and the golden document of the original sample is directly replaced as the final golden text block. If the Similarity value is less than the preset threshold, it is determined that the relevant document block does not contain the core information of the golden document, and it is retained as an interference text block. It should be noted that if the Similarity values of multiple relevant document blocks are greater than the preset threshold, only one golden document block is retained to ensure that the single-hop problem corresponds to a single training target of the golden text block. Finally, each sample forms a basic data set of <question-golden text block x 1-interference text block x n (n≤k)>, and the differentiation of the golden text block and the interference text block is completed.

[0057] Further, a large language model (LLM) such as deepseek-v3-0324 can be used to extract key sentences and keywords that connect the question and the answer from the golden text block guided by the preset prompt word model, which can be single or multiple, thereby supplementing the sample metadata to serve the training target of the model identifying key information. Then, the <question-golden text block-interference text block-key sentence-keyword> of the basic data set is integrated with the instruction prompt word and input into the large model to guide the model to generate a long answer with an explicit denoising process, that is, to output the reasoning process of how to locate the golden text block from the interference text block and extract the answer. The output can be used as a long answer corresponding to the short answer in the original sample.

[0058] It should be noted that when organizing the training data set, in order to avoid the model taking the shortcut of "learning to determine the golden document position based on document position information to improve the reward score" in the subsequent reinforcement learning process, the "golden document-interference document" can be randomly arranged, and only the serial number of the golden document is recorded, without fixing the position. At the same time, in theory, the LLM used in the above two processes can be any generative language model, but the larger or more advanced the language model, the higher the quality of the data set it generates, so deepseek-v3-0324 can be used as the LLM for data processing. Finally, the standardized format of <question-short answer-long answer-golden text block-interference text block-key sentence-keyword> is obtained, that is, the reinforcement learning training sample that meets the training needs of the anti-interference generation model is obtained.

[0059] After obtaining the training sample, the initial generation model can be trained using the training sample and the first reward function, the second reward function, and the third reward function to obtain the anti-interference generation model.

[0060] It should be noted that when training the anti-interference generation model, a method based on reinforcement learning and curriculum learning, combined with multi-scale entropy training noise robust generation model can be used. Specifically, first, the Group Relative Policy Optimization (GRPO) algorithm is used as the core algorithm of reinforcement learning. Exemplarily, the objective function is:

[0061]

[0062]

[0063] wherein, represents the importance sampling ratio, i.e., the new policy and the old policy selects the probability ratio of the action under the given state ; represents the advantage function of the i-th action or decision; represents the reward of the entire group; represents the KL divergence between the new policy and the reference policy , the coefficient is a hyperparameter for adjusting the regularization strength; N represents the number of samples in the group; j is the index of the reward sample in the group, and represents traversing all N reward samples in the group.

[0064] Then, on the basis of reinforcement learning, curriculum learning can be used to train the model gradually from easy to difficult. Specifically, it includes three courses: according to the correctness of the generated answer, the utility of the golden document, and the noise document shielding ability to design the reward function, to perform the curriculum learning of the basic question answering and reference ability; according to the focusing degree of the model internal attention on the golden document, the golden sentence and the golden keyword to design the reward function, to perform the curriculum learning of direct attention guidance; according to the attention entropy and attention allocation accuracy of the sentence level and keyword level in the golden document to design the reward function, to perform the multi-scale attention entropy optimization curriculum learning. And there are three kinds of key reward functions in the three courses.

[0065] Specifically, the first reward function is used to optimize the correct matching degree of the model output answer and the related text block in the first training stage. The first reward function includes answer correctness reward, retrieval utility reward and noise penalty utility reward.

[0066] Exemplarily, the first reward function of the basic question answering and reference is specifically:

[0067]

[0068] wherein, represents the first reward function; represents the answer correctness reward; represents the retrieval utility reward; represents the noise penalty utility reward.

[0069] The answer correctness reward includes:

[0070]

[0071] In the formula, represents the answer correctness reward; represents a weight coefficient, which can be 1.2; represents the use of F1 score (F1-Score / F1 Measure) to calculate the matching degree between the answer generated by the model and the long answer to calculate the reward score; refers to the answer text generated by the model; refers to the standard (golden) answer text.

[0072] Specifically, F1 score is used instead of exact match (EM) as the evaluation indicator of the generated answer, because there is no need to strictly match the standard answer at the beginning of training. The model only needs to learn to locate the relevant documents and generate answers based on the content, even if the final answer is not completely consistent with the standard short answer, as long as the reference document and reasoning logic are similar, it is considered valid.

[0073] The retrieval utility reward includes:

[0074]

[0075] In the formula, represents the retrieval utility reward; represents a weight coefficient, which can be 1.1; refers to the calculation of the reward score according to the reference of the golden document and its key content by the model generation result; represents a binary indicator variable of the golden document reference; refers to the reference content segment generated by the model; refers to the key sentence set of the golden document.

[0076] Specifically, when the model correctly references the serial number of the golden document in the generated answer, takes the value of 1, otherwise 0. refers to the specific content in the golden document that the model explicitly references in the answer, such as sentences, phrases, or paragraphs. refers to the list of core sentences (containing key information needed to answer the question) manually annotated from the golden document. Through The similarity of the reference content to the golden key sentence is calculated to determine whether the model references the correct material. The calculation formula is:

[0077]

[0078]

[0079] In the formula, is used to calculate the reference of the model output to the golden document and the average similarity score of each key sentence in the golden document ; is the i-th key sentence in the key sentence set of the golden document; m is the total number of key sentences in the golden document, and if m = 0, then = 0; Similarity is a content similarity score function that combines ROUGE-L (measures word order and co-occurrence) and embedded cosine similarity (measures semantics) to evaluate the similarity of two texts, and 0.4 is the weight of ROUGE-L; 0.6 is the weight of embedded cosine similarity; refers to the reference content to be evaluated, such as ; refers to the key sentence in the golden document, i.e. .

[0080] That is, is a two-dimensional scoring tool that not only judges whether the word order and co-occurring words of two sentences are similar, but also judges whether the meanings of the two sentences are similar. Finally, two-dimensional scores are given different weights and added according to the comprehensive proportion, and the higher the score, the more similar the content referenced by the model to the key sentence in the golden text block.

[0081] The noise penalty utility reward includes:

[0082]

[0083] In the formula, represents the noise penalty utility reward; represents the weight coefficient, which can be 0.4; is the number of all references to the interference document; is the interference score formula; is the reference of each interference; refers to the key sentence set of the golden document.

[0084] Specifically, the noise penalty utility reward is used to calculate the reward score according to the reference of the model to the interference text block and the interference or potential value of the reference content to determine whether the interference text block is biased. Among them The calculation formula of is as follows:

[0085]

[0086]

[0087] In the formula, A reference for each interference; It refers to the collection of key sentences in a golden document; The usefulness reward coefficient representing interference citations; This represents the average similarity between a single interfering citation and the key phrase. This represents the threshold for judgment; It refers to the i-th key sentence in the set of key sentences in the Golden Document; m is the total number of key sentences in the Golden Document.

[0088] Understandably, since there may be multiple references to non-golden documents (i.e., distracting documents), the score for each reference needs to be calculated separately. That is Each reference in the (scoring formula), that is, the specific content fragment in the non-golden document (distraction document) cited by the model in the answer, is denoted separately as . And when interfering with the quoted content With golden key sentences similarity Exceeding the threshold At the same time, a reward weight is assigned to encourage the model to identify potentially valid information in distracting documents. While citing noisy documents is highly likely to lead to incorrect answers, considering that the training objective is to enable the model to cite relevant documents for analysis as much as possible, and that noisy documents may contain unlabeled key information or parts overlapping with the content of the "golden document" (which is difficult to completely avoid during data creation), a simple penalty cannot be applied to all non-golden document citations. Instead, a relevance evaluation mechanism is introduced to determine the true correlation between the cited parts of non-golden documents and the "question-answer-key content of the golden document," thereby preserving the model's ability to identify potentially valid information while ensuring the main learning objective.

[0089] Therefore, for each reference First, calculate the average similarity between the quoted content and all key statements in the Golden Q&A. Calculation method and rewards In Similarly, a weighted average of ROUGE-L and cosine similarity is used (again, if m=0, then...). =0). When the similarity is calculated based on the reference. If it is greater than the threshold If the similarity is high, the citation is considered to contain useful supplementary information and is given a positive reward. The reward is proportional to the similarity, with a coefficient of 1. If the similarity is lower than or equal to the threshold value, the reference is considered to be noise, and a negative penalty is given, the penalty being proportional to the similarity (i.e. the lower the similarity, the greater the penalty), with a coefficient of . .

[0090] So far, the first reward function quantifies the semantic fit between the answer and the golden answer by F1 score, and measures the reference quality of the model to the golden key sentence by the weighted similarity of ROUGE-L and cosine similarity. At the same time, each reference of the interference document is dynamically rewarded or punished according to its similarity with the golden key sentence, so that the model can be guided to preferentially lock the key information and suppress noise interference at the initial stage of training, thereby significantly improving the accuracy and robustness of the answer.

[0091] The second reward function is used to optimize the overlap between the model output answer and the key words and sentences in the second training stage. The second reward function includes golden document attention reward, key sentence attention reward and key word attention reward.

[0092] Exemplarily, the second reward function directly guided by attention is specifically:

[0093]

[0094] In the formula, represents the second reward function; represents the golden document attention reward; represents the key sentence attention reward; represents the key word attention reward.

[0095] The golden document attention reward includes:

[0096]

[0097] In the formula, represents the golden document attention reward; represents the weight coefficient, which can be 0.3; represents the golden document; represents the average attention weight obtained by the golden document in the process of generating the entire answer.

[0098] The key sentence attention reward includes:

[0099]

[0100] In the formula, represents the key sentence attention reward; represents the weight coefficient, which can be 0.4; represents all key sentences in the golden document; represents the average attention weight obtained for all key words in all key sentences in the golden document in the process of generating the entire answer.

[0101] wherein the keyword attention reward comprises:

[0102]

[0103] wherein, represents the keyword attention reward; represents a weight coefficient, which can be 0.5; represents all key words in all key sentences in the golden document; represents the average attention weight obtained for all key words in a key sentence in the process of generating the entire answer.

[0104] Specifically, the same calculation component Attn_Unit_Avg can be explained as:

[0105]

[0106]

[0107] wherein, is the average attention weight obtained for the jth token in the input sequence in the process of generating the entire answer; is the number of tokens in the generated answer . represents the attention weight of the jth input token when the model generates the tth output token, which can be defined as only taking from the last layer of the TransformerBlock.

[0108] Specifically, the average attention weight of the output token sequence for each input token is obtained, and the average attention weight accumulated by the fixed unit , represents the index set of all tokens corresponding to the unit U in the input sequence. represents the number of tokens in the unit, and the sum of the attention weights obtained by all tokens in the unit U is calculated according to the given token mapping, and then divided by the number of tokens in the unit , to obtain the average density of attention of the tokens inside the golden unit U.

[0109] It should be noted that for the input token sequence, each token needs to be mapped to each word of the original input, and the same tokenizer as the generation model can be used to traverse the offset_mapping obtained after tokenization, and the mapping is obtained according to the index comparison. ​

[0110] So far, by monitoring the change of attention weight of the model on the pre-defined "golden standard" (document / sentence / keyword) in the input document, the quantitative evaluation is carried out by using the attention mechanism weight, which directly encourages the model to focus on the key area in the input when generating the answer, thereby significantly reducing the interference of irrelevant information, making the evidence positioning required by the model answer generation more accurate, and the output more reliable.

[0111] The third reward function is used to optimize the correct attention focus degree of the model on the relevant text block in the third training stage. The third reward function includes an intra-document focus calibration reward and an intra-sentence focus calibration reward.

[0112] Exemplarily, the third reward function of the multi-scale attention entropy optimization is specifically:

[0113]

[0114] In the formula, represents the third reward function; represents the intra-document focus calibration reward; represents the intra-sentence focus calibration reward.

[0115] The intra-document focus calibration reward includes:

[0116]

[0117] In the formula, represents the intra-document focus calibration reward; represents a weight coefficient, which can be 1.0; represents a core content matching degree; represents a document-level attention focus factor.

[0118] Specifically, the intra-document focus calibration reward is used to evaluate whether the attention is focused on the key sentence within the golden document. It is calculated by combining the attention proportion of the key sentence in the golden document and the entropy of the attention distribution between sentences, so as to encourage the attention to be concentrated on the key sentence with low entropy and ensure accurate allocation. Exemplarily, the specific calculation steps can be as follows:

[0119] (1) Calculate the attention value of each sentence in the golden document :

[0120] All sentences in the golden document can be divided into a sentence list according to the punctuation of the sentence itself, whether it is a golden sentence or a non-golden sentence. For any one sentence , calculate its total attention value:

[0121]

[0122] wherein, is the average attention weight obtained in the process of generating the whole answer for the jth token in the input sequence; is the sentence contains the index set of all input tokens.

[0123] (2) Calculate the attention proportion of the key sentences in the golden document, Ratio_GS_in_GD, i.e., the sentence calibration factor:

[0124]

[0125] wherein, represents the sentence calibration factor, and GD can be understood as Global Document; represents the sum of the attention values of all the key sentences in the golden document; represents the sum of the attention values of all the sentences in the golden document, i.e., the total attention in the golden document.

[0126] wherein, , The sentence calibration factor is the ratio of the two, and the value range is [0, 1]. The higher the value, the more accurate the attention to the golden sentence.

[0127] (3) Calculate the attention entropy of the sentences in the golden document :

[0128]

[0129] wherein, represents the attention entropy of the sentences in the golden document; represents the attention probability of the sentence .

[0130] Specifically, this entropy measures the concentration degree of the distribution of attention among the sentences in the golden document.

[0131] wherein, the attention probability of the sentence can be calculated by the following formula:

[0132]

[0133] Specifically, represents the attention weight of the sentence ; represents the sum of the attention values of all the sentences in the golden document, i.e., the total attention in the golden document.

[0134] That is, the total attention value of each sentence in the golden document obtained by step 1) is divided by the sum of the attention values of all sentences in the golden document obtained by step 2), and the attention probability of each sentence is normalized, that is, the attention entropy of the sentence level in the golden document is obtained.

[0135] (4) Calculate the document-level focus calibration reward .

[0136] First, the core content matching degree and the document-level attention focus factor need to be calculated. Among them, the core content matching degree can be calculated by the following formula:

[0137]

[0138] In the formula, represents the core content matching degree; represents the reward weight of the attention proportion of the golden sentence; represents the sentence calibration factor; represents the punishment weight of the attention proportion of the non-golden sentence, when the value is larger, a positive reward is generated, otherwise a negative punishment is generated.

[0139] Subsequently, the document-level attention focus factor can be calculated as follows:

[0140]

[0141] In the formula, represents the document-level attention focus factor, reflecting the concentration degree of the attention distribution inside the golden document, and the sentence entropy value is converted by the tanh function to realize nonlinear scaling, and a high value, such as near 1, indicates that the attention is highly concentrated, i.e. low entropy, and a low value, such as near 0, indicates that the attention is dispersed, i.e. high entropy.

[0142] As can be seen from the above, the training core of the document-level focus calibration reward is that when the model has correctly focused on a certain golden document, it can not only encourage the model to realize attention concentration inside the document, but also requires that this concentration must occur accurately on the golden sentence. If the model presents a highly concentrated attention distribution in the golden document but deviates from the golden sentence, a punishment mechanism is triggered.

[0143] Among them, the intra-sentence focus calibration reward includes:

[0144]

[0145] In the formula, represents the intra-sentence focus calibration reward; represents the weight coefficient, which can be 1.0; ​Total number of representative key sentences; Core matching degree representing the ith key word; Representative word-level attention focus factor.

[0146] Specifically, the key word focus within the key sentence and the calibration reward is used to evaluate whether the attention is focused on the key word within the key sentence. It combines the attention proportion of the key word within the key sentence and the entropy of the inter-word attention distribution to encourage the attention to be concentrated on the golden key word with low entropy and ensure accurate allocation. Exemplarily, since there are multiple key sentences within the golden document, the calculation steps of the partial reward score of a single sentence can be as follows:

[0147] (1) Calculate the attention value of each token within the key sentence This step has been implemented in the calculation of the second reward function R2, and the attention value corresponding to each word is .

[0148] (2) Calculate the attention proportion Ratio_GW of the key word within the key sentence, i.e., the word calibration factor:

[0149]

[0150] In the formula, represent the word calibration factor; represent the sum of the attention values of all relevant golden key words within the current key sentence; represent the sum of the attention values of all words / token within the key sentence, i.e., .

[0151] wherein, , And the word calibration factor is the ratio of the two, indicating the degree of attention of the model to the key word within the scale of the key sentence. The value range is [0, 1], and the higher the value, the more accurate the attention to the key word.

[0152] (3) Calculate the word-level attention entropy Words_ Entropy within the golden sentence:

[0153]

[0154] In the formula, represent the word-level attention entropy within the golden sentence; represent the attention probability of the word.

[0155] wherein, the attention probability of the word can be calculated by the following formula:

[0156]

[0157] where, represents the attention weight of the sentence . represents the sum of attention values of all words in the sentence.

[0158] That is, the lower the information entropy calculated in token unit at the sentence scale, the more focused the attention is on a few words; otherwise, the more scattered the attention is.

[0159] (4) Calculate .

[0160] Firstly, the reward score of each key sentence is determined by the core evaluation item of the single golden sentence and the attention focus factor of the golden sentence . As follows:

[0161]

[0162] where, represents the core matching degree of the ith key word; represents the reward weight of the attention proportion of the golden key word; represents the punishment weight of the attention proportion of non-golden words, such as redundant words; represents the word calibration factor.

[0163] Subsequently, the word-level attention focus factor is calculated, which can be shown as follows:

[0164]

[0165] where, represents the word-level attention focus factor; the entropy value is converted through the tanh function to a factor more suitable for reward calculation, with a value range close to [0, 1], low entropy corresponds to close to 1, and high entropy corresponds to close to 0.

[0166] In summary, according to the above calculation steps, the reward score of each key sentence is calculated, summed and averaged, represents the total number of key sentences, and then multiplied by the weight coefficient , so as to obtain the intra-sentence focus calibration reward The training core of the intra-sentence focus calibration reward is to further guide the model to achieve word-level low-entropy attention on the basis of the model having correctly focused on the golden sentence: on the one hand, the model is encouraged to focus attention on a small number of key words in the sentence, and on the other hand, it is required that such focus must be accurately directed to the true golden words - if the model achieves low-entropy attention but focuses on redundant words, it is punished for the sentence, thereby ensuring that the attention distribution is strictly calibrated with the importance of the key words. Thus, through the joint reward and punishment mechanism of the attention proportion and entropy value at the document-sentence-key word three-level granularity, the model can not only macroscopically lock the golden document but also microscopically focus on the key information in multi-hop question answering, thereby significantly improving the accuracy and robustness of answer positioning.

[0167] Optionally, the specific curriculum learning online evaluation and training process of the anti-interference generation model first needs to let the model on the test set reach 0.4 or more, reach 0.5 or more; the total reward function in this process is to enter the second stage from the first stage. Subsequently, under the premise that the previous monitoring indicators are stable or improved, the model on the test set are all above 1.5; the total reward function in this process is to enter the third stage from the second stage. Finally, under the premise that the previous monitoring indicators are stable or improved, the model on the test set has an attention proportion of > 50% for the key sentence in the golden document and a sentence-level attention entropy of < 1.0 in the golden document, and an attention proportion of > 30% for the key word in the key sentence and a word-level attention entropy of < 0.5 in the golden sentence; the total reward function in this process is to complete the training from the third stage.

[0168] S130, the query answers of any sub-tasks are summarized to obtain the target answer of the original question.

[0169] The target answer is an effective long answer result obtained by decomposing the original complex question into sub-tasks, solving the sub-task answers one by one, and then integrating through a reasonable summarization strategy. It can accurately and comprehensively respond to the original question to meet the user's information needs for the original question. Specifically, since all the query answers of the sub-tasks are stored in the mapping dictionary, they can be finally summarized for final comprehensive reasoning and answer output, thereby allocating resources while minimizing redundant retrieval and noise accumulation. For example, if the sub-task answers are independent segments without conflict supplement relationship, they can be directly spliced into complete text in logical order to form the target answer. If the sub-task answers intersect, they can be summarized and integrated. If there is a logical relationship between the sub-task answers, a simple logical deduction or weight calculation can be performed to obtain the target answer.

[0170] In the above embodiments, by using a three-stage reward function, the generation process is optimized in a hierarchical manner, significantly improving the accuracy of key information extraction in a noisy environment. In addition, by combining problem decomposition and dynamic programming techniques to solve the logical fault problem, the accuracy and robustness of multi-hop question answering are significantly improved, and it can be widely applied to complex multi-hop question answering scenarios.

[0171] In some embodiments, any subtask further includes a task type and a related text block. The task type includes an inference task. The query operation is performed in the following manner:

[0172] In the case where the task type indicates that any subtask is an inference task, the query statement of any subtask is input into the anti-interference generation model, and inference is performed in the related text block according to the preset inference prompt word to perform the query operation of the inference task.

[0173] Wherein, the inference task can refer to a task type of a subtask, which means that information in the related text block, including the gold document and the interference document, etc., is needed to derive implicit conclusions or solve problems through logical deduction, correlation analysis, etc. The preset inference prompt word can refer to a pre-defined instruction text for guiding the anti-interference generation model to perform the inference task. Specifically, its role is to clearly indicate the inference direction and rules to the model, so that the model can combine the related text block and perform inference according to the specified logic when processing the inference task, ensuring that the model output meets the task requirements, which can be understood as a kind of inference guide template.

[0174] It should be noted that since the anti-interference generation model is essentially a noise-robust model that accurately locates key information from documents, and by default, when processing single-hop problems, it needs to locate the gold document and identify key sentences and keywords, and the inference task is a subtask type, its related text block is adapted to the sample structure during model training. Therefore, after confirming that the task type of the subtask is an inference task, the query statement of the subtask and the related text block associated with the subtask can be extracted to provide materials for inference. Then, the query statement is input into the anti-interference generation model, and the preset inference prompt word is added to guide the model to focus on key information in the related text block and perform inference according to the specified logic. Then, the anti-interference generation model can use the trained capabilities to filter effective content in the related text block, combine the logical guidance of the preset inference prompt word, and perform inference operations to finally output the inference result, completing the query operation of the inference task.

[0175] In the above embodiments, by inputting the query statement of the inference task and the associated related text block into the anti-interference generation model, and supplementing it with the preset inference prompt word for logical guidance, the key information in the text environment containing interference information can be accurately located, the multi-step logical deduction can be completed, and the reliable inference result can be output.

[0176] In some embodiments, the task type of any subtask includes a retrieval task. Please refer to the accompanying drawings Figure 2 The query operation is performed in the following manner:

[0177] S210, in the case where the task type of any subtask is a retrieval task, the query statement of any subtask is used to retrieve in the vector database to obtain an additional text block.

[0178] The retrieval task is also one of the task types of the subtasks, and is different from the reasoning task. The retrieval task refers to a task that needs to retrieve a text block related to a question from the vector database based on a subtask query statement and obtain valid information after processing. The vector database refers to a database that stores high-dimensional semantic vectors of text blocks. These vectors are stored after document blocking and vectorization processing. The function of the vector database is to provide semantic matching text block resources for the retrieval task, so that the model can quickly locate the associated content from a large number of documents. The additional text block refers to a text block obtained by real-time retrieval and processing from the vector database in the retrieval task because the relevant text is not included in the task data set in advance.

[0179] It should be noted that the essence of the additional text block and the relevant text block is a text segment associated with the task, which contains potential key information and interference content. However, the relevant text block is preprocessed in the training stage, and the additional text block is newly obtained text content from the vector database in the retrieval task using the same blocking, semantic retrieval, and noise integration methods as training.

[0180] Specifically, in the case where the task type of any subtask is a retrieval task, the query statement of any subtask is used to retrieve in the vector database to obtain an additional text block, including: using the query statement of any subtask to retrieve in the vector database to obtain a basic text block, and dividing the basic text block into at least one sentence; performing similarity calculation on the query statement and each sentence respectively to obtain similarity data of each sentence; determining a target sentence in each sentence based on the similarity data, and performing reorganization processing on the target sentence to obtain the additional text block.

[0181] Exemplarily, as when constructing the training sample, first, the query sentence of the subtask is used to perform semantic retrieval based on cosine similarity algorithm or the like in the vector database, and top-k basic text blocks associated with the query semantics are matched. Then, the basic text blocks are cut into at least one sentence according to the sentence, and the cosine similarity or Similarity function is used again to calculate the semantic similarity between the query sentence and each cut sentence, to obtain the similarity data of each sentence to the query, so as to filter out the most relevant sentences to the query. Finally, based on the similarity data, the target sentences with high correlation are filtered out, and the top 50% of the high-similarity sentences can be retained, and these sentences are reorganized to finally obtain the additional text blocks. At this point, the query sentence is used for semantic retrieval in the vector database, and the high-similarity sentences are cut and reorganized to obtain the additional text blocks with high correlation to the query semantics and refined structure.

[0182] S220, input the query sentence of any subtask into the anti-interference generation model, and filter and integrate in the additional text block to perform the query operation of the retrieval task.

[0183] Specifically, after obtaining the additional text block, the query sentence of the subtask can also be input into the anti-interference generation model, and the additional text block is input at the same time, so that the model can accurately locate the gold document block in the additional text block, identify the key words and sentences, filter and integrate to filter out the core information and exclude the interference content, and finally output the effective result of the retrieval task, to complete the query operation of the retrieval task.

[0184] In the above embodiment, by performing semantic retrieval in the vector database with the query sentence, cutting and reorganizing the high-similarity sentences to generate the additional text block, and inputting the anti-interference generation model for filtering and integration, the core information after accurate denoising is obtained, and the query operation of the retrieval task is completed.

[0185] It should be understood that although each step in the above flowchart is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps of the above flowchart can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0186] The embodiment of the present specification also provides an answer generation device 300 of a multi-hop question, like Figure 3As shown, it comprises: a task processing module 310, a query execution module 320 and an answer generation module 330, wherein:

[0187] The task processing module 310 is configured to receive a user inputted original question and decompose the original question into a subtask list, wherein the subtask list comprises at least two subtasks arranged in a preset order.

[0188] The query execution module 320 is configured to input a query statement contained in any subtask into an anti-interference generation model to perform a query operation and obtain a query answer of the any subtask, wherein the anti-interference generation model is obtained by: determining an initial generation model and a training sample; training the initial generation model by using the training sample and a first reward function, a second reward function and a third reward function to obtain the anti-interference generation model; wherein the training sample comprises a relevant text block and a keyword sentence; the first reward function is used to optimize the correct matching degree of the model output answer and the relevant text block in a first training stage; the second reward function is used to optimize the coincidence degree of the model output answer and the keyword sentence in a second training stage; and the third reward function is used to optimize the correct attention focusing degree of the model to the relevant text block in a third training stage.

[0189] The answer generation module 330 is configured to summarize the query answers of any subtasks to obtain a target answer of the original question.

[0190] In some embodiments, the answer generation device 300 of the multi-hop question further comprises a model training module configured to determine that the first reward function comprises an answer correctness reward, a retrieval utility reward and a noise penalty utility reward.

[0191] In some embodiments, the model training module is further configured to determine that the second reward function comprises a golden document attention reward, a key sentence attention reward and a keyword attention reward.

[0192] In some embodiments, the model training module is further configured to determine that the third reward function comprises a document focusing calibration reward and a sentence focusing calibration reward.

[0193] In some embodiments, if two sequentially adjacent subtasks in the subtask list are connected by a placeholder, the two sequentially adjacent subtasks are respectively denoted as a first subtask and a second subtask; the query execution module 320 is further configured to determine the query answer of the first subtask; and construct a query statement of the second subtask based on the query answer of the first subtask, the placeholder and an initial statement of the second subtask to perform a query operation of the second subtask by using the anti-interference generation model.

[0194] In some embodiments, the task type of any subtask further comprises an inference task; and the query execution module 320 is further configured to, in a case where the task type indicates that any subtask is an inference task, input the query statement of any subtask into the anti-interference generation model, and perform inference in the relevant text block according to a preset inference prompt word, so as to perform the query operation of the inference task.

[0195] In some embodiments, the task type of any subtask comprises a retrieval task; and the query execution module 320 is further configured to, in a case where the task type indicates that any subtask is a retrieval task, perform retrieval in the vector database by using the query statement of any subtask, to obtain an additional text block; input the query statement of any subtask into the anti-interference generation model, and perform filtering and integration in the additional text block, so as to perform the query operation of the retrieval task.

[0196] In some embodiments, the query execution module 320 is further configured to perform retrieval in the vector database by using the query statement of any subtask, to obtain a basic text block, and split the basic text block into at least one sentence; perform similarity calculation on the query statement and each sentence respectively, to obtain similarity data of each sentence; determine a target sentence in each sentence based on the similarity data, and perform reorganization processing on the target sentence, to obtain an additional text block.

[0197] The specific limitations of the answer generation device for a multi-hop question can be seen from the limitations of the answer generation method for a multi-hop question in the foregoing, which will not be repeated here. Each module in the answer generation device for a multi-hop question can be realized by software, hardware, and a combination thereof, in whole or in part. Each module can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0198] The answer generation device for a multi-hop question in the embodiment is presented in the form of a functional unit. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0199] The computer device provided in the embodiment of the present application can be a terminal, and its internal structure diagram can be as shown in Figure 4As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used for wired or wireless communication with external terminals. Wireless mode can be achieved through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a multi-hop problem answer generation method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the computer device shell, or an external keyboard, touchpad or mouse, etc.

[0200] Those skilled in the art can understand that, Figure 4 The skilled in the art can understand that, The skilled in the art can understand that,

[0201] The embodiments of the present application also provide a computer readable storage medium. The method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium through network, so that the method described herein can be processed by such software stored on a storage medium using a general purpose computer, a special purpose processor or programmable or special purpose hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk or a solid state disk, etc. Further, the storage medium can also include a combination of the above types of memories. It can be understood that the computer, processor, microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, processor or hardware, the method shown in the above embodiments is implemented.

[0202] The embodiments of the present application provide a computer program product, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of any embodiment of the present application.

[0203] The method, device and equipment for generating an answer to a multi-hop question illustrated by the above embodiments can be implemented by a computer chip or entity, or by a product with certain functions. A typical implementation equipment is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation equipment, an email equipment, a game console, a tablet computer, a wearable equipment, or a combination of any of these devices.

[0204] For the convenience of description, the above device is described as various units in terms of functions for separate description. Of course, the functions of the units can be implemented in one or more software and / or hardware in the implementation of the present application. Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0205] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to produce a machine, so that the instructions executed by the computer or other programmable data processing equipment produce an article of manufacture including instructions to implement the flowcharts and / or block diagrams. Figure 1 The flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in the flow or multiple flows and / or blocks.

[0206] These computer program instructions can also be stored in a computer-readable storage medium that can guide the computer or other programmable data processing equipment to work in a specific way, so that the instructions stored in the computer-readable storage medium produce a manufactured product including instruction devices that implement the flowcharts and / or block diagrams. Figure 1 The flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in the flow or multiple flows and / or blocks. These computer program instructions can also be loaded into a computer or other programmable data processing equipment, so that a series of operation steps are performed on the computer or other programmable equipment to produce a computer-implemented process, so that the instructions executed on the computer or other programmable equipment provide a method for implementing the flowcharts and / or block diagrams.Figure 1 one or more processes and / or blocks Figure 1 the steps of a function specified in one or more processes and / or blocks.

[0207] In the description of the specification, reference to "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that a particular feature, structure, material, or characteristic being described is included in at least one embodiment or example of the application. The appearances of the above expressions in various places in the specification are not necessarily referring to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0208] In addition, the terms "first", "second", etc. are used herein only to describe various steps in a method, process, and / or block, and are not meant to be limiting or indicative about the importance or a quantity of the features being described. Thus, features defined with "first", "second" can include at least one of the features explicitly or implicitly. In the description of the specification, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically limited. It is also necessary to note that the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0209] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. Since it is basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the part of the method embodiment. The above is only for the embodiments of the application and is not intended to limit the application. The application can have various changes and modifications for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the application shall be included in the scope of the claims of the application. Although the embodiments of the application are described in conjunction with the drawings, those skilled in the art can make various modifications and changes without departing from the spirit and scope of the application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method for generating answers to multi-hop problems, characterized in that, The method comprises: receiving a user inputted original question and decomposing the original question into a subtask list; wherein the subtask list comprises at least two subtasks arranged in a preset order; for any subtask, inputting a query statement contained in the any subtask into an anti-interference generation model to perform a query operation, to obtain a query answer of the any subtask; wherein the anti-interference generation model is obtained by: determining an initial generation model and a training sample; wherein the initial generation model refers to a pre-trained model with basic content generation capability; training the initial generation model by using the training sample and a first reward function, a second reward function and a third reward function, to obtain the anti-interference generation model; wherein the training sample comprises a relevant text block and a keyword sentence; the relevant text block refers to a text segment or a document semantically associated with a question in the training sample, including a golden document and an interference document; the golden document refers to a core text block directly matching the question and containing an accurate answer; the interference document refers to an irrelevant text block semantically associated with the question but not containing the accurate answer; the keyword sentence refers to a single keyword or a complete sentence connecting the question and the answer extracted from the golden document; the first reward function is used to optimize the correct matching degree of the model output answer in the first training stage and the relevant text block; the second reward function is used to optimize the coincidence degree of the model output answer in the second training stage and the keyword sentence; the third reward function is used to optimize the correct attention focusing degree of the model on the relevant text block in the third training stage; the task type of the any subtask comprises an inference task; the query operation is performed by: in the case that the task type indicates that the any subtask is an inference task, inputting the query statement of the any subtask into the anti-interference generation model, performing inference in the relevant text block according to a preset inference prompt word, to perform the query operation of the inference task; and based on the query answer of the any subtask, obtaining a target answer of the original question.

2. The method of claim 1, wherein, The first reward function comprises an answer correctness reward, a retrieval utility reward and a noise penalty utility reward.

3. The method of claim 1, wherein, The second reward function comprises a golden document attention reward, a keyword sentence attention reward and a keyword attention reward.

4. The method of claim 1, wherein, The third reward function comprises a document focusing calibration reward and a sentence focusing calibration reward.

5. The method according to any one of claims 1 to 4, characterized in that, If two sequentially adjacent subtasks in the subtask list are connected by a placeholder, the two sequentially adjacent subtasks are respectively denoted as a first subtask and a second subtask; the method further comprises: determining a query answer of the first subtask; based on the query answer of the first subtask, the placeholder and an initial statement of the second subtask, constructing a query statement of the second subtask, to perform a query operation of the second subtask by the anti-interference generation model.

6. The method of claim 1, wherein, the task type of the any subtask comprises a retrieval task; the query operation is performed by: In a case where the task type indicates that any of the sub-tasks is a retrieval task, performing retrieval in a vector database by using a query statement of the any of the sub-tasks to obtain an additional text block; inputting the query statement of the any of the sub-tasks into an anti-interference generation model, and performing filtering and integration in the additional text block to execute a query operation of the retrieval task.

7. The method of claim 6, wherein, The performing retrieval in the vector database by using the query statement of the any of the sub-tasks to obtain the additional text block comprises: performing retrieval in the vector database by using the query statement of the any of the sub-tasks to obtain a basic text block, and dividing the basic text block into at least one sentence; performing similarity calculation on the query statement and each of the sentences to obtain similarity data of each of the sentences; determining a target sentence from the sentences based on the similarity data, and performing reorganization processing on the target sentence to obtain the additional text block.

8. An answer generation apparatus for a multi-hop question, characterized by, The apparatus comprises: a task processing module configured to receive an original question input by a user and decompose the original question into a sub-task list, the sub-task list comprising at least two sub-tasks arranged in a preset order; a query execution module configured to, for any of the sub-tasks, input a query statement contained in the any of the sub-tasks into an anti-interference generation model to execute a query operation and obtain a query answer of the any of the sub-tasks, wherein the anti-interference generation model is obtained by: determining an initial generation model and a training sample, wherein the initial generation model is a pre-trained model having a basic content generation capability; training the initial generation model by using the training sample and a first reward function, a second reward function and a third reward function to obtain the anti-interference generation model, wherein the training sample comprises a relevant text block and a keyword sentence, the relevant text block is a text segment or a document semantically associated with a question in the training sample, and comprises a golden document and an interference document, the golden document is a core text block directly matched with the question and containing an accurate answer, the interference document is an irrelevant text block semantically associated with the question but not containing the accurate answer, the keyword sentence is a single keyword or a complete sentence connecting the question and the answer extracted from the golden document, the first reward function is used to optimize a correct matching degree of a model output answer in a first training stage and the relevant text block, the second reward function is used to optimize an overlapping degree of the model output answer in a second training stage and the keyword sentence, and the third reward function is used to optimize a correct attention focusing degree of the model on the relevant text block in a third training stage, the task type of the any of the sub-tasks comprises an inference task, and the query operation is executed by: in a case where the task type indicates that the any of the sub-tasks is an inference task, inputting the query statement of the any of the sub-tasks into the anti-interference generation model, performing inference in the relevant text block according to a preset inference prompt word, and executing a query operation of the inference task. An answer generation module is configured to generate a target answer of the original question based on the query answers of the any subtask.

9. A computer device, comprising: The method comprises the following steps: A memory and a processor are in communication connection with each other, and the memory stores computer instructions. The processor executes the computer instructions to perform the method in any one of claims 1 to 7.

Images