A comprehensive processing framework for open-domain question answering tasks

Abstract

The application relates to the technical field of natural language processing and artificial intelligence, in particular to a comprehensive processing framework for an open domain question answering task, which comprises an expansion generation module, an expansion evaluation module, a document reordering module, a reordering evaluation module, an answer generation module and an answer verification module. On the basis of supporting multi-field and multi-task question answering, the framework adopts a model fine-tuning and dynamic prompt optimization strategy, improves the adaptability and performance of the system, proposes an expansion generation answer module, ensures that reasonable answers can be generated when documents are insufficient, guarantees the continuity of the question answering process, simultaneously, a verification and feedback mechanism is added, logical review is conducted on the answers before output, and error occurrence is reduced. The system adopts a deep fusion architecture of retrieval, generation and verification, realizes efficient question answering capability in multiple tasks and multiple fields, and provides reliable technical support for intelligent customer service, medical consultation, legal consultation and the like.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and artificial intelligence, specifically to a comprehensive processing framework for open-domain question answering tasks. Background Technology

[0002] With the rapid growth of internet information, users' demand for intelligent question-answering systems is constantly increasing. Traditional question-answering systems mostly rely on predefined structured knowledge bases or FAQs, making it difficult to handle complex and diverse open-domain questions, resulting in insufficient flexibility, limited coverage, and an inability to meet user needs. Therefore, developing open-domain question-answering systems capable of processing unstructured information and dynamically generating answers has become a research focus. An open-domain question-answering system is an intelligent system designed to extract answers from a wide range of information sources and provide natural language responses.

[0003] Open-domain question answering systems need to extract information from large-scale unstructured data, such as web pages, documents, and news. However, the dispersion and heterogeneity of this data increase the complexity of retrieval and generation. Modern question answering systems mostly adopt a combined retrieval and generation architecture: a retrieval module filters candidate documents, a generation module generates answers, and a validation module ensures the accuracy of the answers. However, existing systems are prone to interruption when there are insufficient documents or retrieval failures, affecting the user experience. The emergence of large-scale language models (LLMs) such as GPT and BERT has brought significant improvements to question answering systems. These models, after pre-training, possess powerful language understanding and generation capabilities, but they also suffer from high resource consumption and a lack of domain specialization.

[0004] Traditional question-answering systems primarily rely on pre-set structured knowledge bases or FAQs, which struggle to handle complex and diverse open-domain questions. This results in limitations in system flexibility and information coverage, making it difficult to fully meet diverse user needs. This invention designs a framework for open-domain question answering, extending the generation and verification of open-domain questions. It achieves greater flexibility and accuracy in evidence collection and answer generation, providing reliable technical support for scenarios such as intelligent customer service, medical consultation, and legal advisory services. Summary of the Invention

[0005] To address the aforementioned shortcomings and problems, this invention provides a comprehensive processing framework for open-domain question-answering tasks. Supporting multi-domain and multi-task question answering, it employs model fine-tuning and dynamic prompt word optimization strategies to improve system adaptability and performance. An extended answer generation module is proposed to ensure reasonable answers are generated even when documents are insufficient, guaranteeing the continuity of the question-answering process. Simultaneously, a verification and feedback mechanism is added to logically review answers before output, reducing errors. This system adopts a deeply integrated architecture of retrieval, generation, and verification, achieving efficient question-answering capabilities across multiple tasks and domains, providing reliable technical support for scenarios such as intelligent customer service, medical consultation, and legal advisory services.

[0006] The solution adopted by this invention to solve its technical problem is: a comprehensive processing framework for open-domain question answering tasks, which includes an extended generation module, an extended evaluation module, a document reordering module, a reordering evaluation module, an answer generation module, and an answer verification module; It also includes the following steps: Step 1, Generate Extensions: Starting from the user-input question, use a large language model (LLM) to generate additional background information. The pre-trained large language model receives the question and its prompts, and outputs extensions. Step 2, Expansion Evaluation: After generating the expansions, each expansion is scored using an LLM evaluator, with a score range of 0 to 1, to select the optimal expansion for subsequent processing. In this way, low-quality expansions are filtered out, ensuring that the data processed in subsequent steps has high quality and relevance. Step 3, document reordering: Based on the initially retrieved document set, the evaluated expansions are combined with the original question, and candidate documents are re-retrieved and reordered. The candidate document set is processed by an LLM sorter to output the optimal document set for answer generation. Step 4, Re-ranking Evaluation: After the documents are re-ranked, a further quality evaluation is performed on the re-ranked document set to ensure the high relevance and quality of the document content. The LLM evaluator is used to score the documents, and high-quality documents are further selected for answer generation based on the scoring results. Step 5, Answer Generation: After document reordering and evaluation, the final answer is generated using an LLM reader based on the question and the filtered document set; Step 6, Answer Verification: After generating the initial answer, a verification mechanism is introduced. The LLM validator checks the logical consistency and accuracy of the answer. If the verification fails, the system will regenerate the answer.

[0007] Furthermore, in step one, the extended generation module performs the following operations: S11. Input problem: Define the core problem that the system needs to answer in order to guide the extended generation module to generate relevant background information and standardize the problem to adapt it to the input format of a large language model; S12. Prompt Generation: Generate a specific prompt based on the input question. e Used to guide large language models in generating background information that meets target requirements; S13, Model Invocation: Combining question q and prompt word q e As input, the large language model G is invoked. e In the extended generation step, the large language model generates extended information based on the input. The model not only handles the initial question but also generates diverse background information by combining instructions given by prompts. e = G e (q; prompt e ); S14. Extended Generation: Generates background information related to the question to support subsequent document retrieval and answer generation steps. The output of this step is extended content e, which is in the form of natural language text paragraphs. score ev = E e (q; e; prompt ev ).

[0008] Furthermore, in the extended generation module, the genetic algorithm generates new Prompt combinations in each generation. These combinations are adjusted according to different settings, or even knowledge in a specific domain, and the generated new Prompts are scored. When the Prompt generated by the genetic algorithm surpasses the current template, it is set as the new benchmark solution.

[0009] Furthermore, in step three, the document reordering module reorders the documents according to the following steps: S31. First, using established information retrieval techniques, retrieve the top n documents relevant to the given question. These retrieval methods provide an initial score for each candidate document to describe the relevance between the document and the question. Then, leveraging the ranking capabilities of a large model, manually design and provide the corresponding prompt. d Tip: Use it to reorder documents; S32. Next, the documents are reordered. This stage involves comparing documents to determine which documents are more relevant and of higher quality to the query, guiding the LLM to act as a document reorderer. d The role of [the document] is to further select the optimal document from the initial candidate document pool as the final evidence D. The overall document re-ranking process can be simplified as follows: D = Rd (D) coarse , q , e ; prompt d ).

[0010] Furthermore, in step four, the re-ranking evaluation module uses the following formula to score the document: S41. The reordered document set D is used as input to the evaluation module. After the initial reordering, the system will select a candidate document set D, which contains documents related to the question and extended information. S42, System call to large language model E d Each document is analyzed to assess whether it accurately answers the input question q and is consistent with the extended information e. The assessment results are presented in the form of a score, ranging from 0 to 1, representing the relevance and quality of the document. S43. Use the prompt word. d Guide the language model to perform document evaluation to ensure that the evaluation process meets system requirements, using the prompt word. d Used to provide instructions to the language model, enabling it to understand how to evaluate the relevance of a document to a question; S44. The evaluation module calculates document scores using the following formula: Furthermore, in steps five and six, the answer generation module and the answer verification module use the following process and formula to generate and verify the answer: S51. Using LLM to generate answers based on questions and evidence: Input the question q and the reordered document set D into the generator, then use the prompt word. at The LLM process guides the answer generation process, ultimately generating the final answer a. t : a t = G a (q, D; prompt at ); S52. Verify the generated answer: Verify the initially generated answer a t Enter the question q and evidence D into the validator, then use the prompt word. av Guide the verification process, and finally verify and output the answer. If the answer passes verification, output the final answer 'a'. v : a v = G v (q, a t , D; prompt av ); S53. When the retrieval fails, the framework directly generates the answer through the generated expansion.

[0011] Furthermore, the steps to regenerate the answer are as follows: S531. When the user enters question q, the system first calls the retrieval module to attempt to retrieve the candidate document set D from the knowledge base or document set. coarse After completing the initial search, the system checks the search results to determine whether the following conditions are met: the search results are empty; the number of documents is insufficient; or the document quality is substandard.

[0012] S532. After determining that the retrieval has failed, the system calls the extended generation module to generate extended information e related to the question q. The system inputs the extended information and the question together into the answer generation module, and then uses model G... a To generate a reasonable answer, the formula for the answer generation process is as follows: a t = G a (q, D; prompt) at ); S533. When generating answers, the system dynamically adjusts the prompt word based on extended information and question type. at The flexibility in the design of the prompts allows the system to generate answers in different styles.

[0013] Furthermore, the framework supports different types of datasets and dynamically adjusts the expansion generation and document reordering strategies based on changes in the dataset to reduce noise interference and improve system performance. The dynamic strategy adjustment of the system can be described by the following formula: strategy t+1 = f (dataset) t+1 prompt t+1 ).

[0014] Furthermore, both the answer generator and validator utilize a fine-tuned large language model, optimized based on task-specific datasets. The LLM is fine-tuned using domain datasets to excel in various application scenarios. Each dataset is carefully selected to ensure the model understands domain-specific terminology and common issues. The fine-tuning formula is as follows: θ′ = fine_tune ( θ ; D task ); Where θ are the initial parameters of the model; D task θ′ is the task dataset used for fine-tuning; θ′ is the model parameters after fine-tuning.

[0015] Furthermore, the workflow of the answer generation module is as follows: The system receives the input question q and extended information e, and generates a high-quality answer based on the user-input question and extended information e. This information is then processed by the model generation module G.a and adaptive prompt at The system guides the generation process, adjusting the generation format according to task requirements. The generation formula is as follows: a t = generate( q , e , prompt at ; θ′); The workflow of the answer validation module: The validation module uses the validation model G. v The system checks the generated answer by inputting the question q and the generated answer a. t Input the extended information 'e' into the validation model, and input it according to the validation prompt. av The system guides the verification process. If verification fails, the system will regenerate the answer based on the feedback. The verification formula is: a v = verify ( q , at , e , prompt av ; θ′ ).

[0016] The beneficial effects of this invention are: Combining generation and verification: The comprehensive framework proposed in this invention combines generation and verification, and evaluates and optimizes at multiple stages, thereby significantly improving the performance of the ODQA system in terms of generation expansion, document selection and answer generation; High-quality extension generation and evaluation: By generating high-quality query extensions and rigorously evaluating them, this invention can effectively improve the accuracy of document reordering and answer generation; Document reordering optimization: By introducing relevance assessment of extended content during the document reordering process, the system can better filter out the documents most relevant to the question and reduce the impact of noisy documents on answer generation; Dynamic verification mechanism: A verification mechanism is introduced after the answer is generated to ensure the accuracy and consistency of the output answer, reduce the possibility of error generation, and improve the overall robustness of the system; Multi-level quality control: Through a multi-stage quality assessment and screening mechanism, this invention can ensure high-quality data processing at every step, ultimately significantly improving the accuracy and performance of the system; A genetic algorithm is introduced to compare and analyze the generated results, select the better ones, and continue to evolve the genetic algorithm. In summary, this invention addresses the shortcomings of existing systems by employing model fine-tuning and dynamic prompt word optimization strategies to improve the system's adaptability and performance, while supporting multi-domain and multi-task question answering. It also proposes an extended answer generation module to ensure reasonable answers are generated even when documents are insufficient, guaranteeing the continuity of the question-and-answer process. Furthermore, a verification and feedback mechanism is added to logically review answers before output, reducing errors. This system adopts a deeply integrated architecture of retrieval, generation, and verification, achieving efficient question-and-answer capabilities across multiple tasks and domains, providing reliable technical support for scenarios such as intelligent customer service, medical consultation, and legal counsel. Attached Figure Description

[0017] Figure 1 A comprehensive processing framework for open-domain question answering; Figure 2 A flowchart illustrating the generation, sorting, and verification process for an open-domain question-answering system; Figure 3 Flowchart of the extension, retrieval, and verification framework for an open-domain question-answering system; Figure 4 A performance comparison of open-domain question answering systems on the TriviaQA, WebQ, and NQ datasets; Figure 5 The results of ablation experiments on the EGV model framework on the TriviaQA, WebQ, and NQ datasets are presented. Detailed Implementation

[0018] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0019] Please see Figure 1-5 This invention provides a technical solution for a comprehensive processing framework for open-domain question answering tasks: Example 1: This example provides a comprehensive generation and verification framework for Open Domain Question Answering (ODQA), specifically including the following steps: Step 1: Generate the extension First, the system starts with the input question and uses a large-scale language model (LLM) to generate additional background information, called query expansion. The purpose of generating expansion is to provide richer context to improve the accuracy of subsequent document selection and answer generation. A pre-trained large-scale language model, such as the GPT series, receives the question and its prompts and outputs the expansion. The generated expansion may include relevant background knowledge, contextual explanations, or other auxiliary information. These expansions help enrich the semantics of the question, thus providing stronger support for answer generation and improving the quality of subsequent retrieval and answer generation.

[0020] Step Two: Extended Assessment After generating extensions, these extensions need to be evaluated for quality to ensure logical consistency and relevance to the problem. To this end, an LLM evaluator is used to score each extension from 0 to 1, selecting the optimal extension for subsequent processing. This method filters out low-quality extensions, ensuring that subsequent steps retain only high-quality, problem-relevant extensions, avoiding noise interference and irrelevant information from affecting the final result.

[0021] The extension generation step leverages the knowledge stored in the large language model to generate additional contextual information, thereby improving the coverage of the context and the depth of understanding of the problem. The extension generation and extension evaluation processes perform the following steps: S11. Input Question: Define the core question the system needs to answer to guide the extended generation module in generating relevant background information. The input question q can be a short natural language question, such as "What is quantum computing?" or "Who was Einstein's mentor?". These questions are user-input queries, typically short and lacking sufficient contextual information. Standardize the question, such as by removing redundant symbols or stop words, to adapt it to the input format of a large language model; S12. Prompt Generation: Generate a specific prompt based on the input question. e Used to guide large language models in generating background information that meets target requirements. (Prompt word) e It is a command or context used to guide the output of a language model; S13, Model Invocation: Combining question q and prompt word q e As input, the large language model G is invoked. e Extended generation is then performed. In this step, a large language model (such as GPT or other pre-trained language models) generates extended information based on the input. The model not only handles the initial question but also generates diverse background information based on the instructions given by the prompts: e = G e (q; prompt e ); S14. Extended Generation: Generates background information relevant to the question to support subsequent document retrieval and answer generation steps. The output of this step is extended content e, which can be in the form of natural language text paragraphs. These extensions not only provide basic background information on the question but also predict relevant information, helping the system find more accurate evidence during the retrieval process. score ev = E e (q; e; prompt ev ).

[0022] For the initial generation, the genetic algorithm generates new Prompt combinations in each generation. These combinations are adjusted according to different settings, or even knowledge in a specific domain, and the generated new Prompts are scored. When the Prompt generated by the genetic algorithm surpasses the current template, it is set as the new benchmark solution. The system can continuously evolve in different task types. In this way, it breaks the limitations of traditional templates and enables the system to flexibly handle various tasks from historical problems to logical reasoning problems.

[0023] Step 3: Document Reordering The system combines the expanded evaluation with the original question and re-retrieves and ranks candidate documents. Through this re-ranking mechanism, the system prioritizes documents most likely to contain the answer. An LLM sorter processes the candidate document set, outputting the optimal document set for answer generation, ensuring that more relevant documents are prioritized. The document ranking considers not only their direct relevance to the question but also their indirect relevance based on the expanded content, ensuring that the final selected documents provide more reliable evidence for answer generation.

[0024] The document reordering module reorders documents according to the following steps: S31. To identify the most suitable documents, we first use established information retrieval techniques such as DPR to retrieve a set of the top n documents that may be relevant to the given question. These retrieval methods provide an initial score for each candidate document to describe the relevance between the document and the question. However, these methods may not always capture the subtle semantic relationships between the query and the documents, leading to false positives or irrelevant documents in the initial set. Therefore, the document re-ranking process still utilizes the ranking capabilities of the large model, and a corresponding prompt is manually designed. d Hint: Use it to complete the sorting; S32. Reordering: Next, the documents are reordered to prioritize those more likely to contain the answer. This stage involves comparing documents to determine which documents are more relevant and of higher quality to the query. Guide the LLM to act as a document reorderer. d The role of [the document] is to further select the optimal document from the initial candidate document pool as the final evidence D. The overall document re-ranking process can be simplified as follows: D = R d (D) coarse , q , e ; prompt d ).

[0025] Step 4: Reordering Evaluation After document reordering, the sorted document set undergoes further quality assessment to ensure high relevance and quality of content. An LLM evaluator is used to score the documents, and high-quality documents are further selected for answer generation based on the scoring results. Through multiple screening and evaluation processes, the documents required for answer generation are ensured to have high credibility and informational value, thus enhancing the quality of the generated answers.

[0026] The re-ranking evaluation module uses the following formula to score documents: S41. The reordered document set D is used as input to the evaluation module. After the initial reordering, the system selects a candidate document set D, which contains documents related to the question and extended information. However, due to possible sorting errors or some noisy information, the quality of these documents needs to be further evaluated. S42, System call to large language model E d Each document is analyzed to assess whether it accurately answers the input question q and its consistency with the extended information e. The evaluation results are presented as a score, ranging from 0 to 1, representing the document's relevance and quality. S43. Use the prompt word. d Guide the language model in document evaluation to ensure the evaluation process meets system requirements. (Prompt word) d Used to provide instructions to the language model, enabling it to understand how to evaluate the relevance of a document to the question. For example, prompts can instruct the model to score based on whether the document directly answers the question, or on whether the document is consistent with extended information; S44. The evaluation module calculates document scores using the following formula: score d = E d (D; q; e; prompt) d ).

[0027] Step 5: Answer Generation After document reordering and evaluation, an LLM reader is used to generate the final answer based on the question and the filtered document set. The answer generated by the model is based not only on direct information in the documents, but also on extended content and question context, resulting in a more accurate and relevant answer.

[0028] Step Six: Answer Verification After generating an initial answer, a validation mechanism is introduced. An LLM validator is used to verify the answer to ensure its consistency and accuracy. If validation fails, the system regenerates the answer. This validation process can detect and correct potential errors during generation, thereby improving the reliability of the final output answer.

[0029] The answer generation and verification modules use the following process and formula to generate and verify answers: S51. Use LLM to generate answers based on questions and evidence. Input questions and evidence: Input the question q and the reordered document set D into the generator. Prompt design: Use prompts. at Guided answer generation process. Answer generation: The final answer a is generated by LLM. t : a t = G a (q, D; prompt at ); S52. Verify the generated answer to ensure its accuracy and consistency. If verification fails, regenerate the answer. Input the question and the generated answer: Input the initially generated answer a t Input the question q and evidence D into the validator. Prompt design: Use the prompt word. av Guide the verification process; Verification and output: If the answer passes verification, output the final answer 'a'. v Otherwise, generate a new answer; a v = G v (q, a t , D; prompt av ); S53. The framework can directly generate answers through the generated expansion when retrieval fails, thereby reducing the impact of missing documents on answer generation and ensuring the continuity of the question-and-answer process. The steps for regenerating answers are as follows: S531. When the user enters question q, the system will first call the retrieval module to attempt to retrieve the candidate document set D from the knowledge base or document set. coarse After completing the initial search, the system will check the search results to determine whether the following conditions are met: the search results are empty; the number of documents is insufficient; or the document quality is substandard.

[0030] S532. After determining that the retrieval has failed, the system will call the extended generation module to generate extended information e related to the question q. The system will input the extended information and the question together into the answer generation module, and then use model G to generate the answer. a To generate a reasonable answer, the formula for the answer generation process is as follows: a t = G a (q, D; prompt) at ); S533. Dynamic Prompt Optimization and Generation Quality Control: When generating answers, the system dynamically adjusts the prompt words based on extended information and question type. at The prompts not only determine the format and depth of the generated answers, but also ensure that the model's output is highly consistent with the extended information. The flexibility in prompt design allows the system to generate answers in different styles, such as brief responses, paragraph explanations, or structured lists. Example 2: Building upon Example 1, the framework also supports different types of datasets and dynamically adjusts the expansion generation and document reordering strategies based on changes in the dataset to reduce noise interference and improve system performance. Specifically, this includes: Dynamic adaptation and update mechanism: Ensures that the open-domain question answering system can adapt to various types of datasets and automatically adjust the expansion generation and document reordering strategies according to changes in the dataset, thereby improving the system's flexibility and robustness.

[0031] Dataset Support and Recognition: The system automatically detects the type of the input dataset and triggers corresponding strategies based on the type; Structured data: Prefers direct retrieval and matching. Unstructured data: Relies on extended generation modules to provide rich background information to optimize retrieval.

[0032] Dynamic monitoring of dataset changes: When a change in the dataset is detected, the system automatically triggers a strategy adjustment, reconfigures the extended generation and reordering modules, and monitors the dataset changes in real time to adapt to the impact of data updates.

[0033] Adjusting the extended generation strategy: When data is updated, the system adjusts the extended generation strategy based on the newly added data to ensure that the generated extended information matches the latest state of the dataset. (Prompt word) e It will automatically update based on data changes, guiding the model to generate more targeted extended content.

[0034] Adjusting the document re-ranking strategy: When the dataset changes, the system adjusts the parameters of the document re-ranking module. For example, when the number of documents increases, the system can dynamically increase the step size of the sliding window to improve sorting efficiency.

[0035] The sliding window mechanism has been optimized: the parameters of window size and step size can be automatically adjusted according to the size of the dataset.

[0036] The dynamic policy adjustment of the system can be described by the following formula: strategy t+1 = f (dataset) t+1 prompt t+1 ). Example 3: Both the answer generator and the validator use a fine-tuned large language model (LLM) and are optimized based on specific datasets for different tasks.

[0037] Fine-tuning Large Language Models (LLMs): LLMs are fine-tuned using domain-specific datasets to excel in different application scenarios. Each dataset is carefully selected to ensure the model understands domain-specific terminology and common questions. For example, in medical question answering, the model is fine-tuned using medical literature and medical records, while in legal question answering, it is optimized using legal regulations and case databases. θ′ = fine_tune ( θ ; D task ); θ are the initial parameters of the model; D task θ′ is the task dataset used for fine-tuning; θ′ is the model parameters after fine-tuning.

[0038] The workflow of the answer generation module is as follows: The system receives the input question q and extended information e, and generates high-quality answers based on the user-input question and extended information e. This information is then processed by the model generation module G. a and adaptive prompt at Guide the generation process. The system adjusts the generated format according to task requirements, such as paragraph-style explanations or brief answers: a t = generate( q , e , prompt at ; θ′).

[0039] The workflow of the answer validation module: The validation module uses the validation model G. v The system checks the generated answer. The system will take the input question q and the generated answer a. t Input the extended information 'e' into the validation model, and input it according to the validation prompt. av The system guides the verification process; if verification fails, the system will regenerate the answer based on the feedback. a v = verify ( q , at , e , prompt av ; θ′ ).

[0040] Dynamic prompts and adaptive optimization: The system generates different prompts based on the complexity of the question and the context. The purpose of the prompts is to guide the model to generate answers that meet the requirements and to ensure that the generated content is consistent.

[0041] The system's multi-task support and robustness: When processing different tasks, the system can dynamically optimize based on the task dataset. Whether in education, healthcare, or the legal field, the system can adapt to task requirements by fine-tuning the model and prompts, and reduce the output of incorrect answers through the verification module. Example 4: This example provides experiments to evaluate the performance of the proposed integrated processing framework in the Open Domain Question Answering (ODQA) task. Multiple benchmark datasets and contrasting models are used in the experiments to comprehensively evaluate the advantages of the proposed method.

[0042] The experiment selected three widely used ODQA benchmark datasets to evaluate model performance: WebQ (WebQuestions), NQ (Natural Questions), and TriviaQA, which are important open-domain question answering datasets, covering questions and answers from Google Suggest API, Google search queries, and intelligent question answering websites, respectively. To verify the model's performance, the study used two types of benchmark models for comparison: The first type is traditional methods that do not rely on large language models (LLMs), such as BM25+BERT combining sparse retrieval with BERT for text representation, REALM retrieving information from a knowledge base and incorporating it into training, DPR using a dense encoder for vector retrieval, RAG combining retrieval and generation tasks, and FiD using a retrieval-reading architecture for question answering; the second type uses LLMs as generators, such as GenRead generating diverse documents based on clustering, and EAR improving evidence quality through query reordering.

[0043] The experimental procedure is as follows Figure 3 As shown, the process includes generation extension, extension evaluation, document reordering, reordering evaluation, genetic evolution, answer generation, and answer verification. In the genetic evolution stage, the answer is compared with the answer generated by the genetic algorithm, and the best answer is selected for the next step.

[0044] The model's performance is evaluated using exact-match scores. For example... Figure 4 As shown, the EM score measures the correctness of an answer by comparing its standardized form with a list of acceptable answers. These assessments validate the effectiveness of the proposed framework in open-domain question-answering tasks. The EM score compares the degree of match between the predicted answer and each true answer, while recall is also a metric for evaluating the quality of evidence.

[0045] EGV achieved significant improvements on three datasets: a 13.3-point improvement in TriviaQA, a 6.9-point improvement in WebQ, and a 9.5-point improvement in NQ. This demonstrates the effectiveness of LLM in open-domain question answering (ODQA) and highlights the impact of model size on results. Furthermore, EGV outperformed FiD-xl by an average of 9.9 points across the three datasets, despite using fewer documents, proving that LLM can outperform previously designed models in different role-playing scenarios.

[0046] Next, ablation experiments were conducted, removing the extended generator, reorderer, and evaluator respectively, to explore the impact of these four aspects on the performance of the open-domain question answering system (ODQA). Figure 5 As shown, the role of the extended generator in LLM (Large Language Model) has the greatest impact among the four different roles, indicating that question extensions can be used as auxiliary documents and evidence documents; the reordering unit also contributes to ODQA because the reordered documents are more relevant to the question; the feasibility of the evaluator is also demonstrated because it can estimate the quality of evidence and select the most appropriate document. The impact of the validator on ODQA is also important, indicating that for existing questions, the validator can play a good role after candidate documents and preliminary answers.

[0047] The above description is only a preferred embodiment of the present invention and does not limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A comprehensive processing framework for open-domain question answering tasks, characterized in that, The framework includes an extended generation module, an extended evaluation module, a document reordering module, a reordering evaluation module, an answer generation module, and an answer verification module. It also includes the following steps: Step 1, Generate Extensions: Starting from the user-input question, use a large language model (LLM) to generate additional background information. The pre-trained large language model receives the question and its prompts, and outputs extensions. Step 2, Expansion Evaluation: After generating the expansions, each expansion is scored using an LLM evaluator, with a score range of 0 to 1, to select the optimal expansion for subsequent processing. In this way, low-quality expansions are filtered out, ensuring that the data processed in subsequent steps has high quality and relevance. Step 3, Document Reordering: Based on the initially retrieved document set D coarse The evaluated optimal extension e is combined with the original question q, and candidate documents are re-retrieved and ranked using the LLM sorter R. d Process the candidate document set and output the optimal document set D for answer generation; In step three, the document reordering module reorders the documents according to the following steps: S31. First, using established information retrieval techniques, retrieve the top n documents relevant to the given question. These retrieval methods provide an initial score for each candidate document to describe the relevance between the document and the question. Then, leveraging the ranking capabilities of a large model, manually design and provide the corresponding prompt. d Tip: Use it to reorder documents; S32. Next, the preliminary candidate document set D coarse The reordering process involves comparing documents to determine which documents are more relevant and of higher quality to the query, guiding the LLM to act as a document reorderer. d The role of further developing from the initial candidate document set D coarse The optimal document is selected as the final evidence D, and the overall document re-ranking process is simplified as follows: D = R d ( D coarse , q , e ; prompt d ); Step 4, Re-ranking Evaluation: After the documents are re-ranked, a further quality evaluation is performed on the re-ranked document set to ensure the high relevance and quality of the document content. The LLM evaluator is used to score the documents, and high-quality documents are further selected for answer generation based on the scoring results. Step 5, Answer Generation: After document reordering and evaluation, the final answer is generated using an LLM reader based on the question and the filtered document set; Step 6, Answer Verification: After generating the initial answer, a verification mechanism is introduced. The LLM validator checks the logical consistency and accuracy of the answer. If the verification fails, the system will regenerate the answer.

2. The comprehensive processing framework for open-domain question answering tasks according to claim 1, characterized in that, In step one, the extended generation module performs the following operations: S11. Input problem: Define the core problem that the system needs to answer in order to guide the extended generation module to generate relevant background information and standardize the problem to adapt it to the input format of a large language model; S12. Prompt Generation: Generate a prompt based on the input question. e Used to guide large language models in generating background information that meets target requirements; S13, Model Invocation: Combining question q and prompt word q e As input, the large language model G is invoked. e In the extended generation step, the large language model generates extended information based on the input. The model not only handles the initial question but also generates diverse background information by combining instructions given by prompts. e = G e (q; prompt e ); S14. Extended Generation: Generate background information related to the question to support subsequent document retrieval and answer generation steps. The output of this step is extended content e, which is in the form of natural language text paragraphs.

3. The comprehensive processing framework for open-domain question answering tasks according to claim 2, characterized in that, In the extended generation module, the genetic algorithm generates new Prompt combinations in each generation. These combinations are adjusted according to different settings and domain knowledge, and the generated new Prompts are scored. When the Prompt generated by the genetic algorithm surpasses the current template, it is set as the new benchmark solution.

4. The comprehensive processing framework for open-domain question answering tasks according to claim 1, characterized in that, In step four, the re-ranking evaluation module uses the following formula to score the document: S41. The reordered document set D is used as input to the evaluation module. After the initial reordering, the system will select a candidate document set D, which contains documents related to the question and extended information. S42, System call to large language model E d Each document is analyzed to assess whether it accurately answers the input question q and is consistent with the extended information e. The assessment results are presented in the form of a score, ranging from 0 to 1, representing the relevance and quality of the document. S43. Use the prompt word. d Guide the language model to perform document evaluation to ensure that the evaluation process meets system requirements, using the prompt word. d Used to provide instructions to the language model, enabling it to understand how to evaluate the relevance of a document to a question; S44. The evaluation module calculates document scores using the following formula: score d = E d ( D ; q ; e ; prompt d )。 5. A comprehensive processing framework for open-domain question answering tasks according to claim 1, characterized in that, In steps five and six, the answer generation module and the answer verification module use the following process and formula to generate and verify the answer: S51. Using LLM to generate answers based on questions and evidence: Input the question q and the reordered document set D into the generator, then use the prompt word. at The LLM process guides the answer generation process, ultimately generating the final answer a. t : a t = G a (q, D; prompt at ); Among them, G a Generate a dedicated, finely tuned large language model for the answer; S52. Verify the generated answer: Verify the initially generated answer a t Enter the question q and evidence D into the validator, then use the prompt word. av Guide the verification process, and finally verify and output the answer. If the answer passes verification, output the final answer 'a'. v : a v = G v (q, a t , D; prompt av ); Among them, G v A dedicated fine-tuning large language model for answer verification; S53. When the retrieval fails, the framework directly generates the answer through the generated expansion.

6. A comprehensive processing framework for open-domain question answering tasks according to claim 5, characterized in that, The steps to regenerate the answer are as follows: S531. When the user enters question q, the system first calls the retrieval module to attempt to retrieve the candidate document set D from the knowledge base or document set. coarse After completing the initial search, the system checks the search results to determine whether the following conditions are met: the search results are empty; the number of documents is insufficient; or the document quality is substandard. S532. After determining that the retrieval has failed, the system calls the extended generation module to generate extended information e related to the question q. The system inputs the extended information and the question together into the answer generation module, and then uses model G... a Generate reasonable answers; S533. When generating answers, the system dynamically adjusts the prompt word based on extended information and question type. at The flexibility in the design of the prompts allows the system to generate answers in different styles.

7. A comprehensive processing framework for open-domain question answering tasks according to claim 1, characterized in that, Both the answer generator and validator utilize a fine-tuned large language model, optimized based on datasets for different tasks. The LLM is fine-tuned using domain datasets to excel in various application scenarios. Each dataset is carefully selected to ensure the model understands domain-specific terminology and common issues. The fine-tuning formula is as follows: θ′ = fine_tune ( θ ; D task ); Where θ are the initial parameters of the model; D task θ′ is the task dataset used for fine-tuning; θ′ is the model parameters after fine-tuning.

8. A comprehensive processing framework for open-domain question answering tasks according to claim 7, characterized in that, Fine-tuning workflow of the answer generation module: The system generates high-quality answers based on user-input questions and extended information. It receives the input question q and extended information e, and generates answers through the model generation module Ga and adaptive prompt words. at The system guides the generation process, adjusting the generation format according to task requirements. The generation formula is as follows: a t = G a ( q , e , prompt at ; θ′); The workflow of the answer validation module: The validation module uses the validation model G. v The system checks the generated answer by inputting the question q and the generated answer a. t Input the extended information e into the verification model (verify), and then input it according to the verification prompt. av The system guides the verification process; if verification fails, the system will regenerate answer 'a' based on the feedback. v The verification formula is: a v = verify ( q , at , e , prompt av ; θ′ )。

Images