Multi-stage search enhancement generation method, computer system, and computer-readable storage medium

Abstract

The application relates to the technical field of natural language processing, and in particular to a multi-stage retrieval enhancement generation method, a computer system and a computer readable storage medium. Whether external retrieval is needed for an input question is detected, ambiguity and decomposition are detected in sequence, in the ambiguity elimination stage, when the model determines that the question has ambiguity, the problem semantics is clarified by constructing a prompt word template, so that the error retrieval caused by ambiguity is avoided. In the question decomposition stage, corresponding decomposition strategies are adopted according to different question types to ensure the integrity and accuracy of the retrieval and reasoning process. Finally, in the answer generation stage, the model comprehensively utilizes the external retrieval content and the self-knowledge according to the discrimination result to generate the answer, and integrates the sub-question answers to obtain the final output result. The application aims to solve the problem of how to realize accurate semantic retrieval of complex questions while controlling the computational overhead of the multi-stage retrieval enhancement generation method.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a multi-stage retrieval enhancement generation method, a computer system, and a computer-readable storage medium. Background Technology

[0002] Retrieval Augmented Generation (RAG) is a method that combines information retrieval with natural language generation. It refers to retrieving relevant, up-to-date, and reliable information from an external knowledge base before allowing a large model to answer questions or generate text. This information is then provided to the large model as context along with the question, ultimately enabling the model to generate more accurate and credible answers.

[0003] In the traditional RAG workflow, RAG divides the knowledge base text into several paragraphs and performs retrieval based on the user's question. The quality of this retrieval directly affects the generated results. This approach is called the single-stage retrieval method, which means that only one retrieval operation is performed in the retrieval-enhanced generation process. After that, the most relevant information fragments are directly obtained from the knowledge base and provided to the large language model to generate the answer.

[0004] However, when faced with complex problems that require multi-step reasoning or the synthesis of information from multiple different sources, such as comparison, experience summarization, or debate, single-stage retrieval methods struggle to perform multi-level analysis and understanding of complex problems, and are prone to issues such as information redundancy, retrieval errors, or incomplete answers.

[0005] The common multi-stage retrieval method currently used is to first perform a coarse screening and then a fine screening. This approach achieves higher accuracy in detecting complex problems by stacking the number of retrievals. However, its disadvantage is also obvious - the computational cost brought about by stacking the number of retrievals and screenings will increase exponentially.

[0006] In view of this, this application proposes a novel multi-stage retrieval enhancement generation method, which aims to adapt to complex problems while controlling the computational overhead of the multi-stage retrieval enhancement generation method. Summary of the Invention

[0007] The main objective of this application is to provide a multi-stage retrieval enhancement generation method, which aims to solve the problem of how to achieve accurate semantic retrieval for complex problems while controlling the computational overhead of the multi-stage retrieval enhancement generation method.

[0008] To achieve the above objectives, this application provides a multi-stage retrieval enhancement generation method, applied to a large language model that supports multi-stage retrieval enhancement generation. The method includes the following steps:

[0009] Upon receiving the initial question input by the user, determine whether it is necessary to invoke an external search template to perform an external search based on the external search identifier associated with the initial question;

[0010] When the first external search identifier is detected, it is determined whether the disambiguation template needs to be invoked to disambiguate the first question based on the disambiguation identifier associated with the first question after the external search is performed.

[0011] When a disambiguation flag is detected, determine whether it is necessary to invoke the issue decomposition template to decompose the second issue or the first issue that has been updated after disambiguation.

[0012] When the first decomposition identifier is detected, the target question type template corresponding to the third question updated after decomposing the second question or the first question is invoked, and the result corresponding to the third question is generated and output based on the target question type template.

[0013] Optionally, the step of determining whether to invoke an external search template for external retrieval based on the external search identifier associated with the initial question includes:

[0014] Obtain the external retrieval identifier generated by the initial question based on the large language model;

[0015] When the external search identifier is the first external search identifier, the external search template is invoked to perform an external search on the initial question, and the initial question is updated to the first question based on the external search results;

[0016] When the external search identifier is the second external search identifier, the internal search template is invoked to generate and output the result corresponding to the initial question.

[0017] Optionally, the step of determining whether to invoke a disambiguation template to disambiguate the first question based on the disambiguation identifier associated with the first question updated after external retrieval includes:

[0018] Obtain the disambiguation identifier generated for the first question based on the large language model;

[0019] When the disambiguation identifier is the first disambiguation identifier, the disambiguation template is invoked to disambiguate the first question, and the first question is updated to the second question based on the disambiguation result;

[0020] When the disambiguation identifier is the second disambiguation identifier, the disambiguation template is not invoked.

[0021] Optionally, the step of determining whether it is necessary to invoke the problem decomposition template to decompose the second problem or the first problem after disambiguation includes:

[0022] Obtain the decomposition identifier of the second question or the first question based on the large language model;

[0023] When the decomposition identifier is the first decomposition identifier, the problem decomposition template is invoked to decompose the second problem or the first problem;

[0024] When the decomposition identifier is the second decomposition identifier, the second question or the first question is used as the retrieval request, and the vectorized semantic retrieval method is used to obtain relevant paragraphs from the external corpus. The relevant paragraphs are used as the retrieval results, and the answer corresponding to the second question or the first question is output.

[0025] Optionally, the step of using the second question or the first question as a retrieval request, employing a vectorized semantic retrieval method to obtain relevant paragraphs from an external corpus, using the relevant paragraphs as retrieval results, and outputting the answer corresponding to the second question or the first question includes:

[0026] The second question or the first question is encoded into a user question vector representation using a pre-trained text encoder, and candidate paragraphs from an external corpus are encoded into candidate paragraph vector representations.

[0027] Calculate the cosine similarity between the user question vector representation and each of the candidate paragraph vector representations;

[0028] Sort the cosine similarities from largest to smallest, and select the top K candidate segments. As the relevant paragraph ;

[0029] The relevant paragraphs are used as search results to generate and output the answer to the second question or the first question.

[0030] Optionally, the target question type template includes a comparative question template, an experiential question template, and a debate question template. The step of calling the target question type template corresponding to the third question updated after decomposing the second question or the first question, and generating and outputting the result corresponding to the third question based on the target question type template, includes:

[0031] Obtain the question type identifier generated by the large language model for the third question;

[0032] When the question type identifier is the first question type identifier, the comparison question template is invoked to generate and output the result corresponding to the third question;

[0033] When the problem type identifier is the second problem type identifier, the experience-based problem template is invoked to generate and output the result corresponding to the third problem;

[0034] When the question type identifier is the third question type identifier, the debate-type question template is invoked to generate and output the result corresponding to the third question.

[0035] Optionally, the step of calling the comparison question template to generate and output the result corresponding to the third question includes:

[0036] Based on the comparison question template, the third question is broken down into multiple comparison types, comparison keywords, and comparison sub-questions. Sub-answers corresponding to each comparison sub-question are generated and output according to the comparison types and comparison keywords.

[0037] The step of calling the experience-based question template to generate and output the result corresponding to the third question includes:

[0038] Based on the experience-based question template, identify multiple experience-based keywords and experience-based sub-questions in the third question, and generate and output sub-answers corresponding to each experience-based sub-question according to the experience-based keywords;

[0039] The step of calling the debate-type question template to generate and output the result corresponding to the third question includes:

[0040] Based on the debate-type question template, multiple debate subjects, debate-type keywords, and debate-type sub-questions in the third question are identified. Sub-answers corresponding to each debate-type sub-question are generated and output according to the debate subjects and the debate-type keywords.

[0041] In addition, to achieve the above objectives, this application also provides a computer system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the multi-stage retrieval enhancement generation method as described in any of the preceding claims.

[0042] In addition, to achieve the above objectives, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the multi-stage retrieval enhancement generation method as described in any of the preceding claims.

[0043] This application has at least the following beneficial effects:

[0044] 1. Unlike traditional "overlay" multi-stage retrieval enhancement generation methods, the method in this embodiment performs at most two retrieval and filtering operations on the input question. The remaining actions, such as identifier assignment, template invocation, and ambiguity judgment, which have low computational overhead, are implemented by the large language model calling pre-set templates or rules. The processing object is the input question itself, rather than the filtering of the candidate answers corresponding to the question, thereby effectively controlling and saving computational costs;

[0045] 2. By incorporating ambiguity resolution and problem decomposition mechanisms, the ability to handle complex problems is enhanced, enabling the model to reasonably analyze ambiguous and hierarchical problems;

[0046] 3. By dynamically generating answers by combining internal model knowledge with external search content, the completeness, accuracy, and organization of answers are significantly improved, making it suitable for various question types such as comparative, experience-based, and debate-based questions;

[0047] 4. Multi-stage processing and dynamic fusion strategies reduce the risk of information conflicts and improve the robustness and scalability of the system;

[0048] 5. The methods involved in this application can be widely applied to fields such as intelligent question answering, knowledge services, academic retrieval, intelligent customer service, and multi-scenario human-computer interaction, thereby enhancing the practical value and user experience of artificial intelligence systems. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the multi-stage retrieval enhancement generation method involved in the embodiments of this application;

[0050] Figure 2 This is a schematic diagram illustrating the operational framework of the multi-stage retrieval enhancement generation method involved in the embodiments of this application;

[0051] Figure 3 This is a schematic diagram of the hardware operating environment of the computer system involved in the embodiments of this application.

[0052] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0054] First Embodiment

[0055] This embodiment provides a multi-stage retrieval enhancement generation method, which is applied to a large language model that supports multi-stage retrieval enhancement generation.

[0056] As one of the alternative implementations, this embodiment provides a method for fine-tuning a large language model to support multi-stage retrieval enhancement generation, as detailed below:

[0057] First, various types of raw datasets are collected and preprocessed, including cleaning, formatting, and deduplication, to ultimately retain the final datasets. The question-and-answer pair format, where x is the question and y is the answer;

[0058] Then, based on the task requirements, a prompt template I is constructed to guide the large model to output control labels r, including retrieval requirement, disambiguation requirement and decomposition requirement labels, with output values ​​of True / False; and question type labels, with output values ​​of comparative c, empirical e and argumentative d.

[0059] Next, the original question-answer pair {x, y} is processed using the prompt template I. An external large model API is called to infer for each question x and generate control labels r=M(I(x, y)) corresponding to the question, where M represents the external large model, thus generating the enhanced training dataset. .

[0060] Finally, based on the enhanced training dataset The objective function for supervised fine-tuning of a large open-source model is:

[0061]

[0062] Here, M represents the open-source large model. During training, the large model simultaneously predicts the target answer. With control labels It also supports the generation of control tags by expanding the vocabulary.

[0063] By constructing a dataset and supervising fine-tuning, the pre-trained language model is made capable of outputting labels indicating whether retrieval is needed, whether ambiguity exists, and whether decomposition is required. Based on this large language model, the following multi-stage retrieval enhancement generation method is then implemented:

[0064] Reference Figure 1 The multi-stage retrieval enhancement generation method includes the following steps:

[0065] Step S10: Upon receiving the initial question input by the user, determine whether it is necessary to call the external search template to perform an external search based on the external search identifier associated with the initial question;

[0066] In this embodiment, when the user's initial input question is received, the large language model is used to determine whether external retrieval is needed. Regardless of whether external retrieval is needed, the large language model will generate a corresponding external retrieval identifier. Based on the meaning represented by the external retrieval identifier, it is determined whether to call the external retrieval template to perform external retrieval.

[0067] External retrieval templates are targets for large language models to generate instructions to retrieve relevant paragraphs from external corpora. External retrieval refers to retrieving relevant paragraphs from external corpora.

[0068] It should be noted that when the large language model identifies that the initial question requires external retrieval, it assigns a first external retrieval identifier to the initial question, indicating that the initial question needs to be searched externally; while when the large language model identifies that the initial question does not require external retrieval, it assigns a second external retrieval identifier to the initial question, indicating that the initial question can be searched internally directly.

[0069] Step S20: When the first external search identifier is detected, determine whether it is necessary to call the disambiguation template to disambiguate the first question based on the disambiguation identifier associated with the first question after the external search is performed.

[0070] In this embodiment, when the first external retrieval identifier is detected, it means that an external corpus has been introduced for retrieval. A judgment is made on whether disambiguation is needed for the first question after the external retrieval.

[0071] Disambiguation, or eliminating ambiguity, refers to modifying content that the large language model considers ambiguous in the first problem. This can involve synonym replacement, addition, or deletion. A disambiguation template is a template that implements this function.

[0072] It is worth noting that, to distinguish it from the initial question mentioned above, this embodiment refers to the initial question after external retrieval as the first question. However, unlike the second or third question that follows, the first question does not change the format of the original user-input question. The difference from the initial question is that the first question is assigned a first external retrieval identifier by the large language model after the external retrieval operation.

[0073] Additionally, it should be noted that questions that are not assigned a first external search identifier are considered initial questions and will not be processed in this step.

[0074] Step S30: When a disambiguation flag is detected, determine whether it is necessary to call the problem decomposition template to decompose the second problem or the first problem after disambiguation.

[0075] In this embodiment, after performing a disambiguation judgment on the first question, the large language model assigns a disambiguation identifier to the first question. This disambiguation identifier is generated regardless of the judgment result. At this time, if it is determined that disambiguation is required, the first question after disambiguation is updated to the second question; if disambiguation is not required, it remains the first question.

[0076] It is worth noting that the content of the second question has been modified by the large language model based on the first question.

[0077] In this embodiment, the problem is decomposed regardless of whether disambiguation has been performed. The problem is broken down into multiple sub-problems, and the answers to each sub-problem are then searched. The problem decomposition template is the template for implementing this function.

[0078] Step S40: When the first decomposition identifier is detected, the target question type template corresponding to the third question updated after decomposing the second question or the first question is invoked, and the result corresponding to the third question is generated and output based on the target question type template.

[0079] In this embodiment, the large language model assigns a first decomposition identifier to the problem that needs to be decomposed (i.e., the second problem or the first problem), and the decomposed problem is uniformly updated to the third problem.

[0080] It should be emphasized that the content of the third question has been adjusted again by the large language model based on the original questions (whether the second or the first question).

[0081] It is understandable that problems that are not assigned the first decomposition marker are problems that do not need to be decomposed and will not be executed in this step.

[0082] In this embodiment, there are multiple question type templates. The large language model matches the corresponding target question type template according to the content of the third question, thereby generating and outputting the result corresponding to the third question based on the target question type template.

[0083] Furthermore, it should be noted that during the entire execution process from step S10 to step S40, the input question undergoes at most two retrieval and filtering operations: one external retrieval and one retrieval after question decomposition. The remaining actions, such as identifier assignment, template invocation, and ambiguity judgment, are all implemented by the large language model calling pre-set templates or rules. The processing object is the input question itself, rather than filtering the candidate answers corresponding to the question. The computational cost of these actions is far less than that of filtering candidate answers. Therefore, unlike the traditional "overlay" multi-stage retrieval enhancement generation method, the method involved in this embodiment can effectively control and save computational costs.

[0084] In the technical solution provided in this embodiment, the input question undergoes sequential checks for external retrieval, ambiguity, and decomposition. During the disambiguation stage, when the model determines that the question is ambiguous, it constructs a prompt word template to clarify the question's semantics, thereby avoiding erroneous retrievals due to ambiguity. In the question decomposition stage, corresponding decomposition strategies are adopted according to different question types to ensure the completeness and accuracy of the retrieval and reasoning process. Finally, in the answer generation stage, the model generates an answer by comprehensively utilizing external retrieval content and its own knowledge based on the discrimination results, and integrates the answers to sub-questions to obtain the final output result.

[0085] Second Embodiment

[0086] Based on the first embodiment, this embodiment provides an implementation step for performing external retrieval using a large language model, wherein S10 includes:

[0087] S11, Obtain the external retrieval identifier generated by the initial question based on the large language model;

[0088] S12, when the external search identifier is the first external search identifier, the external search template is invoked to perform an external search on the initial question, and the initial question is updated to the first question based on the external search results;

[0089] S13, when the external search identifier is the second external search identifier, the internal search template is called to generate and output the result corresponding to the initial question.

[0090] For example, the external search identifier can be set to a first external search identifier True or a second external search identifier False. True means that a search is required, so S12 is executed; False means that a search is not required, so S13 is executed.

[0091] For example, an internal search template can be generated by referring to the following text:

[0092] "Please answer the following questions using only the model's own knowledge, without using external retrieval or search."

[0093] Question: {First Question} }”

[0094] In some alternative implementations, for user questions where the external search flag is False ,Will This is appended to the end of the template and used as input to fine-tune the large model M, directly generating the answer. .

[0095] Third Embodiment

[0096] Based on any embodiment, this embodiment provides an implementation step for performing disambiguation on a large language model, wherein S20 includes:

[0097] Step S21: Obtain the disambiguation identifier generated by the large language model for the first question;

[0098] Step S22: When the disambiguation identifier is the first disambiguation identifier, the disambiguation template is invoked to disambiguate the first question, and the first question is updated to the second question based on the disambiguation result;

[0099] Step S23: When the disambiguation identifier is the second disambiguation identifier, the disambiguation template is not invoked.

[0100] For example, for the first question marked as True in external search. The input is the fine-tuning large model M, and the output is a disambiguation flag, True or False. True is the first disambiguation flag, which means that disambiguation is required, and False is the second disambiguation flag, which means that disambiguation is not required.

[0101] For example, a template for implementing disambiguation functionality:

[0102] "Your task is to identify the multiple interpretations that may exist in the problem and rewrite them in an unambiguous and clear form."

[0103] Question: {First Question} }

[0104] Issues after processing: {}”

[0105] Fourth embodiment

[0106] Based on any embodiment, this embodiment provides an implementation step for the decomposition of a large language model, wherein step S30 includes:

[0107] Step S31: Obtain the decomposition identifier of the second question or the first question based on the large language model;

[0108] Step S32: When the decomposition identifier is the first decomposition identifier, the problem decomposition template is invoked to decompose the second problem or the first problem;

[0109] Step S33: When the decomposition identifier is the second decomposition identifier, the second question or the first question is used as the retrieval request, and the vectorized semantic retrieval method is used to obtain relevant paragraphs from the external corpus. The relevant paragraphs are used as the retrieval results, and the answer corresponding to the second question or the first question is output.

[0110] For example, the first question that does not require disambiguation will be determined. Or the second question after disambiguation Unification as a clear third issue Input the fine-tuning large model M, and output the first decomposition flag True or the second decomposition flag False, where True means decomposition is needed and False means decomposition is not needed.

[0111] In this embodiment, if the decomposition requirement control tag is False, it is directly used as a retrieval request, and the vectorized semantic retrieval method is used to obtain relevant paragraphs from an external corpus. Specifically, this includes:

[0112] Step S331: Encode the second question or the first question into a user question vector representation using a pre-trained text encoder, and encode the candidate paragraphs in the external corpus into candidate paragraph vector representations;

[0113] Step S332: Calculate the cosine similarity between the user question vector representation and each of the candidate paragraph vector representations;

[0114] For example, by calculating cosine similarity The semantic relevance score between the candidate paragraph and the candidate paragraph is calculated using the following formula:

[0115]

[0116] in, The vector representation of the third problem, Let be the vector representation of the candidate paragraph. Represents the vector dot product. This represents the L2 norm of a vector.

[0117] Step S333: Sort the cosine similarities from largest to smallest, and select the top K candidate segments. As the relevant paragraph ;

[0118] Step S334: Use the relevant paragraphs as search results to generate and output the answer to the second question or the first question.

[0119] For problems that require decomposition, the process jumps to step S40, which in this embodiment is implemented according to the following steps:

[0120] Step S41: Obtain the question type identifier generated by the third question based on the large language model;

[0121] Step S42: When the problem type identifier is the first problem type identifier, the comparison problem template is invoked to generate and output the result corresponding to the third problem;

[0122] Further and optionally, step S42 includes:

[0123] Step S421: Based on the comparison question template, the third question is split into multiple comparison types, comparison keywords, and comparison sub-questions. Sub-answers corresponding to each comparison sub-question are generated and output according to the comparison types and comparison keywords.

[0124] As an example, a comparison question template can be set up in the following format:

[0125] "Your task is to decompose the following comparison problems, determine the comparison types and identify the key comparison items, and then break the problems down into several well-structured subproblems."

[0126] question:{ }

[0127] Comparison type: {}

[0128] Comparative keywords: {}

[0129] Comparison type problems: { }”

[0130] As an example, the template for generating answers to comparison questions (c) can be set as follows:

[0131] Your task is to generate the final answer to the following comparison questions. Given the comparison type, keywords, and answers to each sub-question, analyze these sub-answers, compare them according to the comparison type and keywords, and output a structured comparison conclusion.

[0132] Comparison type problems: { }

[0133] Comparison type: {}

[0134] Comparative keywords: {}

[0135] Sub-answer: { }

[0136] Final answer: {}”

[0137] Step S43: When the problem type identifier is the second problem type identifier, the experience-based problem template is invoked to generate and output the result corresponding to the third problem;

[0138] Further and optionally, step S43 includes:

[0139] Step S431: Based on the experience-based question template, identify multiple experience-based keywords and experience-based sub-questions in the third question, and generate and output sub-answers corresponding to each experience-based sub-question according to the experience-based keywords;

[0140] As an example, an experience-based question template can be set up in the following format:

[0141] "Your task is to break down the following experience-based questions, identify experience-based keywords (including key entities or keywords), and then break the questions down into several well-structured sub-questions."

[0142] question:{ }

[0143] Experience-based keywords: {}

[0144] Experience-based subproblems: { }”

[0145] As an example, the template for generating answers to experience-based questions can be set as follows:

[0146] "Your task is to generate the final answer to the following experience-based questions. Given the keywords and answers to each sub-question, please analyze these sub-answers, extract the key points based on the keywords, and output the integrated experience-based answer."

[0147] Experience-based subproblems: { }

[0148] Experience-based keywords: {}

[0149] Sub-answer: { }

[0150] Final answer: {}”;

[0151] Step S44: When the question type identifier is the third question type identifier, the debate-type question template is invoked to generate and output the result corresponding to the third question.

[0152] Further and optionally, step S44 includes:

[0153] Step S441: Based on the debate question template, identify multiple debate subjects, debate keywords, and debate sub-questions in the third question; generate and output sub-answers corresponding to each debate sub-question according to the debate subjects and the debate keywords.

[0154] As an example, a template for debate-style questions can be set up in the following format:

[0155] "Your task is to break down the following debate questions, identify the debate topics and key entities or keywords, and then break the questions down into several well-structured sub-questions."

[0156] question:{ }

[0157] Debate topic: {}

[0158] Debate-style keywords: {}

[0159] Debate-style sub-questions: { }”;

[0160] As an example, a template for generating answers to debate-style questions:

[0161] Your task is to generate the final answer to the following debate questions. Given the debate topic, keywords, and answers to each sub-question, please analyze these sub-answers, organize them into the affirmative and negative viewpoints, and output the debate conclusion.

[0162] Debate-style sub-questions: { }

[0163] Debate topic: {}

[0164] Debate-style keywords: {}

[0165] Sub-answer: { }

[0166] Final answer: {}”;

[0167] Fifth Embodiment

[0168] As a verification embodiment, this embodiment provides an example of an implementation method for multi-stage retrieval enhancement generation using the method described in any of the above embodiments.

[0169] Assuming user problem for:

[0170] "Under the same quality conditions, which material, copper or aluminum, is more suitable for use as a cable conductor? Please provide reasons and data comparisons."

[0171] The large language model determines that this problem requires external retrieval and proceeds directly to the disambiguation stage:

[0172] For users whose disambiguation flag is True (first disambiguation flag is True) ,Will It is concatenated with a template and used as input to fine-tune a large model M, outputting a disambiguated user question. "Under the same quality conditions, compare the conductivity, resistivity, mechanical strength and weight effects of copper and aluminum as cable conductor materials to determine which material is more suitable as a cable conductor."

[0173] For the first question where the disambiguation flag is set to the second disambiguation flag False. User issues after disambiguation of output Unified as a clear third problem input, denoted as :

[0174] "Under the same quality conditions, compare the conductivity, resistivity, mechanical strength and weight effects of copper and aluminum as cable conductor materials to determine which material is more suitable as a cable conductor."

[0175] After disambiguation, regarding the issue ,Will Concatenated with the template and used as input to fine-tune the large model M, the output is the decomposed form:

[0176] Comparison type: {Comparison of advantages and disadvantages};

[0177] Keywords: {copper and aluminum};

[0178] Set of subproblems: { What are the electrical conductivities of copper and aluminum respectively? What are the densities of copper and aluminum? What is the difference in volume for the same mass? What are the differences between copper and aluminum in terms of mechanical strength and ductility? Considering both electrical properties and weight, which material is more suitable as a cable conductor?

[0179] For a set of subproblems obtained by decomposing the large model M by fine-tuning { }, respectively, retrieve relevant paragraphs from the external corpus using the same retrieval method as in Step 6.3:

[0180] { The electrical conductivity of copper is approximately 5.96 × 10⁻⁶. 7 S / m; the electrical conductivity of aluminum is approximately 3.5 × 10⁻⁶. 7 S / m. The density of copper is approximately 8.96 g / cm³; the density of aluminum is approximately 2.70 g / cm³. Copper has excellent ductility and high tensile strength; aluminum has good ductility, but its mechanical strength is lower than that of copper. For the same mass, aluminum cables have a larger volume and slightly lower conductivity per unit area than copper cables, but their weight is significantly reduced.

[0181] The search results and corresponding sub-questions are input into the fine-tuning model M to generate a set of answers to the sub-questions:

[0182] { Copper has a significantly higher electrical conductivity than aluminum. Aluminum has a much lower density than copper, and its volume is about three times that of copper for the same mass. Copper has better mechanical strength and ductility than aluminum, and is more reliable. In terms of overall performance, copper conductors offer high efficiency and reliability; aluminum conductors are lightweight and suitable for weight-sensitive applications.

[0183] In addition, as an implementation scheme, refer to Figure 2 The illustrated embodiment of this application relates to a multi-stage retrieval enhancement generation method, which includes the following operational framework:

[0184] The dataset building module includes:

[0185] First, various types of raw datasets are collected and preprocessed, including cleaning, formatting, and deduplication, to ultimately retain the final datasets. The question-and-answer pair format, where x is the question and y is the answer;

[0186] Then, based on the task requirements, a prompt template I is constructed to guide the large model to output control labels r, including retrieval requirement, disambiguation requirement and decomposition requirement labels, with output values ​​of True / False; and question type labels, with output values ​​of comparative c, empirical e and argumentative d.

[0187] Next, the original question-answer pair {x, y} is processed using the prompt template I. An external large model API is called to infer for each question x and generate control labels r=M(I(x, y)) corresponding to the question, where M represents the external large model, thus generating the enhanced training dataset. .

[0188] The model fine-tuning module includes:

[0189] Based on the enhanced training dataset Supervised fine-tuning of the open-source large model is performed, and the objective function is optimized as follows:

[0190]

[0191] Here, M represents the open-source large model. During training, the large model simultaneously predicts the target answer. With control labels It also supports the generation of control tags by expanding the vocabulary.

[0192] The inference phase module includes: implementing the multi-stage retrieval enhancement generation method as described in any of the above embodiments.

[0193] The answer generation module includes: querying and retrieving the corpus to generate the answer to the question after it has been processed by the reasoning stage module.

[0194] As one implementation scheme, Figure 3 This is a schematic diagram of the hardware operating environment of the computer system involved in the embodiments of this application.

[0195] like Figure 3 As shown, the computer system may include: a processor 1001, such as a CPU; a memory 1005; a user interface 1003; a network interface 1004; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0196] Those skilled in the art will understand that Figure 3 The computer system architecture shown does not constitute a limitation on the computer system and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0197] like Figure 3 As shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and computer programs. The operating system is a program that manages and controls the hardware and software resources of the computer system, as well as the operation of the computer programs and other software or programs.

[0198] exist Figure 3 In the computer system shown, the user interface 1003 is mainly used to connect to the terminal and communicate with the terminal; the network interface 1004 is mainly used to communicate with the backend server; and the processor 1001 can be used to call the computer program stored in the memory 1005.

[0199] In this embodiment, the computer system includes: a memory 1005, a processor 1001, and a computer program stored in the memory and executable on the processor, wherein:

[0200] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0201] Upon receiving the initial question input by the user, determine whether it is necessary to invoke an external search template to perform an external search based on the external search identifier associated with the initial question;

[0202] When the first external search identifier is detected, it is determined whether the disambiguation template needs to be invoked to disambiguate the first question based on the disambiguation identifier associated with the first question after the external search is performed.

[0203] When a disambiguation flag is detected, determine whether it is necessary to invoke the issue decomposition template to decompose the second issue or the first issue that has been updated after disambiguation.

[0204] When the first decomposition identifier is detected, the target question type template corresponding to the third question updated after decomposing the second question or the first question is invoked, and the result corresponding to the third question is generated and output based on the target question type template.

[0205] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0206] Obtain the external retrieval identifier generated by the initial question based on the large language model;

[0207] When the external search identifier is the first external search identifier, the external search template is invoked to perform an external search on the initial question, and the initial question is updated to the first question based on the external search results;

[0208] When the external search identifier is the second external search identifier, the internal search template is invoked to generate and output the result corresponding to the initial question.

[0209] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0210] Obtain the disambiguation identifier generated for the first question based on the large language model;

[0211] When the disambiguation identifier is the first disambiguation identifier, the disambiguation template is invoked to disambiguate the first question, and the first question is updated to the second question based on the disambiguation result;

[0212] When the disambiguation identifier is the second disambiguation identifier, the disambiguation template is not invoked.

[0213] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0214] Obtain the decomposition identifier of the second question or the first question based on the large language model;

[0215] When the decomposition identifier is the first decomposition identifier, the problem decomposition template is invoked to decompose the second problem or the first problem;

[0216] When the decomposition identifier is the second decomposition identifier, the second question or the first question is used as the retrieval request, and the vectorized semantic retrieval method is used to obtain relevant paragraphs from the external corpus. The relevant paragraphs are used as the retrieval results, and the answer corresponding to the second question or the first question is output.

[0217] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0218] The second question or the first question is encoded into a user question vector representation using a pre-trained text encoder, and candidate paragraphs from an external corpus are encoded into candidate paragraph vector representations.

[0219] Calculate the cosine similarity between the user question vector representation and each of the candidate paragraph vector representations;

[0220] Sort the cosine similarities from largest to smallest, and select the top K candidate segments. As the relevant paragraph ;

[0221] The relevant paragraphs are used as search results to generate and output the answer to the second question or the first question.

[0222] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0223] Obtain the question type identifier generated by the large language model for the third question;

[0224] When the question type identifier is the first question type identifier, the comparison question template is invoked to generate and output the result corresponding to the third question;

[0225] When the problem type identifier is the second problem type identifier, the experience-based problem template is invoked to generate and output the result corresponding to the third problem;

[0226] When the question type identifier is the third question type identifier, the debate-type question template is invoked to generate and output the result corresponding to the third question.

[0227] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0228] Based on the comparison question template, the third question is broken down into multiple comparison types, comparison keywords, and comparison sub-questions. Sub-answers corresponding to each comparison sub-question are generated and output according to the comparison types and comparison keywords.

[0229] The step of calling the experience-based question template to generate and output the result corresponding to the third question includes:

[0230] Based on the experience-based question template, identify multiple experience-based keywords and experience-based sub-questions in the third question, and generate and output sub-answers corresponding to each experience-based sub-question according to the experience-based keywords;

[0231] The step of calling the debate-type question template to generate and output the result corresponding to the third question includes:

[0232] Based on the debate-type question template, multiple debate subjects, debate-type keywords, and debate-type sub-questions in the third question are identified. Sub-answers corresponding to each debate-type sub-question are generated and output according to the debate subjects and the debate-type keywords.

[0233] Furthermore, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the process steps of the embodiments of the above methods.

[0234] Therefore, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the various steps of the multi-stage retrieval enhancement generation method as described in the above embodiments.

[0235] The computer-readable storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0236] It should be noted that, since the storage medium provided in the embodiments of this application is the storage medium used to implement the methods of the embodiments of this application, those skilled in the art can understand the specific structure and variations of the storage medium based on the methods described in the embodiments of this application, and therefore will not be repeated here. All storage media used in the methods of the embodiments of this application fall within the scope of protection of this application.

[0237] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0238] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0239] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0240] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0241] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0242] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for multi-stage retrieval augmentation generation, the method comprising: Applied to large language models that support multi-stage retrieval enhancement generation, the method includes the following steps: Upon receiving the initial question input by the user, determine whether it is necessary to invoke an external search template to perform an external search based on the external search identifier associated with the initial question; When the first external search identifier is detected, it is determined whether the disambiguation template needs to be invoked to disambiguate the first question based on the disambiguation identifier associated with the first question after the external search is performed. The first external search identifier indicates that the initial question needs to be searched externally. When a disambiguation flag is detected, it is determined whether the problem decomposition template needs to be invoked to decompose the second problem or the first problem after disambiguation and updating. The disambiguation flag is assigned by the large language model after performing a disambiguation judgment on the first problem. When the first decomposition identifier is detected, the target problem type template corresponding to the third problem updated after decomposing the second problem or the first problem is invoked. The result corresponding to the third problem is generated and output based on the target problem type template. The first decomposition identifier represents the second problem or the first problem that needs to be decomposed. Problems that are not assigned the first decomposition identifier are problems that do not need to be decomposed. The steps for determining whether to invoke the problem decomposition template to decompose the second problem or the first problem after disambiguation include: Obtain the decomposition identifier of the second question or the first question based on the large language model; When the decomposition identifier is the first decomposition identifier, the problem decomposition template is invoked to decompose the second problem or the first problem; When the decomposition identifier is the second decomposition identifier, the second question or the first question is used as the retrieval request, and the vectorized semantic retrieval method is used to obtain relevant paragraphs from the external corpus. The relevant paragraphs are used as the retrieval results, and the answer corresponding to the second question or the first question is output. The target question type template includes a comparative question template, an experiential question template, and a debate question template. The step of calling the target question type template corresponding to the third question updated after decomposing the second question or the first question, and generating and outputting the result corresponding to the third question based on the target question type template includes: Obtain the question type identifier generated by the large language model for the third question; When the question type identifier is the first question type identifier, the comparison question template is invoked to generate and output the result corresponding to the third question; When the problem type identifier is the second problem type identifier, the experience-based problem template is invoked to generate and output the result corresponding to the third problem; When the question type identifier is a third question type identifier, the debate-type question template is invoked to generate and output the result corresponding to the third question; The steps of generating and outputting the result corresponding to the third question by calling the comparison question template include: Based on the comparison question template, the third question is broken down into multiple comparison types, comparison keywords, and comparison sub-questions. Sub-answers corresponding to each comparison sub-question are generated and output according to the comparison types and comparison keywords. The steps for generating and outputting the result corresponding to the third question by calling the experience-based question template include: Based on the experience-based question template, identify multiple experience-based keywords and experience-based sub-questions in the third question, and generate and output sub-answers corresponding to each experience-based sub-question according to the experience-based keywords; The steps for generating and outputting the result corresponding to the third question by calling the debate-style question template include: Based on the debate-type question template, multiple debate subjects, debate-type keywords, and debate-type sub-questions in the third question are identified. Sub-answers corresponding to each debate-type sub-question are generated and output according to the debate subjects and the debate-type keywords.

2. The method of claim 1, wherein, The step of determining whether to invoke an external search template for external retrieval based on the external search identifier associated with the initial question includes: Obtain the external retrieval identifier generated by the initial question based on the large language model; When the external search identifier is the first external search identifier, the external search template is invoked to perform an external search on the initial question, and the initial question is updated to the first question based on the external search results; When the external search identifier is the second external search identifier, the internal search template is invoked to generate and output the result corresponding to the initial question.

3. The multi-stage retrieval enhancement generation method as described in claim 1, characterized in that, The step of determining whether to call the disambiguation template to disambiguate the first question based on the disambiguation identifier associated with the first question updated after external retrieval includes: Obtain the disambiguation identifier generated for the first question based on the large language model; When the disambiguation identifier is the first disambiguation identifier, the disambiguation template is invoked to disambiguate the first question, and the first question is updated to the second question based on the disambiguation result; When the disambiguation identifier is the second disambiguation identifier, the disambiguation template is not invoked.

4. The multi-stage retrieval enhancement generation method as described in claim 1, characterized in that, The steps of using the second question or the first question as a search request, employing a vectorized semantic retrieval method to obtain relevant paragraphs from an external corpus, using the relevant paragraphs as search results, and outputting the answer corresponding to the second question or the first question include: The second question or the first question is encoded into a user question vector representation using a pre-trained text encoder, and candidate paragraphs from an external corpus are encoded into candidate paragraph vector representations. Calculate the cosine similarity between the user question vector representation and each of the candidate paragraph vector representations; Sort the cosine similarities in descending order and select the top K candidate segments. As the relevant paragraph ; The relevant paragraphs are used as search results to generate and output the answer to the second question or the first question.

5. A computer system, characterized in that, The computer system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the multi-stage retrieval enhancement generation method as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the multi-stage retrieval enhancement generation method as described in any one of claims 1 to 4.

Images