A search enhancement generation method, electronic device, and program product

Abstract

This application provides a retrieval enhancement generation method, electronic device, and program product. By executing a thinking mode selection task, it can intelligently determine whether to enable the thinking chain reasoning mode based on the user-input query text and the retrieved product details. For complex query text, the model automatically switches to the RAG task driven by the first cue word with a thinking chain, decomposing the question and integrating key content from the product details information through a structured logical reasoning link, significantly improving the accuracy and reliability of the response results in complex scenarios. For simple query text, the model can automatically select the RAG task driven by the second cue word without a thinking chain, directly generating the answer based on the retrieved product details information, effectively shortening the response time for simple questions, ensuring the rapid response needs in pre-sales intelligent customer service scenarios, and improving the smoothness and satisfaction of the user interaction process.

Description

Technical Field

[0001] This application relates to the field of large model technology, and more specifically, to a retrieval enhancement generation method, electronic device, and program product. Background Technology

[0002] Currently, pre-sales intelligent customer service systems in the e-commerce field commonly employ Retrieval-Augmented Generation (RAG) technology, which generates answers to user questions by retrieving product details in real time. RAG is a technology that combines information retrieval and text generation, aiming to improve the accuracy and timeliness of content generated by Large Language Models (LLMs). Its core idea is to obtain relevant information by retrieving external knowledge bases in real time, and then combine the retrieval results to generate more reliable answers. It is widely used in scenarios such as enterprise question answering, search enhancement, intelligent customer service, code assistance, and knowledge graphs. The core principles of RAG include: Retrieval: When a user enters a question, RAG first retrieves documents or paragraphs related to the question from external databases (such as Wikipedia, professional documents, enterprise knowledge bases, etc.). Generation: The retrieved relevant text is input along with the user's question into the generation model (such as GPT, LLaMA, etc.), and the model generates the final answer based on the retrieved content.

[0003] One existing approach uses a direct answer mode without reasoning links, simply concatenating the retrieved product information and inputting it into a large language model to generate an answer. This approach lacks in-depth analysis of user intent and logical reasoning processes. When faced with complex questions that require multi-dimensional analysis (such as product comparison and contextualized recommendations), it is prone to factual errors or one-sided answers, resulting in a significant decrease in accuracy.

[0004] Another approach employs a response model that introduces a reasoning chain, employing Chain-of-Thought (COT) reasoning to improve accuracy by forcing step-by-step reasoning for all user questions. However, when processing simple factual queries, redundant reasoning steps significantly increase response time, resulting in low answer generation efficiency and negatively impacting user experience. Summary of the Invention

[0005] The purpose of this application is to provide a search enhancement generation method, electronic device, and program product to solve the problems of decreased accuracy in the face of complex questions and low efficiency in generating answers to simple questions, which affect user experience.

[0006] This application provides a retrieval enhancement generation method, including: Get the query text entered by the user; Based on the query text, retrieve the corresponding business details from the information source; The first major language model uses thinking patterns to determine prompt words and performs a thinking pattern selection task: based on the query text and business details, it selects a thinking pattern with or without a thinking chain. If the thought process with a thought chain is selected, the first major language model will use the first cue word with the thought chain to perform the RAG task. If the thought process without a thought chain is selected, the first major language model uses the second cue word without a thought chain to perform the RAG task. The RAG task is to generate the final answer based on the query text, business details, and hints.

[0007] In the aforementioned technical solution, this application executes the thinking mode selection task through a first large language model. Based on the user-input query text and retrieved business details, it intelligently determines whether to enable the Thinking Chain (COT) reasoning mode. For complex query texts, the model automatically switches to the RAG task driven by the first cue word with a thinking chain. By using a structured logical reasoning chain to break down the problem and integrate key content from the business details, it avoids logical loopholes or incomplete information issues caused by directly generating answers based on search results in existing technologies, significantly improving the accuracy and reliability of responses in complex scenarios. For simple query texts, the model of this application can automatically select the RAG task driven by the second cue word without a thinking chain, eliminating redundant COT reasoning steps and directly generating answers based on the retrieved business details. This effectively shortens the response time for simple questions, solves the inefficiency problem of existing technologies, ensures rapid response requirements in pre-sales intelligent customer service scenarios, and improves the smoothness and satisfaction of user interaction.

[0008] In some alternative implementations, the first prompt word instructs the first large language model to perform the following phased tasks: Perform the thinking phase task: Generate a reasoning path based on the query text and business details; Execute the answering phase task: Generate a preliminary answer based on the reasoning path; Perform the reflection phase task: verify the consistency between the preliminary answer and the detailed information. If the verification passes, output the preliminary answer as the final answer. If the validation fails, an error feedback prompt is generated, and a correction phase task is executed: the first language model regenerates the answer based on the error feedback prompt and outputs it as the final answer.

[0009] In the above technical solution, the first prompt word breaks down the RAG task with a thought chain into four distinct stages: thinking, answering, reflecting, and correcting. The thinking stage generates a structured reasoning path based on the query text and product details, providing clear logical support for answer generation and avoiding information omissions or logical gaps during the reasoning process, making the answer more evidence-based. The reflecting stage verifies the matching degree between the initial answer and the product details, avoiding the problem of answers not matching product details due to model illusions or reasoning biases from the source, ensuring that the output answer is entirely based on official product details, and improving the authority and credibility of the answer in pre-sales customer service scenarios. For initial answers that fail verification, an error feedback prompt word is generated to trigger the correction stage, driving the model to re-optimize the answer based on the feedback. Error self-checking and correction can be achieved without manual intervention, significantly reducing the error rate of the final answer and improving the response quality in complex question scenarios.

[0010] In some alternative implementations, verifying the consistency between the preliminary answer and the detailed information includes at least one of the following: attribute matching verification, logical verification, and integrity check.

[0011] Among them, attribute matching verification can avoid basic errors such as mismatched values ​​and parameters. For example, if the answer mentions "24-hour battery life", but the product details clearly state "4000mAh battery capacity, theoretical battery life of 12 hours", it will be marked as inconsistent.

[0012] Logical verification can avoid reasoning biases such as reversed cause and effect and logical conflicts. For example, if the product description states "material: pure cotton", but the answer infers "it is breathable because it contains chemical fibers", it will be marked as a logical conflict.

[0013] Completeness checks can prevent incomplete answers from missing core user needs, such as checking whether the answer omits the key points asked by the user.

[0014] In some alternative implementations, model training for the thinking pattern selection task and the RAG task includes: The model for performing the thinking mode selection task and the model for performing the RAG task share the underlying model parameters; The model performing the thinking mode selection task uses the first training data during training and calculates the first training loss; The model performing the RAG task uses the second training data during training and calculates the second training loss. Based on the first training loss and the second training loss, the joint training loss is obtained; By utilizing joint training loss for model training, unified training and learning can be achieved for both the thinking pattern selection task and the RAG task.

[0015] In some alternative implementations, the first training loss is:

[0016] The second training loss is:

[0017] The joint training loss is:

[0018] Where x represents the query text, k represents the business details, y1 represents the thinking mode selection result, and y2 represents the final answer.

[0019] In some alternative implementations, the first training loss and the second training loss employ cross-entropy.

[0020] In the above technical solution, the thinking mode selection task is essentially a classification task (selecting a mode with / without a thinking chain). Cross-entropy loss can accurately measure the difference between the mode category predicted by the model and the true category, guiding the model to quickly learn the optimal thinking mode corresponding to different query texts and business details. The answer generation stage of the RAG task can be transformed into a sequence classification problem, and cross-entropy loss can also effectively evaluate the matching degree between the generated answer and the standard answer. The first training loss and the second training loss use the same cross-entropy calculation paradigm, eliminating the need to design differentiated loss transformation and weighting strategies for different tasks, greatly simplifying the calculation logic of the joint training loss. During model training, the two cross-entropy losses can be directly weighted and summed to obtain the joint training loss, reducing the development complexity of the training framework and improving the stability and reproducibility of the training process. Cross-entropy loss has the characteristic of obvious gradient changes, which can quickly feed back the prediction error in the early stage of model training and drive the model parameters to update in the optimal direction. For dual-task models that share underlying parameters, the use of cross-entropy loss can effectively accelerate the overall convergence speed of the model, reduce the number of training iterations, and thus reduce the consumption of computing resources, making it more suitable for training scenarios with large-scale pre-sales customer service corpora.

[0021] In some alternative implementations, the acquisition of the first training data includes: Obtain the actual query text and corresponding business details; The actual query text, its corresponding business details, and the thought pattern judgment prompts are input into the second language model to obtain the thought pattern judgment results; the second language model has more parameters than the first language model. The actual query text, its corresponding business details, and the judgment results of the thinking patterns are used as the first training data.

[0022] In the above technical solution, the second language model has more parameters than the first, possessing stronger semantic understanding, logical reasoning, and scene judgment capabilities. Its judgment results on the thinking patterns of real query text and business details output can more accurately match the optimal decision logic—using a thought chain pattern for complex problems and a non-thought chain pattern for simple problems—and can be directly used as high-quality real labels for the first training data. Compared to the high cost and low efficiency of manual labeling, and the low accuracy of labels generated by models with small parameter counts, this embodiment significantly reduces the cost of acquiring training data while ensuring label quality.

[0023] In some optional implementations, the acquisition of the second training data includes: Obtain the actual query text and its corresponding business details and judgment results based on the thinking pattern; Determine the result based on the thinking pattern and identify the prompt words; The actual query text, its corresponding business details, and prompt words are input into the second language model to obtain the final answer; the second language model has more parameters than the first language model. The actual query text, its corresponding business details, and the final answer were used as the second training data.

[0024] In the above technical solution, the second language model with a larger number of parameters possesses stronger semantic understanding and reasoning integration capabilities. Its output answers accurately match the response needs in pre-sales customer service scenarios, ensuring consistency between the answers and product details while fully covering the core demands of user questions. It can serve as a highly reliable standard answer for RAG task training. Compared to the high cost and low efficiency of manual labeling, and the low accuracy of labels generated by models with a small number of parameters, this embodiment significantly reduces the cost of acquiring training data while ensuring label quality.

[0025] In some alternative implementations, the number of samples corresponding to the thinking pattern with a thought chain and the number of samples corresponding to the thinking pattern without a thought chain are the same in the first training data. In the second training data, the number of samples corresponding to the thinking pattern with a thought chain and the number of samples corresponding to the thinking pattern without a thought chain are the same.

[0026] In the above technical solution, the number of samples with and without thought chains in the first training data is kept consistent. This allows the model to receive two types of pattern feature signals of equal strength when learning the pattern selection rules. This avoids the problem that the model tends to select the pattern corresponding to high-frequency samples due to the high proportion of samples of a certain type of pattern. It ensures that the model can make accurate pattern judgments based on text features and business details when facing user queries of different complexities.

[0027] The balanced configuration of the number of samples for the two modes in the second training data enables the model to learn responses in both the mode with and without the thought chain to an equal extent during joint training. This avoids the problem of one mode performing well while the other performs poorly due to sample imbalance, ensuring high accuracy in complex problem scenarios and high efficiency in simple problem scenarios, and achieving a balanced and optimal performance of the two modes.

[0028] An electronic device provided in this application includes a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions, when executed by the processor, perform any of the methods described above.

[0029] This application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of any of the methods described above.

[0030] An electronic device provided in this application includes a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions, when executed by the processor, perform any of the methods described above.

[0031] This application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of any of the methods described above.

[0032] This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the retrieval enhancement generation method as described in any of the above claims. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart illustrating the steps of a retrieval enhancement generation method provided in this application embodiment; Figure 2 A flowchart illustrating a thinking pattern with a thought chain, provided for embodiments of this application; Figure 3 A flowchart of the training data synthesis steps provided in this application embodiment; Figure 4This is a schematic diagram of a possible structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0036] Please refer to Figure 1 , Figure 1 A flowchart of a retrieval enhancement generation method provided in this application embodiment includes: Step S1: Obtain the query text input by the user; Step S2: Based on the query text, retrieve the corresponding business details from the information source; Among them, business details information refers to documents or paragraphs retrieved from external databases (such as Wikipedia, professional documents, enterprise knowledge bases, etc.) that are related to the query text; there are two ways to retrieve business details information: The first approach involves pre-slicing and vectorizing the text resources, then semantically matching the query text with the vector library during retrieval to identify the most relevant texts, which are then reordered by a fine-ranking model and provided to the large language model for response. The second approach involves using external search tools (such as Bing and Google) to retrieve relevant text and providing it to a large language model for response.

[0037] Step S3: The first major language model uses thinking patterns to determine prompt words and performs the thinking pattern selection task: based on the query text and business details, select a thinking pattern with or without a thinking chain. Among them, the first major language model can be Qwen, LLaMA, etc.

[0038] Qwen is an open-source large language model series developed by Alibaba Cloud, covering versions with different parameter levels from 7B to 72B. It supports both Chinese and English and possesses excellent semantic understanding, logical reasoning, and instruction following capabilities. Based on the Transformer architecture, the model is trained using massive amounts of multi-source data and performs outstandingly in tasks such as dialogue generation, text summarization, and logical reasoning. Lightweight versions (such as Qwen-7B and Qwen-14B) are also available to support deployment on low-computing-power devices.

[0039] Qwen boasts the following advantages: strong instruction compliance, enabling precise execution of thinking mode selection tasks and phased RAG tasks, adapting to the multi-stage prompt-driven task flow in this application; good Chinese-English compatibility, directly applicable to Chinese query text processing in domestic pre-sales customer service scenarios without additional language adaptation fine-tuning; a robust open-source ecosystem, supporting lightweight fine-tuning based on users' own business data (such as pre-sales customer service business details and query text), and combined with the joint training mechanism of this application, can quickly optimize mode selection and RAG task performance; and low deployment cost for the lightweight version, suitable for the localized deployment needs of pre-sales customer service systems, meeting the efficiency requirements of real-time response.

[0040] LLaMA is an open-source large language model series released by Meta AI, covering parameter sizes from 7B to 65B. It is trained on pure English data, and its core advantages lie in its efficient Transformer architecture design and excellent few-shot learning capabilities. By optimizing the model structure and training strategies, the LLaMA series models consume less computational power with the same number of parameters. At the same time, it has also spawned many optimized Chinese branches (such as LLaMA-2-Chinese, ChatGLM, and other improved models based on the LLaMA architecture).

[0041] LLaMA has the following advantages: its model architecture is streamlined and efficient, with fast inference speed, which can adapt to the efficiency requirements of simple problems without thought chains and rapid response in this application, reducing the time spent on answer generation; the open source community is active, and the derived Chinese optimization branch solves the shortcomings of the original LLaMA in Chinese processing, which can be directly applied to Chinese pre-sales customer service scenarios; the parameter selection is flexible, from a lightweight version of 7B to a high-performance version of 65B, which can be flexibly selected according to different deployment computing power conditions and business needs (such as the inference accuracy requirements of complex problems).

[0042] Chain-of-Thought (COT) is a cue word technique in the field of large language models. Its core is to guide the model to break down the problem step by step and output the logical reasoning process, rather than directly giving the answer, thereby improving the model's reasoning accuracy in complex tasks.

[0043] Step S4: If the thinking mode with thought chain is selected, the first language model uses the first cue word with thought chain to execute the RAG task; if the thinking mode without thought chain is selected, the first language model uses the second cue word without thought chain to execute the RAG task; wherein, the RAG task is: to generate the final answer based on the query text, business details and cue words.

[0044] The first set of prompts can include thinking prompts, answer prompts, reflection prompts, and correction prompts. Thinking prompts instruct the model to generate a preliminary answer with a reasoning path. For example, if the query text is "Is this T-shirt hot?", the model, combining information such as the applicable season and fabric from the product details, provides the reasoning path: "From the product details, we know this T-shirt is suitable for summer, and the fabric is linen. Linen fabric is breathable; therefore, we can infer that this T-shirt will not be very hot." Answer prompts instruct the model to generate a preliminary answer based on the reasoning path. Reflection prompts instruct the model to check for logical conflicts between the preliminary answer and the product details. Correction prompts are used to regenerate the answer if logical conflicts exist.

[0045] In this embodiment, the application executes the thinking mode selection task through a first large language model. Based on the user-input query text and retrieved business details, it intelligently determines whether to enable the Thinking Chain (COT) reasoning mode. For complex query text, the model automatically switches to the RAG task driven by the first cue word with a thinking chain. By using a structured logical reasoning chain to break down the problem and integrate key content from the business details, it avoids logical loopholes or incomplete information issues caused by directly generating answers based on search results in existing technologies, significantly improving the accuracy and reliability of responses in complex scenarios. For simple query text, the model can automatically select the RAG task driven by the second cue word without a thinking chain, eliminating redundant COT reasoning steps and directly generating answers based on the retrieved business details. This effectively shortens the response time for simple questions, solves the inefficiency problem of existing technologies, ensures rapid response requirements in pre-sales intelligent customer service scenarios, and improves the smoothness and satisfaction of user interaction.

[0046] Please refer to Figure 2 , Figure 2 A flowchart illustrating a thinking pattern with a thought chain, provided for an embodiment of this application.

[0047] In some alternative implementations, the first prompt word instructs the first large language model to perform the following phased tasks: Perform the thinking phase task: Generate a reasoning path based on the query text and business details; Execute the answering phase task: Generate a preliminary answer based on the reasoning path; Perform the reflection phase task: verify the consistency between the preliminary answer and the detailed information. If the verification passes, output the preliminary answer as the final answer. If the validation fails, an error feedback prompt is generated, and a correction phase task is executed: the first language model regenerates the answer based on the error feedback prompt and outputs it as the final answer.

[0048] In this embodiment, the first prompt word breaks down the RAG task with a thought chain into four distinct stages: thinking, answering, reflecting, and correcting. The thinking stage generates a structured reasoning path based on the query text and product details, providing clear logical support for answer generation and avoiding information omissions or logical gaps during the reasoning process, making the answer more evidence-based. The reflecting stage verifies the matching degree between the preliminary answer and the product details, avoiding the problem of answers not matching product details due to model illusions or reasoning biases from the source, ensuring that the output answer is entirely based on official product details, and improving the authority and credibility of the answer in pre-sales customer service scenarios. For preliminary answers that fail verification, an error feedback prompt word triggers the correction stage, driving the model to re-optimize the answer based on the feedback. Error self-checking and correction can be achieved without manual intervention, significantly reducing the error rate of the final answer and improving the response quality in complex problem scenarios.

[0049] In some alternative implementations, verifying the consistency between the preliminary answer and the detailed information includes at least one of the following: attribute matching verification, logical verification, and integrity check.

[0050] Among them, attribute matching verification can avoid basic errors such as mismatched values ​​and parameters. For example, if the answer mentions "24-hour battery life", but the product details clearly state "4000mAh battery capacity, theoretical battery life of 12 hours", it will be marked as inconsistent.

[0051] Logical verification can avoid reasoning biases such as reversed cause and effect and logical conflicts. For example, if the product description states "material: pure cotton", but the answer infers "it is breathable because it contains chemical fibers", it will be marked as a logical conflict.

[0052] Completeness checks can prevent incomplete answers from missing core user needs, such as checking whether the answer omits the key points asked by the user.

[0053] In some alternative implementations, model training for the thinking pattern selection task and the RAG task includes: The model for performing the thinking mode selection task and the model for performing the RAG task share the underlying model parameters; The model performing the thinking mode selection task uses the first training data during training and calculates the first training loss; The model performing the RAG task uses the second training data during training and calculates the second training loss. Based on the first training loss and the second training loss, the joint training loss is obtained; By utilizing joint training loss for model training, unified training and learning can be achieved for both the thinking pattern selection task and the RAG task.

[0054] In some alternative implementations, the first training loss is:

[0055] The second training loss is:

[0056] The joint training loss is:

[0057] Where x represents the query text, k represents the business details, y1 represents the thinking mode selection result, and y2 represents the final answer.

[0058] In some alternative implementations, the first training loss and the second training loss employ cross-entropy.

[0059] In this embodiment, the thinking mode selection task is essentially a classification task (selecting a mode with / without a thinking chain). Cross-entropy loss can accurately measure the difference between the mode category predicted by the model and the true category, guiding the model to quickly learn the optimal thinking mode corresponding to different query texts and business details. The answer generation stage of the RAG task can be transformed into a sequence classification problem, and cross-entropy loss can also effectively evaluate the matching degree between the generated answer and the standard answer. The first training loss and the second training loss use the same cross-entropy calculation paradigm, eliminating the need to design differentiated loss transformation and weighting strategies for different tasks, greatly simplifying the calculation logic of the joint training loss. During model training, the two cross-entropy losses can be directly weighted and summed to obtain the joint training loss, reducing the development complexity of the training framework and improving the stability and reproducibility of the training process. Cross-entropy loss has the characteristic of significant gradient changes, which can quickly feed back the prediction error in the early stage of model training and drive the model parameters to update in the optimal direction. For dual-task models that share underlying parameters, the use of cross-entropy loss can effectively accelerate the overall convergence speed of the model, reduce the number of training iterations, and thus reduce the consumption of computing resources, making it more suitable for training scenarios with large-scale pre-sales customer service corpora.

[0060] Please refer to Figure 3 , Figure 3 A flowchart illustrating the training data synthesis steps provided in this application embodiment.

[0061] In some alternative implementations, the acquisition of the first training data includes: Obtain the actual query text and corresponding business details; The actual query text, its corresponding business details, and the thought pattern judgment prompts are input into the second language model to obtain the thought pattern judgment results; the second language model has more parameters than the first language model. The actual query text, its corresponding business details, and the judgment results of the thinking patterns are used as the first training data.

[0062] In this embodiment, the second language model has a larger number of parameters than the first language model, possessing stronger semantic understanding, logical reasoning, and scene judgment capabilities. Its judgment results on the thinking patterns of real query text and business details output can more accurately match the optimal decision logic for complex problems using a thought chain model and simple problems using a non-thought chain model, and can be directly used as high-quality real labels for the first training data. Compared to the high cost and low efficiency of manual labeling, and the low accuracy of labels generated by models with a small number of parameters, this embodiment significantly reduces the cost of acquiring training data while ensuring label quality.

[0063] In some optional implementations, the acquisition of the second training data includes: Obtain the actual query text and its corresponding business details and judgment results based on the thinking pattern; Determine the result based on the thinking pattern and identify the prompt words; The actual query text, its corresponding business details, and prompt words are input into the second language model to obtain the final answer; the second language model has more parameters than the first language model. The actual query text, its corresponding business details, and the final answer were used as the second training data.

[0064] In this embodiment, the second language model with a larger number of parameters possesses stronger semantic understanding and reasoning integration capabilities. Its output answer accurately matches the response needs in pre-sales customer service scenarios, ensuring consistency between the answer and product details while fully covering the core demands of user questions. It can serve as a highly reliable standard answer for RAG task training. Compared to the high cost and low efficiency of manual labeling, and the low accuracy of labels generated by models with a small number of parameters, this embodiment significantly reduces the cost of acquiring training data while ensuring label quality.

[0065] In some alternative implementations, the number of samples corresponding to the thinking pattern with a thought chain and the number of samples corresponding to the thinking pattern without a thought chain are the same in the first training data. In the second training data, the number of samples corresponding to the thinking pattern with a thought chain and the number of samples corresponding to the thinking pattern without a thought chain are the same.

[0066] In this embodiment of the application, the number of samples with and without thought chains in the first training data is kept consistent. This allows the model to receive two types of pattern feature signals of equal strength when learning the pattern selection rules. This avoids the problem that the model tends to select the pattern corresponding to high-frequency samples due to the high proportion of samples of a certain type of pattern. It ensures that the model can make accurate pattern judgments based on text features and business details when facing user queries of different complexities.

[0067] The balanced configuration of the number of samples for the two modes in the second training data enables the model to learn responses in both the mode with and without the thought chain to an equal extent during joint training. This avoids the problem of one mode performing well while the other performs poorly due to sample imbalance, ensuring high accuracy in complex problem scenarios and high efficiency in simple problem scenarios, and achieving a balanced and optimal performance of the two modes.

[0068] Figure 4 This illustration shows a possible structure of an electronic device provided in an embodiment of this application. (Refer to...) Figure 4 The electronic device includes a processor, memory, and a communication interface, which are interconnected and communicate with each other via a communication bus and / or other forms of connection mechanism (not shown).

[0069] The memory includes one or more (only one is shown in the figure), which can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor and other possible components can access the memory to read and / or write data to it.

[0070] The processor comprises one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Microcontroller Unit (MCU), a Network Processor (NP), or other conventional processors; it can also be a special-purpose processor, including a Neural-network Processing Unit (NPU), a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Furthermore, when there are multiple processors, some can be general-purpose processors, and others can be special-purpose processors.

[0071] The communication interface includes one or more (only one is shown in the figure), which can be used to communicate directly or indirectly with other devices to exchange data. The communication interface may include interfaces for wired and / or wireless communication.

[0072] One or more computer program instructions may be stored in the memory, and the processor may read and execute these computer program instructions to implement the methods provided in the embodiments of this application.

[0073] Understandable. Figure 4 The structure shown is for illustrative purposes only; the electronic device may also include structures that are more complex than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different structures shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof. Electronic devices may be physical devices, such as PCs, laptops, tablets, mobile phones, servers, embedded devices, etc., or they may be virtual devices, such as virtual machines, virtualized containers, etc. Furthermore, electronic devices are not limited to a single device; they can also be a combination of multiple devices or a cluster of a large number of devices.

[0074] This application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of any of the methods described above.

[0075] This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the retrieval enhancement generation method as described in any of the above claims.

[0076] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0077] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0078] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0079] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0080] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A retrieval enhancement generation method, characterized in that, include: Get the query text entered by the user; Based on the query text, retrieve the corresponding business details from the information source; The first major language model uses thinking patterns to determine prompt words and performs a thinking pattern selection task: based on the query text and business details, it selects a thinking pattern with or without a thinking chain. If the thought process with a thought chain is selected, the first major language model will use the first cue word with the thought chain to perform the RAG task. If the thought process without a thought chain is selected, the first major language model uses the second cue word without a thought chain to perform the RAG task. The RAG task is to generate the final answer based on the query text, business details, and hints.

2. The method as described in claim 1, characterized in that, The first cue word instructs the first major language model to perform the following phased tasks: Perform the thinking phase task: Generate a reasoning path based on the query text and business details; Perform the answering phase task: Generate a preliminary answer based on the reasoning path; Perform the reflection phase task: verify the consistency between the preliminary answer and the detailed information. If the verification passes, output the preliminary answer as the final answer. If the verification fails, an error feedback prompt is generated, and a correction phase task is executed: the first language model regenerates the answer based on the error feedback prompt and outputs it as the final answer.

3. The method as described in claim 2, characterized in that, The verification of the consistency between the preliminary answer and the detailed information includes at least one of the following: attribute matching verification, logical verification, and integrity check.

4. The method as described in claim 2, characterized in that, The model training for the thinking mode selection task and the RAG task includes: The model for performing the thinking mode selection task and the model for performing the RAG task share the underlying model parameters; The model performing the thinking mode selection task uses the first training data during training and calculates the first training loss; The model performing the RAG task uses the second training data during training and calculates the second training loss. Based on the first training loss and the second training loss, the joint training loss is obtained; The model is trained using the joint training loss to achieve unified training and learning for the thinking mode selection task and the RAG task.

5. The method as described in claim 4, characterized in that, The first training loss is: The second training loss is: The joint training loss is: Where x represents the query text, k represents the business details, y1 represents the thinking mode selection result, and y2 represents the final answer.

6. The method as described in claim 5, characterized in that, The first training loss and the second training loss use cross-entropy.

7. The method as described in claim 4, characterized in that, The acquisition of the first training data includes: Obtain the actual query text and corresponding business details; The actual query text, its corresponding business details, and the thought pattern judgment prompts are input into the second language model to obtain the thought pattern judgment results; the second language model has more parameters than the first language model. The actual query text, its corresponding business details, and the judgment results of the thinking patterns are used as the first training data.

8. The method as described in claim 7, characterized in that, The acquisition of the second training data includes: Obtain the actual query text and its corresponding business details and judgment results based on the thinking pattern; Determine the result based on the thinking pattern and identify the prompt words; The actual query text, its corresponding business details, and prompt words are input into the second language model to obtain the final answer; the second language model has more parameters than the first language model. The actual query text, its corresponding business details, and the final answer were used as the second training data.

9. The method as described in claim 1, characterized in that, In the first training data, the number of samples corresponding to the thinking pattern with the thought chain is the same as the number of samples corresponding to the thinking pattern without the thought chain. In the second training data, the number of samples corresponding to the thinking pattern with a thought chain and the number of samples corresponding to the thinking pattern without a thought chain are the same.

10. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when executed by the processor, perform the method as described in any one of claims 1-9.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the retrieval enhancement generation method as described in any one of claims 1 to 9.

Images