An integrated retrieval enhancement method of a large language model, an electronic device, and a storage medium

Abstract

The application belongs to the technical field of natural language processing, and specifically designs an integrated retrieval enhancement method of a large language model, an electronic device and a storage medium. The integrated retrieval enhancement method specifically is that a controller calls multiple retrievers according to user input, returns multiple-source multiple-document, and integrates and disturbs different documents into multiple knowledge sections through multiple processing modes, and respectively inputs a generation module. After the generation module returns multiple replies, the control module selects the optimal reply through the consistency between the replies and an objective scorer. The optimal parameter search is converted into a non-derivable optimization problem of a target function, the optimal retrieval enhancement configuration can be found through a meta-heuristic search algorithm, and the ability of the large language model on factual problems is effectively improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of natural language processing, and specifically designs an integrated retrieval enhancement method of a large language model, an electronic device and a storage medium. BACKGROUND

[0002] Large language models have shown amazing capabilities on traditional natural language processing tasks, but still face the challenges of generating false facts and failing to update real-time knowledge. Retrieval enhancement technology can effectively enhance the authenticity and timeliness of the reply by retrieving external knowledge sources as additional inputs for the model. However, blind retrieval can also cause problems. When the document is too long or too many, it will be difficult for the model to effectively use the information in the retrieved document. Moreover, using unfiltered documents may contain false and harmful information, which will affect the model's reply. To address the above problems, there are currently two mainstream strategies: learn to search and rejection sampling at the generation end. Learn to search requires the model to pre-judge whether to retrieve external knowledge. Sparrow proposed by DeepMind in 2022 and the retrieval-enhanced large model UNIWEB proposed by Renmin University of China in 2023 both use this strategy and have achieved good results. Rejection sampling is a strategy in which the model samples a large number of candidate replies based on external knowledge, then ranks all the replies and selects the best one as the final reply and outputs it. LaMDA2 proposed by Google in 2022 and GopherCite model (280B) proposed by DeepMind in 2022 also use this strategy to achieve significant improvement.

[0003] Both of these strategies have their drawbacks. First, learn to search can allow the model to choose not to use the retrieved knowledge, but learn to search requires the model to know what they don't know, i.e., the model needs to be able to judge in advance whether it can correctly answer the question. This is a very difficult problem, even for ChatGPT. On the contrary, rejection sampling at the generation end starts from the generated result and uses posterior evaluation to filter out problematic answers. This can solve the problem of using document knowledge that is not stable for the model, but it still cannot solve the problem of the quality of the document itself. Sampling replies based on a false document is likely to be a false reply.

[0004] Therefore, a rejection sampling method based on retrieval-side sampling and generation-side reordering is proposed. Starting from the generated results, the question of whether to retrieve a document is directly transformed into an evaluation problem. By assessing the quality of the enhanced response, the decision-making process of whether to retrieve a document is linked to the final generated result. This allows the system to dynamically decide whether to retrieve a document or which document to trust more, theoretically ensuring that this decision-making process provides positive optimization for the generation process. Furthermore, this proposed method can provide a more comprehensive evaluation of the model's response in a more systematic way, aligning the generated results with human expectations. Additionally, the evaluation module in the system can be used to evaluate the model's ability to further improve the generative model through reinforcement learning, or as a filtering module to automatically construct data and improve the generative model's capabilities through supervised learning. Summary of the Invention

[0005] This invention provides an integrated retrieval enhancement method for large language models, which transforms the search for optimal parameters into an optimization problem with a non-differentiable objective function. The optimal retrieval enhancement configuration can be found through metaheuristic search algorithms (such as the Nelder-Mead Method), effectively improving the ability of large language models to handle factual questions.

[0006] The present invention also provides an electronic device.

[0007] The present invention also provides a computer-readable storage medium.

[0008] This invention is achieved through the following technical solution:

[0009] An ensemble retrieval enhancement method for a large language model is disclosed. Specifically, the controller invokes multiple retrieval tools based on user input, returning multiple documents from multiple sources. These documents are then integrated and perturbed into multiple knowledge segments through various processing methods, which are then input into a generation module. After the generation module returns multiple responses, the control module selects the optimal response based on the consistency between the responses and an objective scorer.

[0010] Furthermore, the controller is used to control the overall rejection sampling process; the rejection sampling method is divided into: a sampling phase, a voting phase, and an evaluation phase.

[0011] Furthermore, the sampling stage specifically involves the control module receiving user input, repeatedly calling the retrieval module and the generation module to construct different prompts based on the user input, and then sequentially sending the constructed prompts into the generation module to generate response candidates.

[0012] Furthermore, methods for constructing different prompts include, but are not limited to:

[0013] Yes / No concatenation of search documents;

[0014] Or retrieve different documents from different knowledge sources;

[0015] Or process the retrieved documents.

[0016] Furthermore, the voting stage specifically involves judging the consistency among candidate responses, filtering out responses inconsistent with the majority, and sending the remaining responses to the evaluation stage. Given responses generated based on N different sampling methods, each response can obtain a similarity score with another N-1 responses. Through pooling, the N-1 scores are integrated into a final score to obtain a comprehensive similarity score for each response. This comprehensive similarity score is then multiplied by the prior weight of the sampling method to obtain the voter score for the responses generated by that sampling method. The specific formula is as follows:

[0017]

[0018] Among them, y m ,y n These represent the responses generated by the m-th and n-th sampling methods, respectively. Sim is the semantic similarity evaluation function, and θ... s The evaluation function has built-in parameters; Pooling is the pooling function, ω m t represents the prior weight of the m-th method; t is the voting module threshold, and all responses below the threshold will be deleted.

[0019] Furthermore, the evaluation stage specifically involves using a trained objective scorer to independently evaluate the quality of each candidate response and select the response that best meets human expectations. The evaluation stage is combined with the voting stage, and the response with the highest combined score from the last two modules is the final response.

[0020] Furthermore, the final response specifically states that the final scoring formula is:

[0021]

[0022] in For the scorer's output results, For scoring integration functions;

[0023] Model final response This is the highest-rated reply:

[0024] m * =argmax m S * (y m |y n≠m ;ω m ,θ s ,θ r ).

[0025] Furthermore, the parameter search for rejection sampling specifically involves finding the highest-rated response with ω. m θ s θ r All parameters are trainable, given a training set. Where q represents user input and y represents the standard answer; by fixing the retrieval module, generation module, and sampling module, the response corresponding to each sampling scheme can be obtained, and the parameters are constructed to search the training set accordingly.

[0026] Model the parameter search as a simple optimization problem:

[0027]

[0028] The function g(·,·) is an evaluation function, which can be an automatic indicator or a manual evaluation.

[0029] Clearly, the objective function is neither differentiable nor continuous. Therefore, either a metaheuristic search algorithm can be used, or the model can automatically learn the optimal sampling method by adding a regularization term or removing samples with excessively low weights.

[0030] An electronic device, characterized in that it includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0031] Memory, used to store computer programs;

[0032] When a processor executes a program stored in memory, it implements the above method steps.

[0033] A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-described method steps.

[0034] The beneficial effects of this invention are:

[0035] 1. This invention can greatly improve the accuracy and factuality of large language model generation, making it more reliable for application in downstream fields.

[0036] 2. Indirectly improve the accuracy of retrieval and the stability of knowledge utilization of the retrieval enhancement model, enabling the model to dynamically determine which knowledge to use.

[0037] 3. Improve the interpretability of the large language model generation, making its generation verifiable and facilitating user verification of the accuracy of responses, thereby enhancing credibility. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the method of the present invention.

[0039] Figure 2 This is a flowchart of the rejection sampling method of the present invention. Detailed Implementation

[0040] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0041] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0042] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0043] The following is in conjunction with the appendix to this application specification. Figures 1-2 The technical solutions in the embodiments of this application are clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0044] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0045] Example 1

[0046] Mainstream retrieval enhancement models consist of two modules: a retrieval module and a generation module. The retrieval module takes the user's question as input and outputs the retrieved documents; the generation module takes the retrieved documents and the user's question as input and generates the final answer. The algorithm proposes a third module: a control module, such as... Figure 1Based on user input, the controller invokes multiple retrieval tools, returning multiple documents from multiple sources. It then integrates and perturbs these documents into multiple knowledge segments using various processing methods, which are then input into the generation module. After the generation module returns multiple responses, the control module selects the optimal response based on consistency among the responses (voting mechanism) and an objective scorer.

[0047] This embodiment provides an integrated retrieval enhancement method for a large language model. Specifically, the controller invokes multiple retrieval tools based on user input, returning multiple documents from multiple sources. These documents are then integrated and perturbed into multiple knowledge segments through various processing methods, which are then input into a generation module. After the generation module returns multiple responses, the control module selects the optimal response based on consistency among the responses (voting) and an objective scorer. The control module incorporates multiple parameters, such as the weights of different document processing methods, the weights of the voting tool, and the weights of the scorer. Given a training set, the search for optimal parameters is transformed into an optimization problem with a non-differentiable objective function. The optimal retrieval enhancement configuration can be found using metaheuristic search algorithms (such as the Nelder-Mead Method), effectively improving the large language model's ability to handle factual questions.

[0048] To improve the coverage of correct answers, initial documents are obtained from three different retrieval sources, categorized into three types: search engines, document repositories, and knowledge from the large model's internal structure. Search engines utilize user input, directly accessing search engine APIs (such as Bing and Baidu), and then cleaning the returned documents. For document repositories (such as a processed Wikipedia document repository), a dense retrieval algorithm is used to encode both user input and the document repository into dense vectors, and then similarity matching is used to obtain the documents. Knowledge from the large model's internal structure involves directly instructing the model to generate documents that can respond to user input.

[0049] The generation module can utilize any model with good generalization ability and employ context learning methods to enable the model to generate responses to a given document.

[0050] Furthermore, the controller is used to control the overall rejection sampling process; the rejection sampling method is divided into three stages: sampling stage, voting stage, and evaluation stage.

[0051] Furthermore, the sampling stage specifically involves the control module receiving user input, repeatedly calling the retrieval module and the generation module to construct different prompts (model inputs) based on the user input (query), and then sequentially sending the constructed prompts into the generation module to generate response candidates.

[0052] Furthermore, methods for constructing different prompts include, but are not limited to:

[0053] Yes / No concatenation of search documents;

[0054] Or retrieve different documents from different knowledge sources;

[0055] Or process the retrieved documents (such as segmenting, splicing, rearranging, summarizing, etc.).

[0056] The diverse prompts provide the generation module with different options for utilizing the retrieved documents, facilitating the selection of the most suitable retrieval augmentation method using the rejection approach. If the final model chooses not to use the responses obtained from the retrieved documents, it is equivalent to the model indirectly choosing not to use retrieval augmentation. Therefore, the framework also allows the model to dynamically choose whether to utilize knowledge.

[0057] Furthermore, the voting stage specifically involves filtering out responses inconsistent with the majority by judging the consistency among candidate responses, and sending the remaining responses to the evaluation stage. This stage is based on the assumption that current large models can effectively utilize input information and rarely exhibit inconsistencies with the input document information. Therefore, if the responses after retrieval enhancement are consistent with each other, they are likely to be genuine. It is believed that this method can effectively reduce the fabrication of responses. If the consistency among responses is not high, it can be assumed that the model lacks confidence in answering the question, thus causing the model to refuse to answer the question, which serves a similar purpose as confidence calibration.

[0058] Given two model responses generated using different sampling methods, semantic similarity metrics (such as BertScore, natural language inference models, etc.) can be used to obtain similarity scores between the two responses. Given responses generated using N different sampling methods, each response can obtain similarity scores with another N-1 responses. Pooling operations (such as averaging) are then used to integrate the N-1 scores into a final score, obtaining a comprehensive similarity score for each response. Multiplying this comprehensive similarity score by the prior weight of the sampling method yields the voting score for the responses generated by that sampling method. The specific formula is as follows:

[0059]

[0060] Among them, y m ,y n These represent the responses generated by the m-th and n-th sampling methods, respectively. Sim is the semantic similarity evaluation function, and θ... s The evaluation function has built-in parameters; Pooling is the pooling function, ω m t represents the prior weight of the m-th method; t is the voting module threshold, and all responses below the threshold will be deleted.

[0061] Furthermore, the evaluation stage specifically involves using a trained objective scorer to independently evaluate the quality of each candidate response and select the response that best meets human expectations. The evaluation can be approached from multiple perspectives, constructing multiple evaluation models. The evaluation aspects that can be considered include, but are not limited to, authenticity, safety, helpfulness, and human preferences. The evaluation stage can also be combined with the voting stage, and the response with the highest combined score from the last two modules is the final response. Currently, a publicly available human preference dataset is used to train the scorer to score the coherence, usefulness, and factuality of the responses.

[0062] Furthermore, the final response specifically states that the final scoring formula is:

[0063]

[0064] in For the scorer's output results, For scoring integration functions;

[0065] Model final response This is the highest-rated reply:

[0066] m * =argmax m S * (y m |y n≠m ;ω m ,θ s ,θ r ).

[0067] Furthermore, the parameter search for rejection sampling specifically involves finding the highest-rated response with ω. m θ s θ r All parameters are trainable, given a training set. Where q is the user input and y is the standard answer; by fixing the retrieval module, the generation module and the sampling module, the response corresponding to each sampling scheme can be obtained, and the parameters can be constructed to search the training set accordingly;

[0068] The parameter search can be modeled as a simple optimization problem:

[0069]

[0070] The function g(·,·) is an evaluation function, which can be an automatic indicator, such as EM, or a manual evaluation.

[0071] Clearly, the objective function is neither differentiable nor continuous. Therefore, we can either use metaheuristic search algorithms, such as the Nelder-Mead algorithm, to find the optimal parameter configuration, or we can add regularization terms or remove samples with excessively low weights to allow the model to automatically learn the optimal sampling method.

[0072] Specifically, this invention conducts experiments on the open-domain question-answering datasets Natural Question (NQ), WebQA, and TriviaQA.

[0073] The retrieval module of this invention uses Google, Wikipedia (December 2018), and GenRead as internal knowledge sources for the model, and combines 16 different document sources using methods such as segmentation, truncation, concatenation, rearrangement, and summarization. The generation module of this invention uses ChatGPT as the basic model.

[0074] In the control module, this invention uses BertScore, Exact Match, and the weights of the natural language inference model as the similarity evaluation function, Sim. An average pooling function is used as the pooling function, and the voting module threshold is set to 0. Since the experimental dataset consists of short responses, a scoring module is not used; the score integration function outputs... This is the final output of the model. Table 1 shows the final results of the response accuracy:

[0075]

[0076] Table 1 shows the accuracy of responses. Optimal single-source retrieval refers to the optimal result using only one knowledge source.

[0077] Example 2

[0078] This invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. The memory stores software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory and processor are connected via a bus. Specifically, the processor implements any of the steps in Embodiment 1 by running the computer program stored in the memory.

[0079] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0080] Memory may include read-only memory, flash memory, and random access memory, and provides instructions and data to the processor. Some or all of the memory may also include non-volatile random access memory.

[0081] As can be seen from the above, the electronic device provided in this embodiment of the invention can implement the semi-supervised image classification method as described in Embodiment 1 by running a computer program. This embodiment of the invention, through a novel semi-supervised learning framework combined with metric learning methods, aims to strengthen inter-class differences while considering intra-class differences. This framework can better utilize intra-class information and improve model performance. Experimental results show that this method achieves significant improvements in classification performance compared to existing semi-supervised learning methods.

[0082] It should be understood that if the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0083] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0084] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0085] It should be noted that the methods and detailed examples provided in the above embodiments can be incorporated into the apparatus and devices provided in the embodiments, and can be referred to each other, without further elaboration.

[0086] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0087] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units described above is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0088] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An integrated retrieval enhancement method for a large language model, characterized in that, The integrated retrieval enhancement method specifically involves the controller calling multiple retrieval tools based on user input, returning multiple documents from multiple sources, and integrating and perturbing the different documents into multiple knowledge segments through various processing methods, which are then input into the generation module. After the generation module returns multiple responses, the control module selects the optimal response based on the consistency between the responses and an objective scorer. The controller is used to control the overall rejection sampling process; the rejection sampling method is divided into: a sampling phase, a voting phase, and an evaluation phase; The voting stage specifically involves judging the consistency among candidate responses, filtering out responses inconsistent with the majority, and sending the remaining responses to the evaluation stage. Given responses generated based on N different sampling methods, each response receives a similarity score with another N-1 responses. Through pooling, these N-1 scores are combined into a final score, yielding a comprehensive similarity score for each response. This comprehensive similarity score is then multiplied by the prior weight of the sampling method to obtain the voter score for the responses generated by that sampling method. The specific formula is as follows: Among them, y m ,y n These represent the responses generated by the m-th and n-th sampling methods, respectively. Sim is the semantic similarity evaluation function, and θ... s The evaluation function has built-in parameters; Pooling is the pooling function, ω m t represents the prior weight of the m-th method; t is the voting module threshold, and all responses below the threshold will be deleted.

2. The integrated retrieval enhancement method according to claim 1, characterized in that, The sampling stage specifically involves the control module receiving user input, repeatedly calling the retrieval module and the generation module to construct different prompts based on the user input, and then sequentially sending the constructed prompts into the generation module to generate response candidates.

3. The integrated retrieval enhancement method according to claim 2, characterized in that, Methods for constructing different prompts include, but are not limited to: Yes / No concatenation of search documents; Or retrieve different documents from different knowledge sources; Or process the retrieved documents.

4. The integrated retrieval enhancement method according to claim 1, characterized in that, The evaluation phase specifically involves using a trained objective scorer to independently evaluate the quality of each candidate response and select the response that best meets human expectations. The evaluation phase is combined with the voting phase, and the response with the highest combined score from the last two modules is the final response.

5. The integrated retrieval enhancement method according to claim 4, characterized in that, The final response specifically states that the final scoring formula is: in For the scorer's output results, For the scoring integration function, ω m ,θ s ,θ r All are trainable parameters, and ω m Let θ be the prior weight of the m-th method. s These are built-in parameters for the evaluation function; Model final response This is the highest-rated reply: The parameter search for rejecting sampling is specifically the search for ω in the highest-rated response. m θ s θ r All parameters are trainable, given a training set. Where q i For user input, For q i The standard answer; with fixed retrieval, generation, and sampling modules, it can obtain the response corresponding to each sampling scheme, and construct parameters to search the training set accordingly; 6. The integrated retrieval enhancement method according to claim 5, characterized in that, Model the parameter search as a simple optimization problem: The function g(·,·) is an evaluation function, which can be an automatic indicator or a manual evaluation. This is the model's final response; Clearly, the objective function is neither differentiable nor continuous. Therefore, a metaheuristic search algorithm can be used, or the model can automatically learn the optimal sampling method by adding regularization terms or removing samples with excessively low weights.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

Images