A large language model retrieval enhancement generation method and system

Abstract

The present invention discloses a large language model retrieval enhancement generation method and system, which obtains a first proxy model, a second proxy model and a third proxy model according to a large language model; inputs a user question into the first proxy model to obtain a preliminary answer; determines whether the user question and the preliminary answer match through the second proxy model, and if so, directly inputs the user question into the large language model to generate a final answer; if not, rewrites the user question and the preliminary answer through the third proxy model to obtain a first rewritten question-answer pair; determines whether each question-answer pair in the first rewritten question-answer pair matches through the second proxy model to obtain a second matching result, retrieves a retrieval document according to the second matching result; inputs the retrieval document and the user question into the large language model to obtain a final answer. By adopting the embodiment of the present invention, intermediate process tasks can be quickly executed, response time can be reduced, and throughput can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of retrieval enhancement generation, and in particular to a large language model retrieval enhancement generation method and system. Background Art

[0002] At present, the classic way to combine large language models with various industries is to combine them with the knowledge bases of various industries, that is, the search enhancement generation system in professional fields. This system searches the knowledge base through the user's questions, and uses the user's questions and search results to generate the documents required by the user, such as answers to industry professional knowledge, industry knowledge summaries, and industry reports. This system enables the large language model to accept knowledge that it did not have before, thereby enhancing the quality of generated documents and reducing the hallucination problem of the large language model.

[0003] However, current methods are mainly based on large language models with large parameter scales. Some methods use multiple large language models to construct operation processes based on the large language model, which makes the response time of the entire system longer and affects the user experience. Summary of the invention

[0004] The technical problem to be solved by the present invention is how to quickly perform reasoning on a large language model, and to provide a large language model retrieval enhancement generation method and system, which can quickly execute intermediate process tasks through a large language proxy model, reduce response time, and improve throughput.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a large language model retrieval enhancement generation method, including:

[0006] According to the large language model used to answer questions, a large language proxy model is obtained; the architecture of the large language proxy model is the same as that of the large language model, but the number of parameters is smaller than that of the large language model; the large language proxy model includes a first proxy model, a second proxy model and a third proxy model;

[0007] Obtaining a user question, inputting the user question into the first agent model, and obtaining a preliminary answer;

[0008] Determining whether the user's question and the preliminary answer match by using the second proxy model, and obtaining a first matching result;

[0009] If the first matching result is a match, directly inputting the user's question into the large language model to generate a final answer;

[0010] If the first matching result is no match, rewriting the user question and the preliminary answer by using the third proxy model to obtain a first rewritten question-answer pair;

[0011] Determining whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model to obtain a second matching result, and retrieving a search document from the industry knowledge base according to the second matching result;

[0012] The retrieved document and the user question are input into the large language model to obtain a final answer.

[0013] As an improvement of the above solution, before determining whether the user question and the preliminary answer match by the second proxy model to obtain the first matching result, the search enhancement generation method further includes:

[0014] Obtaining a labeled question-answer pair from an industry knowledge base; the labeled question-answer pair includes a labeled question and a labeled answer;

[0015] Inputting the label question into the first agent model to obtain a preliminary label answer;

[0016] Calculating the correlation between the preliminary label answer and the label answer;

[0017] According to the correlation, parameters of the second proxy model are fine-tuned.

[0018] As an improvement of the above solution, the calculating the correlation between the preliminary label answer and the label answer includes:

[0019] pass Calculating the correlation between the preliminary label answer and the label answer;

[0020] Among them, a p,i The preliminary labeling response described in paragraph i; a label,i is the label answer of the i-th paragraph; precision is the accuracy of the j-gram in the preliminary label answer; N is a preset constant.

[0021] As an improvement of the above solution, fine-tuning the parameters of the second proxy model according to the correlation includes:

[0022] According to the relationship between the correlation and a preset threshold, the ability of the first agent model to answer the label question is obtained;

[0023] Constructing fine-tuning data for a second agent model according to the ability of the first agent model to answer the label question;

[0024] According to the fine-tuning data, a cross entropy loss function is used to fine-tune the parameters of the second proxy model.

[0025] As an improvement of the above solution, according to the fine-tuning data, the parameters of the second proxy model are fine-tuned using a cross entropy loss function, including:

[0026] Acquire an input part and an output part from the fine-tuning data; wherein the input part is the user's question and the corresponding preliminary label answer, and the output part is the ability of the first agent model to answer the label question;

[0027] pass For the second proxy model P J,Φ Fine-tune the parameters of J is the input part; y J is the output part; t is the data identifier currently involved in the calculation.

[0028] As an improvement of the above solution, if the first matching result is a mismatch, rewriting the user question and the preliminary answer by using the third proxy model to obtain a rewritten question-answer pair, the retrieval enhancement generation method further includes:

[0029] Inputting the user question and the preliminary answer into the large language model to obtain a second rewritten question-answer pair;

[0030] Based on the second rewritten question-answer pair, parameters of the third proxy model are fine-tuned.

[0031] As an improvement of the above solution, fine-tuning the parameters of the third proxy model according to the second question-answer pair includes:

[0032] pass For the third proxy model P R,Φ Fine-tune the parameters of G Asking questions to the user and providing the corresponding preliminary answers; G Rewrite the question-answer pair for the second one; t is the data identifier currently involved in the calculation.

[0033] As an improvement of the above solution, if the first matching result is a mismatch, rewriting the user question and the preliminary answer by using the third proxy model to obtain a first rewritten question-answer pair includes:

[0034] If the first matching result is no match, rewriting the user question by using the third proxy model to obtain several detailed user questions for details;

[0035] Rewriting the preliminary answer through the third agent model to obtain several preliminary detailed answers for the details, wherein the detailed user questions correspond to the preliminary detailed answers one by one;

[0036] A first rewritten question-answer pair is obtained based on the detailed user question and the detailed preliminary answer.

[0037] As an improvement of the above solution, the second proxy model is used to determine whether each question-answer pair in the first rewritten question-answer pair matches, to obtain a second matching result, and the search document is retrieved from the industry knowledge base according to the second matching result, including:

[0038] Determine whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model, and obtain a second matching result;

[0039] Filtering the question-answer pairs whose second matching results are unmatched in the first rewritten question-answer pairs as unknown question-answer pairs;

[0040] According to the unknown question-answer pair, a search is performed in the industry knowledge base to obtain a search document.

[0041] The embodiment of the present invention also provides a large language model retrieval enhancement generation system, including:

[0042] A proxy model acquisition module, used to obtain a large language proxy model according to the large language model used to answer questions; the architecture of the large language proxy model is the same as that of the large language model, but the number of parameters is smaller than that of the large language model; the large language proxy model includes a first proxy model, a second proxy model and a third proxy model;

[0043] A user question module, used for obtaining user questions, inputting the user questions into the first agent model, and obtaining preliminary answers;

[0044] A first matching result generating module, configured to determine whether the user's question and the preliminary answer match by using the second proxy model, and obtain a first matching result;

[0045] a first final answer generating module, configured to directly input the user's question into the large language model to generate a final answer if the first matching result is a match;

[0046] a first rewritten question-answer pair generating module, configured to rewrite the user question and the preliminary answer by using the third proxy model to obtain a first rewritten question-answer pair if the first matching result is a mismatch;

[0047] A knowledge base retrieval module, used to determine whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model, obtain a second matching result, and retrieve a retrieval document from the industry knowledge base according to the second matching result;

[0048] The second final answer generating module is used to input the search document and the user question into the large language model to obtain a final answer.

[0049] Compared with the prior art, the present invention discloses a large language model retrieval enhancement generation method and system, which obtains a large language proxy model according to a large language model used to answer questions; the architecture of the large language proxy model is the same as that of the large language model, but the number of parameters is smaller than that of the large language model; the large language proxy model includes a first proxy model, a second proxy model and a third proxy model; a user question is obtained, and the user question is input into the first proxy model to obtain a preliminary answer; the second proxy model is used to determine whether the user question and the preliminary answer match to obtain a first matching result; if the first matching result is a match, the user question is directly input into the large language model to generate a final answer; if the first matching result is a mismatch, the user question and the preliminary answer are rewritten by the third proxy model to obtain a first rewritten question-answer pair; the second proxy model is used to determine whether each pair of question-answer pairs in the first rewritten question-answer pair matches to obtain a second matching result, and a retrieval document is retrieved from an industry knowledge base according to the second matching result; the retrieval document and the user question are input into the large language model to obtain a final answer. By adopting the embodiments of the present invention, intermediate process tasks can be quickly executed through a large language proxy model, thereby reducing response time and improving throughput. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the steps of a large language model retrieval enhancement generation method provided by an embodiment of the present invention;

[0051] Figure 2 It is a structural diagram of a large language model retrieval enhancement generation system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In the description of the specification and claims, it is to be understood that the terms first, second, etc. in the specification and claims are only used for the purpose of describing the same technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor necessarily describing the order or time sequence. The terms are interchangeable where appropriate. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0054] As the parameters of large language models become larger and larger, and their capabilities become stronger and stronger, large language models are gradually being widely used in various industries. However, it is still challenging for large language models to directly generate documents required by users. Large language models are generally trained based on general data, which lacks industry knowledge and will generate documents with hallucinations when facing industry-specific questions. Secondly, the huge parameter scale of large language models makes it difficult to reason about large language models, affecting user experience.

[0055] The existing technology searches the existing knowledge base based on the input and output of the large language model to achieve retrieval enhancement of the large language model. However, the existing technology has at least the following technical problems: the large language model has a huge number of parameters, and directly building the operation process on the large language model makes the response time of the entire system longer, which affects the user experience.

[0056] Based on the above considerations, an embodiment of the present invention provides a large language model retrieval enhancement generation method. Figure 1 In this embodiment, the large language model retrieval enhancement generation method is specifically performed through steps S1 to S7:

[0057] S1. According to the large language model used to answer questions, a large language proxy model is obtained; the architecture of the large language proxy model is the same as that of the large language model, but the number of parameters is smaller than that of the large language model; the large language proxy model includes a first proxy model, a second proxy model and a third proxy model.

[0058] It should be noted that compared with the large language model, the large language proxy model has the same structure but smaller parameters. It has similar language capabilities to the large language model and can understand simpler tasks in the system. In addition, the language model with smaller parameters can also quickly execute intermediate process tasks, thereby reducing reasoning time, reducing system response time, and improving system throughput.

[0059] It should also be noted that, in the embodiment of the present invention, the first agent model is used to generate a preliminary answer, the second agent model determines the matching result of the preliminary answer, and the third agent model is used to rewrite the user's question and the preliminary answer.

[0060] S2. Obtain user questions, input the user questions into the first agent model, and obtain preliminary answers.

[0061] S3. Determine whether the user's question and the preliminary answer match by using the second agent model to obtain a first matching result.

[0062] S4. If the first matching result is a match, directly input the user question into the large language model to generate a final answer.

[0063] It is understandable that if the preliminary answer generated by the first proxy model matches the user's question, it means that the first proxy model has the ability to answer the user's question. The parameters of the first proxy model are smaller than those of the large language model. If the first proxy model can answer the user's question, it can be directly considered that the large language model also has the ability to answer the user's question. Therefore, the user's question is directly input into the large language model to generate the final answer without searching in the industry knowledge base.

[0064] S5. If the first matching result is a mismatch, rewrite the user question and the preliminary answer using the third proxy model to obtain a first rewritten question-answer pair.

[0065] S6. Determine whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model to obtain a second matching result, and retrieve a retrieval document from the industry knowledge base based on the second matching result.

[0066] S7: input the search document and the user question into the large language model to obtain a final answer.

[0067] It should be noted that the industry knowledge base is usually a description of details. If the search is performed directly based on the user question and the preliminary answer, it is difficult to mine effective content. In the embodiment of the present invention, the user question and the preliminary answer are rewritten by the third proxy model, and then the first rewritten question-answer pair is judged and searched, so that the original expression can be used to ask and search for more detailed questions, enriching the search results.

[0068] For example, the parameters of the large language model G are 72B, and the first proxy model G P The parameter of the second proxy model is 3B, J and the third proxy model G R The parameter of is 1.5B. The specific parameter amounts of the large language model and the parameter amounts of the large language proxy model are only taken as examples, and the parameters need to be selected according to the actual application situation. The specific parameter values ​​do not affect the beneficial effects achieved by the present invention.

[0069] As a preferred implementation, before step S3, the retrieval enhancement generation method further includes:

[0070] Obtaining a labeled question-answer pair from an industry knowledge base; the labeled question-answer pair includes a labeled question and a labeled answer;

[0071] Inputting the label question into the first agent model to obtain a preliminary label answer;

[0072] Calculating the correlation between the preliminary label answer and the label answer;

[0073] According to the correlation, parameters of the second proxy model are fine-tuned.

[0074] It should be noted that the initial judgment model does not have the judgment capability of utilizing the knowledge in the industry knowledge base, and therefore the second agent model is fine-tuned based on the industry knowledge base.

[0075] For example, for the industry knowledge base D, there are labeled question and answer data pairs (q lable,i ,a label,i ), the first agent model generates a preliminary answer based on the question P,i =G P (q lable,i ). The correlation between the preliminary label answer and the label answer is further calculated by the BLUE formula:

[0076]

[0077] Among them, a p,i The preliminary labeling response described in paragraph i; a label,i is the label answer of the i-th paragraph; precision is the accuracy of the j-gram in the preliminary label answer; N is a preset constant.

[0078] Preferably, in some embodiments, fine-tuning the parameters of the second proxy model according to the correlation comprises:

[0079] According to the relationship between the correlation and a preset threshold, the ability of the first agent model to answer the label question is obtained;

[0080] Constructing fine-tuning data for a second agent model according to the ability of the first agent model to answer the label question;

[0081] According to the fine-tuning data, a cross entropy loss function is used to fine-tune the parameters of the second proxy model.

[0082] For example, given a threshold θ, when the correlation ri >θ, indicating that the first agent model has the label question q lable,i knowledge; when the correlation r i <θ, indicating that the first agent model does not have the label question q lable,i knowledge.

[0083] Furthermore, preferably, the fine-tuning the parameters of the second proxy model using a cross entropy loss function according to the fine-tuning data includes:

[0084] Acquire an input part and an output part from the fine-tuning data; wherein the input part is the user's question and the corresponding preliminary label answer, and the output part is the ability of the first agent model to answer the label question;

[0085] For the second proxy model P J,Φ Fine-tune the parameters of , specifically by:

[0086]

[0087] Among them, x J is the input part; y J is the output part; t is the data identifier currently involved in the calculation.

[0088] In an embodiment of the present invention, an initial answer is generated according to a user's question by a first proxy model, and the correlation is calculated using the initial answer and a standard answer, i.e., a labeled answer, and then fine-tuning data for a second proxy model is constructed by setting a threshold.

[0089] As a preferred implementation, before step S5, the retrieval enhancement generation method further includes:

[0090] Inputting the user question and the preliminary answer into the large language model to obtain a second rewritten question-answer pair;

[0091] Based on the second rewritten question-answer pair, parameters of the third proxy model are fine-tuned.

[0092] Further, preferably, the fine-tuning of the parameters of the third proxy model according to the second question-answer pair includes:

[0093] pass For the third proxy model P R,Φ Fine-tune the parameters of G Asking questions to the user and providing the corresponding preliminary answers; G Rewrite the question-answer pair for the second one; t is the data identifier currently involved in the calculation.

[0094] In the above scheme, the user question and the corresponding preliminary answer are used as the input part of the fine-tuning data of the third proxy model, the second rewritten question and answer pair is used as the output part of the fine-tuning data of the third proxy model, and the third proxy model is fine-tuned using the cross entropy loss function.

[0095] As a preferred implementation, in step S5, if the first matching result is no match, rewriting the user question and the preliminary answer by using the third proxy model to obtain a first rewritten question-answer pair includes:

[0096] If the first matching result is no match, rewriting the user question by using the third proxy model to obtain several detailed user questions for details;

[0097] Rewriting the preliminary answer through the third agent model to obtain several preliminary detailed answers for the details, wherein the detailed user questions correspond to the preliminary detailed answers one by one;

[0098] A first rewritten question-answer pair is obtained based on the detailed user question and the detailed preliminary answer.

[0099] Exemplarily, if the first matching result is no match, then the third proxy model G R A preliminary answer P Rewrite and get more detailed user questions and details user answer

[0100] It should be noted that the current retrieval enhancement generation technology mainly focuses on how to more accurately retrieve the content of the knowledge base for the user's question so as to generate better results. However, these methods ignore the fact that the content of the knowledge base is irrelevant to the user's question, so that the retrieved results do not match the user's question, thereby causing the large language model to generate relevant documents using irrelevant documents, reducing the quality of the generated documents. In the embodiment of the present invention, by analyzing and judging the generated preliminary answer, the problem of mismatch between the internal knowledge of the large language model and the knowledge base knowledge is solved.

[0101] As a preferred implementation, step S6, determining whether each question-answer pair in the first rewritten question-answer pair matches by the second proxy model, obtaining a second matching result, and retrieving a retrieval document from the industry knowledge base according to the second matching result, includes:

[0102] Determine whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model, and obtain a second matching result;

[0103] Screening the question-answer pairs whose second matching results are unmatched in the first rewritten question-answer pairs as unknown question-answer pairs;

[0104] According to the unknown question-answer pair, a search is performed in the industry knowledge base to obtain a search document.

[0105] For example, through the second proxy model G J Matching judgment of the first rewritten question-answer pair of detail representation like Then the i-th rewrite question-answer pair is considered The matching result is a match, no search is required; if Then use the details user questions Perform a search and obtain the search document D from the industry knowledge base using the search engine R ref The retrieval process can be expressed as Among them, D is the details user asks The corresponding search document; D ref is the set of all search results. Finally, the large language model G with larger parameters is used to generate the final answer a based on the set of all search results and the user's question. Output =G(q,D ref ).

[0106] It should be noted that some existing search enhancement generation methods improve the search process, judge and screen the necessity of the search and the search results, so that the large language model can better use the search results to generate relevant documents. However, these methods ignore the fact that the large language model also has inherent knowledge. On the one hand, repeated searches for the inherent knowledge of the large language model result in a waste of search documents. On the other hand, the user's questions are directly searched, and the content of the user's questions lacks details, resulting in low search quality, which in turn affects the generation quality. The present invention searches and reorganizes related questions more accurately by splitting and rewriting the user's questions, so that more accurate and relevant documents can be retrieved.

[0107] A large language model retrieval enhancement generation method provided by an embodiment of the present invention can effectively reduce reasoning time, reduce response time, and improve throughput by using a proxy model that has similar language capabilities to the large language model and can understand simple tasks to quickly execute intermediate process tasks.

[0108] The embodiment of the present invention provides a large language model retrieval enhancement generation system. Figure 2The large language model retrieval enhancement generation system includes a proxy model acquisition module 11, a user question module 12, a first matching result generation module 13, a first final answer generation module 14, a first rewritten question-answer pair generation module 15, a knowledge base retrieval module 16, and a second final answer generation module 17, wherein:

[0109] The proxy model acquisition module 11 is used to obtain a large language proxy model according to the large language model used to answer questions; the architecture of the large language proxy model is the same as that of the large language model, but the number of parameters is smaller than that of the large language model; the large language proxy model includes a first proxy model, a second proxy model and a third proxy model;

[0110] A user question module 12, used to obtain user questions, input the user questions into the first agent model, and obtain a preliminary answer;

[0111] A first matching result generating module 13, configured to determine whether the user's question and the preliminary answer match by using the second agent model, and obtain a first matching result;

[0112] A first final answer generating module 14, configured to directly input the user's question into the large language model to generate a final answer if the first matching result is a match;

[0113] A first rewritten question-answer pair generating module 15, configured to rewrite the user question and the preliminary answer by using the third proxy model to obtain a first rewritten question-answer pair if the first matching result is a mismatch;

[0114] A knowledge base retrieval module 16, configured to determine whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model, obtain a second matching result, and retrieve a retrieval document from the industry knowledge base according to the second matching result;

[0115] The second final answer generating module 17 is used to input the search document and the user question into the large language model to obtain a final answer.

[0116] As a preferred implementation, the large language model retrieval enhancement generation system further includes:

[0117] A label question and answer pair acquisition module is used to acquire a label question and answer pair with a label from an industry knowledge base; the label question and answer pair includes a label question and a label answer;

[0118] A preliminary label answer generation module, used for inputting the label question into the first agent model to obtain a preliminary label answer;

[0119] A correlation calculation module, used to calculate the correlation between the preliminary label answer and the label answer;

[0120] The second proxy model fine-tuning module is used to fine-tune the parameters of the second proxy model according to the correlation.

[0121] Further, preferably, the correlation calculation module is specifically used to:

[0122] pass Calculating the correlation between the preliminary label answer and the label answer;

[0123] Among them, a p,i The preliminary labeling response described in paragraph i; a label,i is the label answer of the i-th paragraph; precision is the accuracy of the j-gram in the preliminary label answer; N is a preset constant.

[0124] Preferably, the second proxy model fine-tuning module comprises:

[0125] An answering ability determination unit, used to obtain the ability of the first agent model to answer the label question according to the magnitude relationship between the correlation and a preset threshold;

[0126] A fine-tuning data construction unit, configured to construct fine-tuning data of a second agent model according to the ability of the first agent model to answer the label question;

[0127] The second proxy model fine-tuning unit is used to fine-tune the parameters of the second proxy model according to the fine-tuning data using a cross entropy loss function.

[0128] Furthermore, in some preferred embodiments, the second proxy model fine-tuning unit is specifically used to:

[0129] Acquire an input part and an output part from the fine-tuning data; wherein the input part is the user's question and the corresponding preliminary label answer, and the output part is the ability of the first agent model to answer the label question;

[0130] pass For the second proxy model P J,Φ Fine-tune the parameters of J is the input part; y J is the output part; t is the data identifier currently involved in the calculation.

[0131] As a preferred implementation, the large language model retrieval enhancement generation system further includes:

[0132] A second rewritten question-answer pair generating module, configured to input the user question and the preliminary answer into the large language model to obtain a second rewritten question-answer pair;

[0133] The third proxy model fine-tuning module is used to fine-tune the parameters of the third proxy model according to the second rewritten question and answer pair.

[0134] Further, preferably, the third proxy model fine-tuning module is specifically used to:

[0135] pass For the third proxy model P R,Φ Fine-tune the parameters of G Asking questions to the user and providing the corresponding preliminary answers; G Rewrite the question-answer pair for the second one; t is the data identifier currently involved in the calculation.

[0136] As a preferred implementation, the first rewriting question-answer pair generating module 15 is specifically used for:

[0137] If the first matching result is no match, rewriting the user question by using the third proxy model to obtain several detailed user questions for details;

[0138] Rewriting the preliminary answer through the third agent model to obtain several preliminary detailed answers for the details, wherein the detailed user questions correspond to the preliminary detailed answers one by one;

[0139] A first rewritten question-answer pair is obtained based on the detailed user question and the detailed preliminary answer.

[0140] As a preferred implementation, the knowledge base retrieval module 16 is specifically used for:

[0141] Determine whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model, and obtain a second matching result;

[0142] Filtering the question-answer pairs whose second matching results are unmatched in the first rewritten question-answer pairs as unknown question-answer pairs;

[0143] According to the unknown question-answer pair, a search is performed in the industry knowledge base to obtain a search document.

[0144] A large language model retrieval enhancement generation system provided by an embodiment of the present invention can effectively reduce reasoning time, lower response time, and improve throughput by using a proxy model that has similar language capabilities to the large language model and can understand simple tasks to quickly execute intermediate process tasks.

[0145] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0146] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A large language model retrieval enhancement generation method, characterized in that: include: According to the large language model used to answer questions, a large language proxy model is obtained; the architecture of the large language proxy model is the same as that of the large language model, but the number of parameters is smaller than that of the large language model; the large language proxy model includes a first proxy model, a second proxy model and a third proxy model; Obtaining a user question, inputting the user question into the first agent model, and obtaining a preliminary answer; Determining whether the user's question and the preliminary answer match by using the second proxy model, and obtaining a first matching result; If the first matching result is a match, directly inputting the user's question into the large language model to generate a final answer; If the first matching result is no match, rewriting the user question and the preliminary answer by using the third proxy model to obtain a first rewritten question-answer pair; Determining whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model to obtain a second matching result, and retrieving a search document from the industry knowledge base according to the second matching result; The retrieved document and the user question are input into the large language model to obtain a final answer.

2. A large language model retrieval enhancement generation method as claimed in claim 1, characterized in that: Before determining whether the user's question and the preliminary answer match by the second proxy model to obtain a first matching result, the search enhancement generation method further includes: Obtaining a labeled question-answer pair from an industry knowledge base; the labeled question-answer pair includes a labeled question and a labeled answer; Inputting the label question into the first agent model to obtain a preliminary label answer; Calculating the correlation between the preliminary label answer and the label answer; According to the correlation, parameters of the second proxy model are fine-tuned.

3. A large language model retrieval enhancement generation method as claimed in claim 2, characterized in that: The calculating the correlation between the preliminary label answer and the label answer comprises: pass Calculating the correlation between the preliminary label answer and the label answer; Among them, a p,i The preliminary labeling response described in paragraph i; a label,i The label answer for the i-th paragraph; precision j is the accuracy of j-gram in the preliminary tag answer; N is a preset constant.

4. A large language model retrieval enhancement generation method as claimed in claim 2, characterized in that: The fine-tuning of the parameters of the second proxy model according to the correlation includes: According to the relationship between the correlation and a preset threshold, the ability of the first agent model to answer the label question is obtained; Constructing fine-tuning data for a second agent model according to the ability of the first agent model to answer the label question; According to the fine-tuning data, a cross entropy loss function is used to fine-tune the parameters of the second proxy model.

5. A large language model retrieval enhancement generation method as claimed in claim 4, characterized in that: The step of fine-tuning the parameters of the second proxy model using a cross entropy loss function according to the fine-tuning data includes: Acquire an input part and an output part from the fine-tuning data; wherein the input part is the user's question and the corresponding preliminary label answer, and the output part is the ability of the first agent model to answer the label question; pass For the second proxy model P J,Φ Fine-tune the parameters of J is the input part; y J is the output part; t is the data identifier currently involved in the calculation.

6. A large language model retrieval enhancement generation method as claimed in claim 1, characterized in that: Before the step of rewriting the user question and the preliminary answer by using the third proxy model to obtain a rewritten question-answer pair if the first matching result is a mismatch, the retrieval enhancement generation method further includes: Inputting the user question and the preliminary answer into the large language model to obtain a second rewritten question-answer pair; Based on the second rewritten question-answer pair, parameters of the third proxy model are fine-tuned.

7. A large language model retrieval enhancement generation method as claimed in claim 6, characterized in that: The fine-tuning of the parameters of the third proxy model according to the second question-answer pair includes: pass For the third proxy model P R,Φ Fine-tune the parameters of G Asking questions to the user and providing the corresponding preliminary answers; G Rewrite the question-answer pair for the second one; t is the data identifier currently involved in the calculation.

8. A large language model retrieval enhancement generation method as claimed in claim 1, characterized in that: If the first matching result is a mismatch, rewriting the user question and the preliminary answer by using the third proxy model to obtain a first rewritten question-answer pair, including: If the first matching result is no match, rewriting the user question by using the third proxy model to obtain several detailed user questions for details; Rewriting the preliminary answer through the third agent model to obtain several preliminary detailed answers for the details, wherein the detailed user questions correspond to the preliminary detailed answers one by one; A first rewritten question-answer pair is obtained based on the detailed user question and the detailed preliminary answer.

9. A large language model retrieval enhancement generation method as claimed in claim 1, characterized in that: The determining whether each question-answer pair in the first rewritten question-answer pair matches by the second proxy model to obtain a second matching result, and retrieving a retrieval document from the industry knowledge base according to the second matching result includes: Determine whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model, and obtain a second matching result; Filtering the question-answer pairs whose second matching results are unmatched in the first rewritten question-answer pairs as unknown question-answer pairs; According to the unknown question-answer pair, a search is performed in the industry knowledge base to obtain a search document.

10. A large language model retrieval enhancement generation system, characterized in that: include: A proxy model acquisition module, used to obtain a large language proxy model according to the large language model used to answer questions; the architecture of the large language proxy model is the same as that of the large language model, but the number of parameters is smaller than that of the large language model; the large language proxy model includes a first proxy model, a second proxy model and a third proxy model; A user question module, used for obtaining user questions, inputting the user questions into the first agent model, and obtaining preliminary answers; A first matching result generating module, configured to determine whether the user's question and the preliminary answer match by using the second proxy model, and obtain a first matching result; a first final answer generating module, configured to directly input the user's question into the large language model to generate a final answer if the first matching result is a match; a first rewritten question-answer pair generating module, configured to rewrite the user question and the preliminary answer by using the third proxy model to obtain a first rewritten question-answer pair if the first matching result is a mismatch; A knowledge base retrieval module, used to determine whether each question-answer pair in the first rewritten question-answer pair matches by using the second proxy model, obtain a second matching result, and retrieve a retrieval document from the industry knowledge base according to the second matching result; The second final answer generating module is used to input the search document and the user question into the large language model to obtain a final answer.

Images