Large language model answer reasoning method and system fusing memory and iterative optimization

Abstract

The invention discloses a memory and iterative optimization fused big language model answer reasoning method and system. Firstly, feedback information in a natural language form is generated for answers of sub-questions through an answer verification module, and the situation that following reasoning steps all deviate from a correct reasoning path after a certain step of the model has an error is avoided. Secondly, through a sub-problem verification module, verification pruning is performed on sub-problems generated in thinking chain reasoning, and interference of information of irrelevant sub-problems on the model is avoided. And finally, by utilizing feedback information and iteratively optimizing the output of the model, the self-reflection and correction process of human is simulated, and the situation that the large language model generates the same error in the same reasoning step is avoided. According to the method, the large language model is not finely adjusted, so that resources required for finely adjusting the large language model are saved, and the performance of the large language model in the aspect of reasoning is improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and large language models, and specifically relates to a large language model answer reasoning method that integrates memory and iterative optimization, in order to obtain feedback in the form of natural language on the model output based on a check module, and to improve the answer performance of the large language model by prompting the feedback; and to use a memory module to optimize the thinking chain reasoning process of the large language model. Background Art

[0002] Large language models are an important technology in the field of artificial intelligence. In natural language processing tasks, large language models can learn more language rules and patterns, so as to understand and generate text more accurately. In addition, large language models can also improve the fluency and naturalness of conversations through multiple rounds of dialogue and context understanding. In terms of application, large language models are also widely used in various fields, such as natural language translation, intelligent customer service, sentiment analysis, text generation, etc.

[0003] In the prior art, large language models such as GPT-3 are already able to generate seemingly reasonable texts and perform some simple reasoning tasks. However, these models still have many defects in reasoning ability. For example, when relatively complex questions are input, large language models often output wrong answers. First, large language models do not perform well in logical reasoning tasks. Most language models are trained based on large-scale corpora. They can generate seemingly reasonable texts, but for logical reasoning tasks, especially those that require rigorous logic and proofs, the performance of the model may not be ideal. For some tasks that require understanding of complex mathematical structures or concepts, the model needs to be able to understand and operate abstract concepts and structures. Secondly, large language models lack self-reflection ability. The model cannot reflect and correct itself like humans. Even if the model makes mistakes during reasoning, it cannot consciously realize and correct these mistakes, and in the same reasoning step, the large language model will repeatedly make the same mistakes.

[0004] At present, the optimization technologies for the reasoning ability of large language models mainly include external knowledge bases, thought chains, and optimization through feedback information. External knowledge base technology supports the reasoning process of the model by providing additional information. Thought chain technology simulates the human problem-solving process by guiding the model to generate a series of intermediate reasoning steps, so that the model can make more accurate predictions on problems that require multi-step reasoning. Feedback technology generates feedback in natural language form for the output of the model through multiple feedback generation modules, improving the accuracy and reliability of the model. However, the traditional way of using Faiss tools to build an external knowledge base is difficult to accurately retrieve relevant information; simple thought chain technology still cannot guide the model to solve most complex problems. In the process of generating thought chains, the model is often unable to output the correct answer due to an error in one link.

[0005] Therefore, based on the research content, the present invention designs a large language model answer reasoning method that integrates memory and iterative optimization to provide technical support for improving the reasoning performance of large language models. Summary of the invention

[0006] The present invention aims at the reasoning process of the large language model, and improves the performance of the large model in answer reasoning by using a verification module, a memory module and feedback information. This paper invents a large language model answer reasoning method that integrates memory and iterative optimization, providing technical support for improving the reasoning performance of the large language model.

[0007] The present invention provides a large language model answer reasoning method integrating memory and iterative optimization, comprising the following steps:

[0008] S1, input question Q and initialize known information K to empty.

[0009] S2, using a large language model, generates M sub-question sets related to question Q based on known information K and question Q.

[0010] S3, for each sub-question, use the sub-question verification module to verify the relevance of the generated sub-question to question Q.

[0011] S4, if the sub-question is related to question Q, use the answer verification module to generate the answer to the sub-question.

[0012] S5, add the sub-questions and the answers corresponding to the sub-questions to the known information to form new known information K. Determine whether the answer to question Q can be obtained based on K. If the answer can be obtained, output the answer, otherwise repeat steps S2, S3, S4 and S5 until the model outputs the answer to question Q.

[0013] Furthermore, step S2 specifically includes the following sub-steps:

[0014] S21, Designing Prompts for Large Language Models Q , used to prompt the large language model to generate a sub-problem set S = {q1,q2,q3...q l}.

[0015] S22, when generating sub-problem set S, prompt P Q Contains examples, known information K and questions Q. The examples are used to prompt the model to generate a sub-question set S according to the format, and the known information K is initially empty.

[0016] Furthermore, step S3 specifically includes the following sub-steps:

[0017] S31, use sub-problem verification module V Q Check the sub-problem set S and generate feedback information f, which contains the relevance of the sub-problem to the problem Q.

[0018] S32, add sub-problems with higher relevance to the relevant sub-problem set S A middle.

[0019] S33, if S A The number of neutron problems is less than L S The feedback information f, known information K and question Q are input into the model to generate new sub-questions, and the sub-question verification module V is used Q Check until S A The number of neutron problems is at least L S indivual.

[0020] Furthermore, step S4 specifically includes the following sub-steps:

[0021] S41, design external component A to assist the large language model in answering questions.

[0022] S42, Design Tips P A , used to prompt the large language model to generate the output o for the sub-question, prompting P A Contains information I provided by external components, feedback information f and sub-problems q.

[0023] S43, using the answer verification module V A Check o and generate feedback information f.

[0024] S44, if f indicates that o passes the check, then o is taken as subproblem q i The answer.

[0025] S45, if f indicates that o fails the check, repeat steps S42, S43, S44 and S45 to regenerate o.

[0026] Furthermore, step S5 specifically includes the following sub-steps:

[0027] S51, adding the sub-question set and the corresponding answers to the known information K.

[0028] S52, use the model to determine whether the answer to question Q can be obtained based on K. If the answer to question Q can be obtained, the answer is output. If not, S2, S3, S4 and S5 are repeated until the answer to question Q can be obtained based on K.

[0029] Furthermore, in step S2, the generation of the sub-question set can be completed by prompting the large language model with sub-question decomposition examples of similar questions. A sub-question decomposition example vector database can be constructed, and related questions can be retrieved as prompts by similarity.

[0030] Further, in step S3, the output f of the sub-problem verification module represents the sub-problem q i The relevance of question Q. Set the scoring range and relevance threshold of the model, and add sub-questions with relevance exceeding the threshold to the new sub-question set. If the number of sub-question sets does not meet the requirements, use feedback information f to prompt the model to generate new questions. f contains sub-questions that have been generated, which can prevent the model from outputting the same question all the time.

[0031] Furthermore, in step S4, the external component A may include a vector database, a knowledge graph, and a search API, etc. Different external components may be selected for different types of questions. According to the input question, each component is used to obtain information related to the question. The information provided by each component is integrated, and the large language model is used to extract information related to the question from the integrated text.

[0032] Further, in step S4, the output f of the answer verification module represents o i Is it a sub-problem q? i The answer. i If it is wrong, the verification fails, and then the model generates new output according to f, and repeats the verification steps until it passes the verification or the number of iterations reaches the upper limit; if it passes the verification, q i and i Add to the known information K. The role of the feedback information f is to prompt the model to generate correct output based on the existing information. By using the feedback information, the accuracy of the model in answering sub-questions can be improved.

[0033] Furthermore, in step S3, an upper limit T of the number of sub-problem iterations is set. Q . Repeat the generation times to T Q Then, if the sub-problem set S A If it is empty, then sort all sub-questions in descending order of relevance and take L s Added questions to S A, go to step S4; if S A If it is not empty, go directly to step S4. In S4, set o i The upper limit of the number of iterations L A In answering sub-question q i When i The number of times generated reaches L A , then o i As a question q i The answer goes to step S5.

[0034] The present invention also provides a large language model answer reasoning system that integrates memory and iterative optimization, including a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a large language model answer reasoning method that integrates memory and iterative optimization as described in the above technical solution.

[0035] Compared with existing research and technology, the present invention has the following beneficial effects:

[0036] The present invention generates feedback information in natural language for the answers to sub-questions through an answer verification module, thereby preventing all subsequent reasoning steps from deviating from the correct reasoning path after an error occurs in a certain step of the large language model.

[0037] The present invention verifies and prunes the sub-problems generated in the thought chain reasoning through the sub-problem verification module, so as to avoid the interference of irrelevant sub-problem information on the large language model.

[0038] The present invention utilizes feedback information to iteratively optimize the output of the model, simulates the human self-reflection and correction process, and avoids the large language model from making the same error in the same reasoning step. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the overall process of the large language model answer reasoning method that integrates memory and iterative optimization provided by the present invention;

[0040] Figure 2 A flowchart of a prompt model generating sub-problems in the large language model answer reasoning method that integrates memory and iterative optimization provided by the present invention;

[0041] Figure 3 A flowchart of performing sub-question relevance checking in the large language model answer reasoning method that integrates memory and iterative optimization provided by the present invention;

[0042] Figure 4 A flowchart for answering sub-questions in the large language model answer reasoning method that integrates memory and iterative optimization provided by the present invention. DETAILED DESCRIPTION

[0043] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings.

[0044] like Figure 1 As shown, the present invention provides a large language model answer reasoning method integrating memory and iterative optimization, comprising the following steps:

[0045] S1, input question Q and initialize known information K to empty.

[0046] S2 uses a large language model to generate M sub-questions related to question Q based on known information K and question Q.

[0047] S2 specifically includes the following steps:

[0048] S21, such as Figure 2 As shown, the model design prompt P Q , used to prompt the large language model to generate a sub-problem set S = {q1,q2,q3...q l}=M(P Q ,Q,K), l is a hyperparameter, indicating the upper limit of the size of the sub-problem set, and M is the large language model;

[0049] S22, when generating sub-problem set S, prompt P Q Contains examples, known information K and questions Q, where the examples are used to prompt the model to generate a sub-question set S according to the format, and the known information K is initially empty;

[0050] Furthermore, in S2, the generation of sub-problem sets can be completed by prompting the large language model with sub-problem decomposition examples of similar problems. A sub-problem decomposition example vector database can be constructed, and decomposition examples can be retrieved as prompts by similarity.

[0051] S3, for each sub-question, use the sub-question verification module to verify the relevance of the generated intermediate question to question Q.

[0052] S3 specifically includes the following steps:

[0053] S31, such as Figure 3 As shown, the sub-problem verification module V is used Q Check each sub-problem in S and generate feedback information f = M (P VQ ,Q,S), where P VQ It is used to prompt the model to score the relevance between the sub-question and question Q. f contains the relevance between the sub-question and question Q.

[0054] S32, add sub-problems with higher relevance to sub-problem set S A middle;

[0055] S33, if S A The number of neutron problems is less than L S The feedback information f, known information K and question Q are input into the model to generate new sub-questions, and the sub-question verification module V is used Q Check until S A The number of neutron problems is at least L S indivual.

[0056] Furthermore, in S3, the output f of the subproblem verification module represents the subproblem q i The relevance to question Q. Set the scoring interval [Nmin, Nmax] and relevance threshold FL of the hyperparameter model. If f represents q i If the correlation with Q is greater than FL, then q i Add to set S A middle.

[0057] Furthermore, in S3, if the number of sub-question sets does not meet the requirement, feedback information f is used to prompt the model to generate new sub-questions. f contains sub-questions that have been generated, so as to avoid the large language model from outputting the same question all the time.

[0058] Furthermore, in step S3, an upper limit T of the number of sub-problem iterations is set. Q . Repeat the generation times to T Q Then, if the sub-problem set S A If it is empty, then sort all sub-questions in descending order of relevance and take L S Added questions to S A , go to step S4; if S A If it is not empty, go directly to step S4. In S4, set o i The upper limit of the number of iterations L A In answering sub-question q i When i The number of times generated reaches L A , then o i As a question q i The answer goes to step S5.

[0059] S4, if the intermediate question is related to question Q, use the answer verification module to generate the answer to the sub-question that passes the verification.

[0060] S4 specifically includes the following steps:

[0061] S41, such as Figure 4 The design of external component A shown assists the model in answering the question;

[0062] S42, Design Tips P A , used to prompt the model to generate S AAll subproblems {q1,q2,q3...q l The output of {o1,o2,o3...o l}=M(P A ,I,F,q i ), prompt P A Contains information I provided by external components, feedback information F and sub-question q i .

[0063] S43, using the answer verification module V A For {o1,o2,o3...o l} Check and generate feedback information f a ={f1,f2,f3...f l}=M(P VA ,S A ,o1,o2,o3...o1), where P VA It is a prompt used to improve the model for inspection;

[0064] S44, if f i Indicates that the inspection has passed. i As a sub-problem q i The answer;

[0065] S45, if f i Indicates o i If the check fails, i Join P A Regenerate output in i .

[0066] Furthermore, in S4, external component A may include vector database, knowledge graph and search API, etc. Different external components may be selected for different types of questions. According to the input question, each component is used to obtain information related to the question. The information provided by each component is integrated, and the format of the integrated information is [[doc1,source i ],[doc2,source2]..], where doc i For the retrieved questions, source i For doc i The source of the question. A large language model is used to extract information related to the question from the integrated text. In the present invention, a vector database and a search API are used to assist the model in answering.

[0067] Furthermore, in S4, the construction process of the vector database is as follows:

[0068] 1. Prepare training data. Build a query data set and prepare corresponding positive sample documents and negative sample documents for each query in the query data set;

[0069] 2. Training model. Two independent BERT models are used as query encoder BQ and document encoder BA respectively. The model training goal is to make the relevant pairs of query and document have a smaller distance;

[0070] 3. Use the document encoder BA to encode each document in the document collection and convert it into a fixed-length vector p;

[0071] 4. Use the query encoder BQ to encode each query in the query set and convert it into a fixed-length vector q;

[0072] 5. Let D be the training data containing m instances, each instance contains a query q i and a related document and n irrelevant documents The optimized loss function is the negative log-likelihood function as follows,

[0073]

[0074] Furthermore, in S4, the process of using the vector database is as follows:

[0075] 1. Use the query encoder BQ to encode the user’s question and convert it into a fixed-length vector q;

[0076] 2. Use q to calculate the cosine similarity with the encoding vectors of all documents, and take the document with the highest similarity as the result.

[0077] Furthermore, in S4, the workflow of the search API is as follows:

[0078] 1. Call the API interface to obtain the relevant web page set W = {w1,w2...w n};

[0079] 2. Use the query encoder BQ to encode the question as q, use BA to encode the text content in the web page as p, and calculate the similarity e sim(q,p) , remove the web pages with lower similarity from W and get W f ;

[0080] 3. Extract W f The text information in each web page is used as a reference for solving the questions.

[0081] Further, in step S4, the output f of the answer verification module a Indicates o iIs it a sub-problem q? i The answer is, if o i If it is wrong, the verification fails, and then the model generates new output according to f, and repeats the verification steps until it passes the verification or the number of iterations reaches the upper limit; if it passes the verification, q i and i Add to the known information K. In S4, the role of the feedback information f is to prompt the model to generate correct output based on the existing information. By using the feedback information, the accuracy of the model in answering sub-questions is improved.

[0082] S5, add the sub-questions and the answers corresponding to the sub-questions to the known information to form new known information K. Determine whether the answer to question Q can be obtained based on K. If the answer can be obtained, output the answer, otherwise repeat steps S2, S3, S4 and S5 until the model outputs the answer to question Q.

[0083] S5 specifically includes the following steps:

[0084] S51, adding the sub-question set and the corresponding answer to the known information K;

[0085] S52, use the model to determine whether the answer to question Q can be obtained based on K. If so, generate the answer to question Q as the final output; if not, repeat S2, S3, S4 and S5 until the answer to question Q can be obtained based on K.

[0086] The present invention also provides a large language model answer reasoning system that integrates memory and iterative optimization, including a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a large language model answer reasoning method that integrates memory and iterative optimization as described in the above technical solution.

[0087] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A large language model answer reasoning method integrating memory and iterative optimization, characterized in that: The steps include: Step S1, input question Q and initialize known information K to be empty; Step S2, using the large language model, based on the known information K and question Q, generate M sub-question sets related to question Q; Step S3, for each sub-question, using the sub-question verification module to verify the relevance of the generated intermediate question with question Q; Step S4, if the sub-question is related to question Q, use the answer verification module to generate an answer to the sub-question that passes the verification; In step S5, the sub-questions and the answers corresponding to the sub-questions are added to the known information to form new known information K, and it is determined whether the answer to question Q can be obtained based on K. If the answer can be obtained, the answer is output. Otherwise, steps S2, S3, S4 and S5 are repeated until the large language model outputs the answer to question Q.

2. A large language model answer reasoning method integrating memory and iterative optimization as claimed in claim 1, characterized in that: Step S2 specifically includes the following sub-steps: S21, Designing Prompts for Large Language Models Q , used to prompt the large language model to generate a sub-problem set S = {q1,q2,q3...q l }, l is a hyperparameter, indicating the upper limit of the size of the sub-problem set; S22, when generating sub-problem set S, prompt P Q It contains examples, known information K and questions Q, where the examples are used to prompt the large language model to generate a sub-question set S according to the format, and the known information K is initially empty.

3. The large language model answer reasoning method integrating memory and iterative optimization as claimed in claim 1, characterized in that: Step S3 specifically includes the following sub-steps: S31, use sub-question verification module V Q Check the sub-problem set S and generate feedback information f, which contains the relevance of the sub-problem to the problem Q; S32, add sub-problems with higher relevance to the relevant sub-problem set S A middle; S33, if S A The number of neutron problems is less than L S The feedback information f, known information K and question Q are input into the large model to generate new sub-problems, and the sub-problem verification module V is used Q Check until S A The number of neutron problems is at least L S indivual.

4. The large language model answer reasoning method integrating memory and iterative optimization as claimed in claim 1, characterized in that: Step S4 specifically includes the following sub-steps: S41, design external component A to assist the large language model in answering questions; S42, Design Tips P A , used to prompt the large language model to generate relevant sub-question set S A All subproblems {q1,q2,q3...q l The output of {o1,o2,o3...o l }=M(P A ,I,F,q i ), l is a hyperparameter, indicating the upper limit of the size of the sub-problem set; Tips A Contains information I provided by external components, feedback information F and sub-question q i ; S43, using the answer verification module V A For {o1,o2,o3...o l } Check and generate feedback information f a ={f1,f2,f3...f l }=M(P VA ,S A ,o1,o2,o3...o l ), where P VA It is used to increase the inspection speed; S44, if f i Indicates that the inspection has passed. i As a sub-problem q i The answer; S45, if f i Indicates o i If the check fails, i Join P A Regenerate output in i .

5. The large language model answer reasoning method integrating memory and iterative optimization as claimed in claim 1, characterized in that: The generation of sub-problems is accomplished by prompting the large language model with sub-problem decomposition examples of similar problems, by constructing a sub-problem decomposition example vector database, and by retrieving decomposition examples as prompts through similarity.

6. The large language model answer reasoning method integrating memory and iterative optimization as claimed in claim 1, characterized in that: In step S3, the output f of the sub-problem verification module represents the sub-problem q i The relevance of the model to question Q, set the scoring range and relevance threshold of the model, add the sub-questions whose relevance exceeds the threshold to the new sub-question set, and if the number of sub-question sets does not meet the requirements, use the feedback information f to prompt the model to generate new questions; In step S3, set the upper limit T of the number of sub-problem iterations Q , the number of repeated generation reaches T Q Then, if the sub-problem set S A If it is empty, then sort all sub-questions in descending order of relevance and take L S Added questions to S A , go to step S4; if S A If it is not empty, go directly to step S4.

7. The large language model answer reasoning method integrating memory and iterative optimization as claimed in claim 1, characterized in that: In step S4, the external component A includes a vector database, a knowledge graph, and a search API, and different external components are selected for different types of questions; The construction process of the vector database is as follows: (1) Construct a query dataset and prepare corresponding positive sample documents and negative sample documents for each query in the query dataset; (2) Two independent BERT models are used as query encoder BQ and document encoder BA respectively, and the training goal is to make the relevant pairs of queries and documents have a smaller distance; (3) Use the document encoder BA to encode each document in the document collection and convert it into a fixed-length vector p; (4) Use the query encoder BQ to encode each query in the query set and convert it into a fixed-length vector q; (5) Let D be the training data containing m instances, each instance containing a query q i and a related document and n irrelevant documents The optimized loss function is the negative log-likelihood function as follows, sim is the similarity function; The process of using the vector database is as follows: (a) Use the query encoder BQ to encode the user’s question and convert it into a fixed-length vector q; (b) Use q to calculate the cosine similarity with the encoding vectors of all documents, and take the document with the highest similarity as the result.

8. A large language model answer reasoning method integrating memory and iterative optimization as claimed in claim 7, characterized in that: The workflow of the search API is as follows: A1, call the API interface to obtain the relevant web page set W = {w1,w2...w n }; A2, use the query encoder BQ to encode the question as q, use BA to encode the text content in the web page as p, and calculate the similarity e sim(q,p) , remove the web pages with lower similarity from W and get W f ; A3, extract W f The text information in each web page is used as a reference for solving the questions.

9. The large language model answer reasoning method integrating memory and iterative optimization as claimed in claim 1, characterized in that: In step S4, the output f of the feedback module represents o i Is it a sub-problem q? i The answer is, if o i If it is wrong, the verification fails, and then the model is prompted to generate new output according to f, and the verification steps are repeated until the verification passes or the number of iterations reaches the upper limit; If the verification is passed, q i and i Add to the known information K; In step S4, set o i The upper limit of the number of iterations L A , in answering sub-question q i When o i The number of times generated reaches L A , then o i As a question q i The answer goes to step S5.

10. A large language model answer reasoning system integrating memory and iterative optimization, characterized by: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a large language model answer reasoning method that integrates memory and iterative optimization as described in any one of claims 1 to 9.

Images