Retrieval iteration method and device by means of generative large language model

Abstract

The embodiment of the invention relates to a retrieval iteration method and device by means of a generative large language model. The method comprises the steps that the generative large language model is selected for a retrieval system R to serve as a model M; recording an instruction set of the model M as an instruction set C; recording information resource libraries of the retrieval system R and the model M as document libraries GR and GM; iteration parameters are set; taking the question text input by the user as a question q; performing a round of retrieval iteration by the retrieval system R and the model M to obtain a document set D; and performing answer text generation based on the question q, the document set D, the instruction set C and the model M. According to the invention, the retrieval quality and the answer generation quality can be improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a retrieval iterative method and device using a generative large language model. Background Art

[0002] The Retrieval Augmented Generation (RAG) framework is a technical framework used to improve the quality of generated answers when handling intelligent question-answering tasks based on large language models (LLMs). The RAG framework configures LLMs with a retrieval system (Retriever). This retrieval system retrieves documents from a pre-defined information database using the question text as a query. The LLMs then use the retrieved documents as a reference to generate corresponding answer texts for the question text. In principle, the RAG framework uses the retrieved documents output by the retrieval system as supplementary knowledge to enhance the LLMs' understanding, reasoning, and generation capabilities of the input text. However, in real-world applications, some retrieval systems are limited by their retrieval performance. This is particularly true for retrieval systems based on sparse retrieval algorithms (such as bag-of-words, TF-IDF, and BM25). The retrieved documents they output cannot fully support answer generation. In such cases, the retrieved documents fed into the LLMs not only fail to enhance information but may also generate useless information, resulting in reduced quality of the model's output answer text.

[0003] The natural language task processing capabilities of LLMs are very powerful, and the prompt engineering (PromptEngineering) based on LLMs can also build a rich set of customized instructions. If LLMs with instruction sets can be reversely applied to the retrieval process, such as identifying whether the retrieved documents meet the answer generation requirements, sorting the relevance of the retrieved documents to the question text, generating pseudo-queries for the question text (such as question rewriting, missing item descriptions, synonym replacement, background elaboration, topic expansion, etc.) to further expand the search scope, and expanding the retrieval information base through pre-trained knowledge and fine-tuning of specific domain knowledge, it is bound to make a positive contribution to improving the retrieval quality. How to use LLMs and their instruction sets to achieve this is also the technical problem that the present invention needs to solve. Summary of the Invention

[0004] The purpose of the present invention is to address the shortcomings of the prior art and provide a retrieval iteration method, device, electronic device and computer-readable storage medium using a generative large language model. The present invention selects a generative large language model that has completed pre-training, natural language task training and prompt engineering fine-tuning as the corresponding model M for the retrieval system R; and records the instruction set obtained by the model M through the prompt engineering as the instruction set C; and uses the model M and its instruction set C to construct a retrieval iteration process for optimizing retrieval quality. In this retrieval iteration process, the retrieval system R first uses the question q as the query Q to query the document library G. * Search and get document set D t , and then the model M uses the document sorting instructions in the instruction set C based on the current document set D t Refresh the document set D, and then use the two types of retrieval verification instructions provided by the instruction set C to verify whether the document set D meets the question q. If the verification passes, the iteration is stopped and the model M uses the retrieval enhancement generation instructions provided by the instruction set C to generate the answer text according to the question q and the document set D. If the verification fails, the model M continues to use the five types of pseudo-query generation instructions provided by the instruction set C to expand the query Q and use the document selection instructions to select the document library G. * Expand, and the retrieval system R searches the latest document library G based on the latest query Q * Retrieve and obtain a new document set D t Until the latest document set D passes the verification. The present invention can not only improve the retrieval quality of the retrieval system, but also improve the answer generation quality of the large language model itself.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present invention provides an iterative retrieval method using a generative large language model, the method comprising:

[0006] Step 1: Select a generative large language model that has completed pre-training, natural language task training, and prompt engineering fine-tuning for the retrieval system R as the corresponding model M; and record the instruction set obtained by the model M through the prompt engineering as the instruction set C; and record the information resource library of the retrieval system R as the document library G R , the information resource library used by the model M during training and fine-tuning is recorded as document library G M ; And set the corresponding iteration threshold T, retrieval threshold N, subpackaging threshold L, verification mode V, and pseudo query configuration W;

[0007] Step 2: Take the question text entered by the user as question q; and * Set as the document library G B ;

[0008] Step 3, initialize the counter t to 1, the document set D to be empty, and the query Q to the question q;

[0009] Step 4: Identify whether the document set D is not empty; if so, go to step 5; if not, go to step 6;

[0010] Step 5: performing pseudo query generation processing based on the pseudo query configuration W, the instruction set C, and the model M and resetting the query Q based on the processing result;

[0011] Step 6: The retrieval system R searches the document library G according to the query Q and the retrieval threshold N. * Search and get document set D t ; and divide the document set D into t Divide into multiple subsets D t,sub ; and from the first to the last of said subset D t,sub Slide step by step, and at each step based on the current subset D t,sub , the instruction set C and the model M refresh the document set D; the first to the second to last subset D t,sub The number of documents is L, and the last subset D t,sub The number of documents is less than or equal to L;

[0012] Step 7: Verify whether the document set D satisfies the question q according to the verification mode V, the instruction set C, and the model M to obtain a corresponding current verification result; the current verification result includes pass and fail;

[0013] Step 8: Identify whether the current verification result is failed; if so, add 1 to the counter t and go to step 9; if not, go to step 11;

[0014] Step 9: Identify whether the counter t exceeds the iteration threshold T; if so, go to step 10; if not, return to step 4;

[0015] Step 10: Based on the document library G M , the instruction set C and the model M to the document library G * Expand multiple information sources; and return to step 3 after completing the expansion;

[0016] Step 11: Generate an answer text based on the question q, the document set D, the instruction set C, and the model M to obtain the corresponding answer text and feed it back to the current user.

[0017] Preferably, the retrieval system R includes at least a sparse retriever and a dense retriever; the sparse retriever includes at least a text retriever based on a bag-of-words algorithm, a text retriever based on a TF-IDF algorithm, and a text retriever based on a BM25 algorithm; the dense retriever is a type of text retriever based on a deep learning model, including at least a BERT series model;

[0018] The model M includes at least a GPT series model, a T5 series model, and a BAT series model;

[0019] The document library G R , G M Each document is composed of multiple documents, each document corresponds to a unique document number; the document library G M The documents come from multiple information sources;

[0020] The instruction set C includes document sorting instructions c sor , Retrieve verification instruction c cls , Retrieve verification instruction c sco , pseudo query generation instruction c pse-que , pseudo query generation instruction c pse-pas , pseudo query generation instruction c pse-syn , pseudo query generation instruction c pse-ctx , pseudo query generation instruction c pse-top , document selection instruction c sel , search enhancement generation instruction c rag All instructions of the instruction set C are formatted natural language prompt templates, and each has a set of configurable instruction parameters;

[0021] The document sorting instruction c sor The configurable instruction parameters include the problem parameter p q , document set parameter p D ; The document sorting instruction c sor Used to guide the model M to set the document set parameter p D The problem parameter p q The document numbers of the k most relevant documents are sorted in descending order of relevance and a corresponding number sorting sequence is generated; the number k is specified as a positive integer;

[0022] The retrieval verification instruction c cls The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The retrieval verification instruction c cls Used to guide the model M to the document set parameter p D Whether the problem parameter p is fully satisfiedq Verify the answer condition and generate a specific verification result of yes or no;

[0023] The retrieval verification instruction c sco The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The retrieval verification instruction c sco It is used to guide the model M to evaluate the document set parameter p according to the set scoring range. D For the problem parameter p q Score the answer support and generate the corresponding score;

[0024] The pseudo query generation instruction c pse-que The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The pseudo query generation instruction c pse-que Parameters p used to guide the model M to answer the question q The document set parameter p is required but D Infer the missing information that is not covered, design corresponding questions for the inferred missing information, and output the designed questions as the generated pseudo query;

[0025] The pseudo query generation instruction c pse-pas The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The pseudo query generation instruction c pse-pas Parameters p used to guide the model M to answer the question q The document set parameter p is required but D Infer the missing information that is not covered, and generate a piece of information description text based on the inferred missing information as the corresponding pseudo query output;

[0026] The pseudo query generation instruction c pse-syn The configurable instruction parameters include the problem parameter p q ; The pseudo query generation instruction c pse-syn Used to guide the model M to the problem parameter p q Generate a set of synonyms or semantically related words as corresponding pseudo-query output;

[0027] The pseudo query generation instruction c pse-ctx The configurable instruction parameters include the problem parameter p q ; The pseudo query generation instruction c pse-ctx Used to guide the model M to the problem parameter pq Generate a background text as the corresponding pseudo query output;

[0028] The pseudo query generation instruction c pse-top The configurable instruction parameters include the problem parameter p q ; The pseudo query generation instruction c pse-top Used to guide the model M to the problem parameter p q Identify the knowledge topics and their knowledge domains, and generate a set of topic texts in the same domain as the corresponding pseudo query outputs for the identified problem topics;

[0029] The document selection instruction c sel The configurable instruction parameters include the problem parameter p q ; The document selection instruction c sel For guiding the model M from the document library G M All information sources select a specified number n of documents to form a selected document set, and require that the selected document set must meet the problem parameter p q The solution condition is that the credibility of all documents in the selected document set exceeds a preset credibility threshold, the content consistency score of all documents in the selected document set exceeds a preset consistency score threshold, there is no semantic contradiction between any two documents in the selected document set, and the corresponding document number set is composed of the n document numbers of the selected document set that are finally confirmed and output; the specified number n is a positive integer;

[0030] The retrieval enhancement generation instruction c rag The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The retrieval enhancement generation instruction c rag Used to guide the model M to the document set parameter p D is a hint, and is the problem parameter p q Generate a corresponding answer text and output;

[0031] The verification mode V includes a first mode and a second mode;

[0032] The pseudo query configuration W includes five types of configuration switches w 1、 w 2、 w 3、 w 4、 w5, the value of each configuration switch is 0 or 1.

[0033] Preferably, the pseudo query generation process based on the pseudo query configuration W, the instruction set C and the model M and resetting the query Q based on the process result specifically includes:

[0034] Extract the corresponding configuration switch w from the pseudo query configuration W 1、 w 2、 w 3、 w 4、 w5;

[0035] And identify the configuration switch w1; if the configuration switch w1 is 1, the pseudo query generation instruction c pse-que The problem parameter p q , the document set parameter p D Set the corresponding question q and document set D, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-que Input the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 1 If the configuration switch w1 is 0, then set the corresponding pseudo query q 1 is empty;

[0036] And identify the configuration switch w2; if the configuration switch w2 is 1, the pseudo query generation instruction c pse-pas The problem parameter p q , the document set parameter p D Set the corresponding question q and document set D, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-pas Input the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 2 If the configuration switch w2 is 0, then set the corresponding pseudo query q 2 is empty;

[0037] And identify the configuration switch w3; if the configuration switch w3 is 1, the pseudo query generation instruction c pse-syn The problem parameter p q Set it as the corresponding question q, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-syn Input the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 3 If the configuration switch w3 is 0, then set the corresponding pseudo query q 3 is empty;

[0038] And identify the configuration switch w4; if the configuration switch w4 is 1, the pseudo query generation instruction c pse-ctx The problem parameter p q Set it as the corresponding question q, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-ctxInput the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 4 If the configuration switch w4 is 0, then set the corresponding pseudo query q 4 is empty;

[0039] And identify the configuration switch w5; if the configuration switch w5 is 1, the pseudo query generation instruction c pse-top The problem parameter p q Set it as the corresponding question q, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-top Input the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 5 If the configuration switch w5 is 0, then set the corresponding pseudo query q 5 is empty;

[0040] And the pseudo query q 1 ,q 2 ,q 3 ,q 4 ,q 5 Perform text splicing to obtain a corresponding current spliced text; and reset the query Q based on the current spliced text.

[0041] Preferably, the retrieval system R searches the document library G according to the query Q and the retrieval threshold N. * Search and get document set D t , specifically including:

[0042] The retrieval system R compares the query Q with the document library G * The relevance of each document in the , and the top N documents with the highest relevance are extracted to form the corresponding document set D t .

[0043] Preferably, each step is based on the current subset D t,sub , the instruction set C and the model M refresh the document set D, specifically including:

[0044] Based on the current subset D t,sub The current temporary document set D corresponding to the document set D * =D∪D t,sub ; and sort the document instructions c sor The problem parameter p q , the document set parameter p D Let the corresponding question q and the current temporary document set D be * ; and complete the document sorting instruction c configured with instruction parameterssor Input the model M for processing and use the numbering sequence output by this processing as the corresponding numbering sequence of the current time; and use the temporary document set D of the current time as the corresponding numbering sequence of the current time. * The k documents that meet the current numbering sequence are extracted to form the corresponding current preferred document set; and the document set D is reset based on the current preferred document set.

[0045] Preferably, verifying whether the document set D satisfies the question q according to the verification mode V, the instruction set C, and the model M to obtain a corresponding current verification result specifically includes:

[0046] Identifying the verification mode V;

[0047] If the verification mode V is the first mode, the retrieval verification instruction c cls The problem parameter p q , the document set parameter p D Set the corresponding question q and the document set D; and complete the retrieval verification instruction c with the instruction parameter configuration cls Input the model M for processing and use the verification result output by this processing as the corresponding first result; and identify the first result; if the first result is yes, set the corresponding current verification result to pass; if the first result is no, set the corresponding current verification result to fail;

[0048] If the verification mode V is the second mode, the retrieval verification instruction c sco The problem parameter p q , the document set parameter p D Set the corresponding question q and the document set D; and complete the retrieval verification instruction c with the instruction parameter configuration sco The model M is input for processing and the score output by this processing is used as the corresponding first score; and whether the first score exceeds the preset verification score threshold is identified; if it exceeds, the corresponding current verification result is set to pass; if it does not exceed, the corresponding current verification result is set to fail.

[0049] Preferably, the document library G M , the instruction set C and the model M to the document library G * Expand multiple information sources, including:

[0050] Step 71, initialize an empty first document set; and send the document selection instruction c sel The problem parameter p q Let it be the corresponding problem q;

[0051] Step 72: The document selection instruction c that has completed the instruction parameter configuration is sel Input the model M for processing and use the document number set output by this processing as the corresponding first number set;

[0052] Step 73: the document library G M The n documents that meet the first number set are extracted and added to the first document set; and the document library G in the current first document set is * The non-intersection documents of the current independent document set are composed of the corresponding current independent document set; and whether the number of documents in the current independent document set is less than n is identified; if so, return to step 72; if not, the current independent document set is sent to the document library G * Add in.

[0053] Preferably, the step of generating an answer text based on the question q, the document set D, the instruction set C, and the model M to obtain a corresponding answer text and feed it back to the current user specifically includes:

[0054] The search enhancement generation instruction c rag The problem parameter p q , the document set parameter p D Set the corresponding question q and the document set D; and complete the search enhancement generation instruction c after configuring the instruction parameters. rag The model M is input for processing and the answer text output by this processing is fed back to the current user as the corresponding current answer text.

[0055] Preferably, the method further comprises:

[0056] Start timing when the counter t is set to 1 for the first time; and during this round of iteration from step 3 to step 10, monitor in real time whether the current timing duration exceeds the preset iteration duration threshold; and when it is confirmed that the current timing duration exceeds the iteration duration threshold, exit this round of iteration and jump to step 11.

[0057] Preferably, the method further comprises:

[0058] After feeding back the answer text to the current user, receiving the user's satisfaction feedback on the answer text; and when the satisfaction feedback is unsatisfactory, optimizing the current document library G based on the preset document library optimization scheme. * Optimize, or optimize the current iteration threshold T, the retrieval threshold N, the subpackaging threshold L, the verification mode V and the pseudo query configuration W based on the preset parameter optimization scheme, and return to step 3 at the end of this optimization to perform another round of iteration and generate a new answer text through this round of iteration to feedback to the current user.

[0059] A second aspect of an embodiment of the present invention provides an apparatus for implementing the retrieval iteration method using a generative large language model as described in the first aspect, the apparatus comprising: a model and parameter preparation module, an iteration preparation module, an iteration initialization module, a document set determination module, a query refresh module, a document set refresh module, a retrieval verification module, a verification judgment module, an iteration loop module, a document library expansion module, and an answer generation module;

[0060] The model and parameter preparation module is used to select a generative large language model that has completed pre-training, natural language task training and prompt engineering fine-tuning for the retrieval system R as the corresponding model M; and record the instruction set obtained by the model M through the prompt engineering as the instruction set C; and record the information resource library of the retrieval system R as the document library G R , the information resource library used by the model M during training and fine-tuning is recorded as document library G M ; And set the corresponding iteration threshold T, retrieval threshold N, subpackaging threshold L, verification mode V, and pseudo query configuration W;

[0061] The iteration preparation module is used to take the question text input by the user as question q; and * Set as the document library G B ;

[0062] The iterative initialization module is used to initialize the counter t to 1, the document set D to be empty, and the query Q to the question q;

[0063] The document set judgment module is used to identify whether the document set D is not empty; if so, it turns to the query refresh module; if not, it turns to the document set refresh module;

[0064] The query refresh module is used to perform pseudo query generation processing based on the pseudo query configuration W, the instruction set C and the model M and reset the query Q based on the processing result;

[0065] The document set refreshing module is used for the retrieval system R to update the document library G according to the query Q and the retrieval threshold N. * Search and get document set D t ; and divide the document set D into t Divide into multiple subsets D t,sub ; and from the first to the last of said subset D t,sub Slide step by step, and at each step based on the current subset D t,sub , the instruction set C and the model M refresh the document set D; the first to the second to last subset D t,subThe number of documents is L, and the last subset D t,sub The number of documents is less than or equal to L;

[0066] The retrieval verification module is used to verify whether the document set D meets the question q according to the verification mode V, the instruction set C and the model M to obtain a corresponding current verification result; the current verification result includes pass and fail;

[0067] The verification judgment module is used to identify whether the current verification result is failed; if so, it increases the counter t by 1 and switches to the iterative loop module; if not, it switches to the answer generation module;

[0068] The iterative loop module is used to identify whether the counter t exceeds the iteration threshold T; if so, it switches to the document library expansion module; if not, it returns to the document set determination module;

[0069] The document library expansion module is used to expand the document library G M , the instruction set C and the model M to the document library G * Perform multi-information source expansion; and return to the iterative initialization module after completing the expansion;

[0070] The answer generation module is used to generate an answer text based on the question q, the document set D, the instruction set C and the model M to obtain a corresponding answer text and feed it back to the current user.

[0071] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0072] The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in the first aspect above;

[0073] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0074] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a computer, the computer executes the instructions of the method described in the first aspect above.

[0075] The embodiment of the present invention provides a retrieval iteration method, device, electronic device and computer-readable storage medium using a generative large language model. As can be seen from the above content, the embodiment of the present invention selects a generative large language model that has completed pre-training, natural language task training and prompt engineering fine-tuning as the corresponding model M for the retrieval system R; and records the instruction set obtained by the model M through the prompt engineering as the instruction set C; and uses the model M and its instruction set C to construct a retrieval iteration process for optimizing the retrieval quality. In this retrieval iteration process, the retrieval system R first uses the question q as the query Q to query the document library G. * Search and get document set D t , and then the model M uses the document sorting instructions in the instruction set C based on the current document set D t Refresh the document set D, and then use the two types of retrieval verification instructions provided by the instruction set C to verify whether the document set D meets the question q. If the verification passes, the iteration is stopped and the model M uses the retrieval enhancement generation instructions provided by the instruction set C to generate the answer text according to the question q and the document set D. If the verification fails, the model M continues to use the five types of pseudo-query generation instructions provided by the instruction set C to expand the query Q and use the document selection instructions to select the document library G. * Expand, and the retrieval system R searches the latest document library G based on the latest query Q * Retrieve and obtain a new document set D t Until the latest document set D passes the verification. The embodiment of the present invention not only improves the retrieval quality of the retrieval system, but also improves the answer generation quality of the large language model itself. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A schematic diagram of an iterative retrieval method using a generative large language model provided in the first embodiment of the present invention;

[0077] Figure 2 A module structure diagram of a retrieval iteration device using a generative large language model provided in the second embodiment of the present invention;

[0078] Figure 3 This is a structural diagram of an electronic device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0079] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0080] The first embodiment of the present invention provides a retrieval iterative method using a generative large language model, such as Figure 1 As shown in the schematic diagram of a retrieval iterative method using a generative large language model provided in the first embodiment of the present invention, the method mainly includes the following steps:

[0081] Step 1: Select a generative large language model that has completed pre-training, natural language task training, and prompt engineering fine-tuning for the retrieval system R as the corresponding model M; and record the instruction set obtained by the model M through the prompt engineering as the instruction set C; and record the information resource library of the retrieval system R as the document library G R , record the information resource library used by model M during training and fine-tuning as document library G M ; and set the corresponding iteration threshold T, retrieval threshold N, subpackaging threshold L, verification mode V, and pseudo query configuration W.

[0082] Here, the retrieval system R of the embodiment of the present invention includes at least a sparse retriever and a dense retriever; wherein, the sparse retriever includes at least a text retriever implemented based on the bag-of-words algorithm, a text retriever implemented based on the TF-IDF algorithm, and a text retriever implemented based on the BM25 algorithm; the dense retriever is a type of text retriever implemented based on a deep learning model, including at least the BERT series model.

[0083] The model M in the embodiment of the present invention includes at least a GPT series model, a T5 series model, and a BAT series model.

[0084] The document library G in the embodiment of the present invention R , G M Each document is composed of multiple documents, each of which corresponds to a unique document number; the document library G M The documents come from multiple information sources. From the training mechanism of the large language model, we know that the document library G M It will be injected into the friction parameters of model M in the same way as training / fine-tuning knowledge injection and directly called by the model.

[0085] The instruction set C of the embodiment of the present invention is an instruction set generated in advance through prompt engineering fine-tuning. The prompt engineering fine-tuning process corresponding to the instruction set does not belong to the technical content of the embodiment of the present invention. The embodiment of the present invention will only directly reference the instruction set. The instruction set C includes at least the following 10 types of customized instructions: document sorting instruction c sor , Retrieve verification instruction c cls , Retrieve verification instruction c sco , pseudo query generation instruction c pse-que , pseudo query generation instruction c pse-pas , pseudo query generation instruction c pse-syn , pseudo query generation instruction c pse-ctx , pseudo query generation instruction cpse-top , document selection instruction c sel , search enhancement generation instruction c rag ; All instructions of instruction set C are formatted natural language prompt templates and each has a set of configurable instruction parameters.

[0086] The functions of the above 10 types of custom instructions are as follows:

[0087] 1) Document sorting instruction c sor The configurable instruction parameters include the problem parameter p q , document set parameter p D ; Document sorting instruction c sor Used to guide the model M to set the document parameter p D The problem parameter p q The document numbers of the k most relevant documents are sorted in descending order of relevance and a corresponding number sorting sequence is generated; the number k is specified as a positive integer;

[0088] 2) Retrieve verification instruction c cls The configurable instruction parameters include the problem parameter p q , document set parameter p D ; Retrieve verification instruction c cls Used to guide the model M to the document set parameter p D Does it fully satisfy the problem parameter p? q Verify the answer condition and generate a specific verification result of yes or no;

[0089] 3) Retrieve verification instruction c sco The configurable instruction parameters include the problem parameter p q , document set parameter p D ; Retrieve verification instruction c sco It is used to guide the model M to adjust the document set parameter p according to the set score range. D For the problem parameter p q Score the answer support and generate the corresponding score;

[0090] 4) Pseudo query generation instruction c pse-que The configurable instruction parameters include the problem parameter p q , document set parameter p D ; Pseudo query generation instruction c pse-que Parameters p used to guide the model M to answer the question q Required but document set parameter p D Infer the missing information that is not covered, design corresponding questions for the inferred missing information, and output the designed questions as the generated pseudo query;

[0091] 5) Pseudo query generation instruction c pse-pas The configurable instruction parameters include the problem parameter p q , document set parameter p D ; Pseudo query generation instruction c pse-pas Parameters p used to guide the model M to answer the question q Required but document set parameter p D Infer the missing information that is not covered, and generate a piece of information description text based on the inferred missing information as the corresponding pseudo query output;

[0092] 6) Pseudo query generation instruction c pse-syn The configurable instruction parameters include the problem parameter p q ; Pseudo query generation instruction c pse-syn Used to guide the model M to the problem parameter p q Generate a set of synonyms or semantically related words as corresponding pseudo-query output;

[0093] 7) Pseudo query generation instruction c pse-ctx The configurable instruction parameters include the problem parameter p q ; Pseudo query generation instruction c pse-ctx Used to guide the model M to the problem parameter p q Generate a background text as the corresponding pseudo query output;

[0094] 8) Pseudo query generation instruction c pse-top The configurable instruction parameters include the problem parameter p q ; Pseudo query generation instruction c pse-top Used to guide the model M to the problem parameter p q Identify the knowledge topics and their knowledge domains, and generate a set of topic texts in the same domain as the corresponding pseudo query outputs for the identified problem topics;

[0095] 9) Document selection instruction c sel The configurable instruction parameters include the problem parameter p q ; Document selection instruction c sel Used to guide model M from document library G M All information sources select a specified number n of documents to form a selected document set, and require that the selected document set must meet the problem parameter p q The solution conditions are as follows: the credibility of all documents in the selected document set exceeds a preset credibility threshold; the content consistency score of all documents in the selected document set exceeds a preset consistency score threshold; there is no semantic contradiction between any two documents in the selected document set; and the corresponding document number set is composed of the n document numbers of the selected document set that are finally confirmed and output; the specified number n is a positive integer; the credibility threshold and the consistency score threshold are two preset threshold parameters;

[0096] 10) Retrieve enhanced generation instruction c rag The configurable instruction parameters include the problem parameter p q , document set parameter p D ; Retrieve enhanced generation instruction c rag Used to guide the model M with document set parameters p D is the hint, and is the question parameter p q Generate a corresponding answer text and output it.

[0097] The verification mode V of the embodiment of the present invention includes a first mode and a second mode; when V is the first mode, it indicates that the model M should be based on the retrieval verification instruction c cls Verify whether the document set fully meets the question answering conditions; when V is the second mode, it means that the model M should be based on the retrieval verification instruction c sco Scoring the question answering support of a document set.

[0098] The pseudo query configuration W of the embodiment of the present invention includes five types of configuration switches w 1、 w 2、 w 3、 w 4、 w5, each type of configuration switch has a value of 0 or 1. If the configuration switch w1 is 1, it means that the model M needs to use the pseudo query generation instruction c pse-que Generate a question for the missing information in the document set and generate a pseudo query. If the configuration switch w2 is 1, it means that the model M needs to use the pseudo query generation instruction c pse-pas Generate a piece of information description text for the missing information in the document set and generate a pseudo query. If the configuration switch w3 is 1, it means that the model M needs to use the pseudo query generation instruction c pse-syn Generate a set of synonyms or semantically related words for the question text and generate a pseudo query. If the configuration switch w4 is 1, it means that the model M needs to use the pseudo query generation instruction c pse-ctx Generate a background description for the question text and generate a pseudo query. If the configuration switch w5 is 1, it means that the model M needs to use the pseudo query generation instruction c pse-top Generate a set of topic texts in the same field for the question text and generate a pseudo query. 1、 w 2、 w 3、 w 4、 If two or more switches w in w5 are simultaneously 1, the two or more pseudo queries obtained can be concatenated into a fused pseudo query containing richer semantic information.

[0099] Step 2: Take the question text entered by the user as question q; and * Set as Document Library G B .

[0100] Here, the current step 2 is a preparatory step before the iterative search process of the embodiment of the present invention. The iterative search process of the embodiment of the present invention consists of the following steps 3-10.

[0101] Step 3: Initialize the counter t to 1, the document set D to be empty, and the query Q to be question q.

[0102] Step 4: Identify whether the document set D is not empty; if so, go to step 5; if not, go to step 6.

[0103] Step 5: Perform pseudo query generation processing based on the pseudo query configuration W, the instruction set C, and the model M, and reset the query Q based on the processing result;

[0104] Specifically including: Step 51, extracting the corresponding configuration switch w from the pseudo query configuration W 1、 w 2、 w 3、 w 4、 w5;

[0105] Step 52, and identify the configuration switch w1; if the configuration switch w1 is 1, then generate the pseudo query instruction c pse-que The problem parameter p q , document set parameter p D Set it as the corresponding question q, document set D, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-que Input model M for processing and use the pseudo query output of this processing as the corresponding pseudo query q 1 ; If the configuration switch w1 is 0, then set the corresponding pseudo query q 1 is empty;

[0106] Step 53, and identify the configuration switch w2; if the configuration switch w2 is 1, the pseudo query generation instruction c pse-pas The problem parameter p q , document set parameter p D Set it as the corresponding question q, document set D, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-pas Input model M for processing and use the pseudo query output of this processing as the corresponding pseudo query q 2 ; If the configuration switch w2 is 0, then set the corresponding pseudo query q 2 is empty;

[0107] Step 54, and identify the configuration switch w3; if the configuration switch w3 is 1, the pseudo query generation instruction c pse-syn The problem parameter p q Set it as the corresponding question q, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-synInput model M for processing and use the pseudo query output of this processing as the corresponding pseudo query q 3 ; If the configuration switch w3 is 0, then set the corresponding pseudo query q 3 is empty;

[0108] Step 55, and identify the configuration switch w4; if the configuration switch w4 is 1, the pseudo query generation instruction c pse-ctx The problem parameter p q Set it as the corresponding question q, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-ctx Input model M for processing and use the pseudo query output of this processing as the corresponding pseudo query q 4 ; If the configuration switch w4 is 0, then set the corresponding pseudo query q 4 is empty;

[0109] Step 56, and identify the configuration switch w5; if the configuration switch w5 is 1, the pseudo query generation instruction c pse-top The problem parameter p q Set it as the corresponding question q, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-top Input model M for processing and use the pseudo query output of this processing as the corresponding pseudo query q 5 ; If the configuration switch w5 is 0, then set the corresponding pseudo query q 5 is empty;

[0110] Step 57, and the pseudo query q 1 ,q 2 ,q 3 ,q 4 ,q 5 Perform text splicing to obtain the corresponding current spliced text; and reset the query Q based on the current spliced text.

[0111] Step 6: The retrieval system R searches the document library G according to the query Q and the retrieval threshold N. * Search and get document set D t ; and divide the document set D into t Divide into multiple subsets D t,sub ; and from the first to the last subset D t,sub Slide step by step, and at each step based on the current subset D t,sub , instruction set C and model M refresh the document set D;

[0112] Specifically, step 61 includes: the retrieval system R searches the document library G according to the query Q and the retrieval threshold N. * Search and get document set D t ;

[0113] Specifically, the retrieval system R compares the query Q with the document library G. * Calculate the relevance of each document in the , and extract the top N documents with the highest relevance to form the corresponding document set D t ;

[0114] Step 62, and divide the document set D into t Divide into multiple subsets D t,sub ;

[0115] Here, the first to the second to last subset D t,sub The number of documents is L, and the last subset D t,sub The number of documents is less than or equal to L;

[0116] Step 63, and from the first to the last subset D t,sub Slide step by step, and at each step based on the current subset D t,sub , instruction set C and model M refresh the document set D;

[0117] Specifically include: Based on the current subset D t,sub The current temporary document set D corresponding to the document set D * =D∪D t,sub ; and sort the documents by c sor The problem parameter p q , document set parameter p D Let the corresponding question q and the temporary document set D be * ; and complete the document sorting instruction c configured with instruction parameters sor Input model M for processing and use the numbered sorting sequence output by this processing as the corresponding numbered sequence of the current time; and use the current temporary document set D * The k documents that meet the current number sequence are extracted to form the corresponding current preferred document set; and the document set D is reset based on the current preferred document set.

[0118] Step 7: Verify whether the document set D satisfies question q according to the verification mode V, instruction set C, and model M to obtain the corresponding current verification result;

[0119] Among them, the current verification results include passed and failed;

[0120] Specifically comprising: step 71, identifying the verification mode V;

[0121] Step 72: If the verification mode V is the first mode, the verification instruction c is retrieved. cls The problem parameter p q , document set parameter p DSet as the corresponding question q, document set D; and complete the retrieval verification instruction c configured with instruction parameters cls Input model M for processing and use the verification result output by this processing as the corresponding first result; and identify the first result; if the first result is yes, set the corresponding current verification result to pass; if the first result is no, set the corresponding current verification result to fail;

[0122] Step 73: If the verification mode V is the second mode, the verification instruction c will be retrieved. sco The problem parameter p q , document set parameter p D Set as the corresponding question q, document set D; and complete the retrieval verification instruction c configured with instruction parameters sco The input model M is processed and the score output by this processing is used as the corresponding first score; and whether the first score exceeds the preset verification score threshold is identified; if it exceeds, the corresponding current verification result is set to pass; if it does not exceed, the corresponding current verification result is set to fail.

[0123] Here, the verification score threshold is a preset threshold parameter.

[0124] Step 8, identify whether the current verification result is failed; if so, add 1 to the counter t and go to step 9; if not, go to step 11.

[0125] Step 9: Identify whether the counter t exceeds the iteration threshold T; if so, go to step 10; if not, return to step 4.

[0126] Step 10, based on document library G M , instruction set C and model M for document library G * Expand multiple information sources; and return to step 3 after completing the expansion;

[0127] Specifically, it includes: step 101, initializing an empty first document set; and sending the document selection instruction c sel The problem parameter p q Let be the corresponding question q;

[0128] Step 102: select the document that has completed the configuration of the command parameters. sel Input model M for processing and use the document number set output by this processing as the corresponding first number set;

[0129] Step 103: Document library G M The n documents that meet the first number set are extracted and added to the first document set; and the document library G in the current first document set is *The non-intersection documents of the current independent document set are composed of the corresponding current independent document set; and whether the number of documents in the current independent document set is less than n is identified; if so, return to step 102; if not, the current independent document set is sent to the document library G * and return to step 3 after the addition is completed.

[0130] Step 11: Generate an answer text based on question q, document set D, instruction set C, and model M to obtain the corresponding answer text and feedback it to the current user;

[0131] Specifically include: enhancing the retrieval to generate instructions c rag The problem parameter p q , document set parameter p D Set the corresponding question q and document set D; and complete the retrieval enhancement generation instruction c with the instruction parameter configuration rag The input model M is processed and the answer text output by this processing is used as the corresponding current answer text to feed back to the current user.

[0132] It should also be noted that the method of the embodiment of the present invention further includes: starting timing when the counter t is set to 1 for the first time; and during the current round of iterations from steps 3 to 10, monitoring in real time whether the current timing duration exceeds a preset iteration duration threshold; and when it is confirmed that the current timing duration exceeds the iteration duration threshold, exiting the current round of iteration and jumping to step 11. Here, the iteration duration threshold is a preset threshold parameter.

[0133] It should also be noted that the method of the embodiment of the present invention further includes: after feeding back the answer text to the current user, receiving the user's satisfaction feedback on the answer text; and when the satisfaction feedback is unsatisfactory, optimizing the current document library G based on the preset document library optimization scheme. * Optimize the current iteration threshold T, retrieval threshold N, subpackaging threshold L, verification mode V, and pseudo query configuration W based on the preset parameter optimization scheme, and return to step 3 at the end of this optimization to perform another round of iteration and generate a new answer text through this round of iteration to feedback to the current user.

[0134] Here, the document library optimization solution of the embodiment of the present invention is a pre-set document library optimization rule / strategy, which can be customized based on the actual application needs; for example, the document library G corresponding to the model M M The documents in the same field as the current question q are loaded into the document library G *The parameter optimization scheme of the embodiment of the present invention is a preset iterative process parameter optimization rule / strategy, which can be customized based on the actual needs of the application; for example, when the iteration time is too long, the iteration threshold T is reduced, the retrieval threshold N and the subpackaging threshold L are increased, the verification mode V is set to the first mode, and the number of switches set to 1 in the pseudo-query configuration W is increased; when the iterative process occupies too many computing resources, the iteration threshold T is reduced, the retrieval threshold N and the subpackaging threshold L are lowered, the verification mode V is set to the first mode, and the number of switches set to 1 in the pseudo-query configuration W is reduced; when the retrieval quality needs to be further improved, the iteration threshold T is increased, the retrieval threshold N and the subpackaging threshold L are increased, the verification mode V is set to the second mode, and the number of switches set to 1 in the pseudo-query configuration W is increased.

[0135] Figure 2 This is a module structure diagram of a retrieval iteration device using a generative large language model provided in the second embodiment of the present invention. The device is a terminal device or server that implements the aforementioned method embodiment, or can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiment. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 2 As shown, the device includes: a model and parameter preparation module 201, an iteration preparation module 202, an iteration initialization module 203, a document set judgment module 204, a query refresh module 205, a document set refresh module 206, a retrieval verification module 207, a verification judgment module 208, an iterative loop module 209, a document library expansion module 210, and an answer generation module 211.

[0136] The model and parameter preparation module 201 is used to select a generative large language model that has completed pre-training, natural language task training and prompt engineering fine-tuning for the retrieval system R as the corresponding model M; and record the instruction set obtained by the model M through the prompt engineering as the instruction set C; and record the information resource library of the retrieval system R as the document library G R , record the information resource library used by model M during training and fine-tuning as document library G M ; and set the corresponding iteration threshold T, retrieval threshold N, subpackaging threshold L, verification mode V, and pseudo query configuration W.

[0137] The iteration preparation module 202 is used to take the question text input by the user as question q; and * Set as Document Library G B .

[0138] The iterative initialization module 203 is used to initialize the counter t to 1, the document set D to be empty, and the query Q to be question q.

[0139] The document set determination module 204 is used to identify whether the document set D is not empty; if so, the process proceeds to the query refresh module 205; if not, the process proceeds to the document set refresh module 206.

[0140] The query refresh module 205 is used to perform pseudo query generation processing based on the pseudo query configuration W, the instruction set C and the model M and reset the query Q based on the processing result.

[0141] The document set refresh module 206 is used for the retrieval system R to update the document library G according to the query Q and the retrieval threshold N. * Search and get document set D t ; and divide the document set D into t Divide into multiple subsets D t,sub ; and from the first to the last subset D t,sub Slide step by step, and at each step based on the current subset D t,sub , instruction set C and model M refresh the document set D; the first to the second to last subset D t,sub The number of documents is L, and the last subset D t,sub The number of documents is less than or equal to L.

[0142] The retrieval verification module 207 is used to verify whether the document set D meets the question q according to the verification mode V, the instruction set C and the model M to obtain the corresponding current verification result; the current verification result includes pass and fail.

[0143] The verification judgment module 208 is used to identify whether the current verification result is failed; if so, it adds 1 to the counter t and turns to the iterative loop module 209; if not, it turns to the answer generation module 211.

[0144] The iterative loop module 209 is used to identify whether the counter t exceeds the iteration threshold T; if so, it goes to the document library expansion module 210; if not, it returns to the document set determination module 204.

[0145] The document library expansion module 210 is used to M , instruction set C and model M for document library G * Perform multi-information source expansion; and return to the iterative initialization module 203 after completing the expansion.

[0146] The answer generation module 211 is used to generate an answer text based on the question q, the document set D, the instruction set C and the model M to obtain the corresponding answer text and feed it back to the current user.

[0147] An embodiment of the present invention provides a retrieval iteration device using a generative large language model, which can execute the method steps in the above method embodiment. Its implementation principles and technical effects are similar and will not be repeated here.

[0148] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by a processing element; they can also all be implemented in the form of hardware; some modules can also be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the model and parameter preparation module can be a separately established processing element, or it can be integrated into a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called by a processing element of the above device to perform the functions of the above-mentioned determination module. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or by instructions in the form of software.

[0149] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0150] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the above method embodiments are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) means. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, hard disk, tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0151] Figure 3 This is a schematic diagram of the structure of an electronic device provided in the third embodiment of the present invention. The electronic device can be a terminal device or server that implements the method of the aforementioned embodiment, or it can be a terminal device or server that implements the method of the aforementioned embodiment connected to the aforementioned terminal device or server. Figure 3 As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303's transceiver actions. Various instructions may be stored in the memory 302 for completing various processing functions and implementing the processing steps described in the aforementioned embodiment method. Preferably, the electronic device involved in the embodiment of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The above-mentioned communication port 306 is used for connecting and communicating between the electronic device and other peripherals.

[0152] exist Figure 3The system bus 305 mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory (Non-Volatile Memory), such as at least one disk storage.

[0153] The above-mentioned processors can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0154] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the methods and processing procedures provided in the above embodiments.

[0155] The embodiment of the present invention provides a retrieval iteration method, device, electronic device and computer-readable storage medium using a generative large language model. As can be seen from the above content, the embodiment of the present invention selects a generative large language model that has completed pre-training, natural language task training and prompt engineering fine-tuning as the corresponding model M for the retrieval system R; and records the instruction set obtained by the model M through the prompt engineering as the instruction set C; and uses the model M and its instruction set C to construct a retrieval iteration process for optimizing the retrieval quality. In this retrieval iteration process, the retrieval system R first uses the question q as the query Q to query the document library G. * Search and get document set D t , and then the model M uses the document sorting instructions in the instruction set C based on the current document set D tRefresh the document set D, and then use the two types of retrieval verification instructions provided by the instruction set C to verify whether the document set D meets the question q. If the verification passes, the iteration is stopped and the model M uses the retrieval enhancement generation instructions provided by the instruction set C to generate the answer text according to the question q and the document set D. If the verification fails, the model M continues to use the five types of pseudo-query generation instructions provided by the instruction set C to expand the query Q and use the document selection instructions to select the document library G. * Expand, and the retrieval system R searches the latest document library G based on the latest query Q * Retrieve and obtain a new document set D t Until the latest document set D passes the verification. The embodiment of the present invention not only improves the retrieval quality of the retrieval system, but also improves the answer generation quality of the large language model itself.

[0156] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0157] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A retrieval iterative method using a generative large language model, characterized in that: The method comprises: Step 1: Select a generative large language model that has completed pre-training, natural language task training, and prompt engineering fine-tuning for the retrieval system R as the corresponding model M; and record the instruction set obtained by the model M through the prompt engineering as the instruction set C; and record the information resource library of the retrieval system R as the document library G R , the information resource library used by the model M during training and fine-tuning is recorded as document library G M ; And set the corresponding iteration threshold T, retrieval threshold N, subpackaging threshold L, verification mode V, and pseudo query configuration W; Step 2: Take the question text entered by the user as question q; and * Set as the document library G B ; Step 3, initialize the counter t to 1, the document set D to be empty, and the query Q to the question q; Step 4: Identify whether the document set D is not empty; if so, go to step 5; if not, go to step 6; Step 5: performing pseudo query generation processing based on the pseudo query configuration W, the instruction set C, and the model M and resetting the query Q based on the processing result; Step 6: The retrieval system R searches the document library G according to the query Q and the retrieval threshold N. * Search and get document set D t ; and divide the document set D into t Divide into multiple subsets D t,sub ; and from the first to the last of said subset D t,sub Slide step by step, and at each step based on the current subset D t,sub , the instruction set C and the model M refresh the document set D; the first to the second to last subset D t,sub The number of documents is L, and the last subset D t,sub The number of documents is less than or equal to L; Step 7: Verify whether the document set D satisfies the question q according to the verification mode V, the instruction set C, and the model M to obtain a corresponding current verification result; the current verification result includes pass and fail; Step 8: Identify whether the current verification result is failed; if so, add 1 to the counter t and go to step 9; if not, go to step 11; Step 9: Identify whether the counter t exceeds the iteration threshold T; if so, go to step 10; if not, return to step 4; Step 10: Based on the document library G M , the instruction set C and the model M to the document library G * Expand multiple information sources; and return to step 3 after completing the expansion; Step 11: Generate an answer text based on the question q, the document set D, the instruction set C, and the model M to obtain the corresponding answer text and feed it back to the current user.

2. The retrieval iterative method using a generative large language model according to claim 1, characterized in that: The retrieval system R includes at least a sparse retriever and a dense retriever; the sparse retriever includes at least a text retriever based on a bag-of-words algorithm, a text retriever based on a TF-IDF algorithm, and a text retriever based on a BM25 algorithm; the dense retriever is a type of text retriever based on a deep learning model, including at least a BERT series model; The model M includes at least a GPT series model, a T5 series model, and a BAT series model; The document library G R , G M Each document is composed of multiple documents, each document corresponds to a unique document number; the document library G M The documents come from multiple information sources; The instruction set C includes document sorting instructions c sor , Retrieve verification instruction c cls , Retrieve verification instruction c sco , pseudo query generation instruction c pse-que , pseudo query generation instruction c pse-pas , pseudo query generation instruction c pse-syn , pseudo query generation instruction c pse-ctx , pseudo query generation instruction c pse-top , document selection instruction c sel , search enhancement generation instruction c rag All instructions of the instruction set C are formatted natural language prompt templates, and each has a set of configurable instruction parameters; The document sorting instruction c sor The configurable instruction parameters include the problem parameter p q , document set parameter p D ; The document sorting instruction c sor Used to guide the model M to set the document set parameter p D The problem parameter p q The document numbers of the k most relevant documents are sorted in descending order of relevance and a corresponding number sorting sequence is generated; the number k is specified as a positive integer; The retrieval verification instruction c cls The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The retrieval verification instruction c cls Used to guide the model M to the document set parameter p D Whether the problem parameter p is fully satisfied q Verify the answer condition and generate a specific verification result of yes or no; The retrieval verification instruction c sco The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The retrieval verification instruction c sco It is used to guide the model M to evaluate the document set parameter p according to the set scoring range. D For the problem parameter p q Score the answer support and generate the corresponding score; The pseudo query generation instruction c pse-que The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The pseudo query generation instruction c pse-que Parameters p used to guide the model M to answer the question q The document set parameter p is required but D Infer the missing information that is not covered, design corresponding questions for the inferred missing information, and output the designed questions as the generated pseudo query; The pseudo query generation instruction c pse-pas The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The pseudo query generation instruction c pse-pas Parameters p used to guide the model M to answer the question q The document set parameter p is required but D Infer the missing information that is not covered, and generate a piece of information description text based on the inferred missing information as the corresponding pseudo query output; The pseudo query generation instruction c pse-syn The configurable instruction parameters include the problem parameter p q ; The pseudo query generation instruction c pse-syn Used to guide the model M to the problem parameter p q Generate a set of synonyms or semantically related words as corresponding pseudo-query output; The pseudo query generation instruction c pse-ctx The configurable instruction parameters include the problem parameter p q ; The pseudo query generation instruction c pse-ctx Used to guide the model M to the problem parameter p q Generate a background text as the corresponding pseudo query output; The pseudo query generation instruction c pse-top The configurable instruction parameters include the problem parameter p q ; The pseudo query generation instruction c pse-top Used to guide the model M to the problem parameter p q Identify the knowledge topics and their knowledge domains, and generate a set of topic texts in the same domain as the corresponding pseudo query outputs for the identified problem topics; The document selection instruction c sel The configurable instruction parameters include the problem parameter p q ; The document selection instruction c sel For guiding the model M from the document library G M All information sources select a specified number n of documents to form a selected document set, and require that the selected document set must meet the problem parameter p q The solution condition is that the credibility of all documents in the selected document set exceeds a preset credibility threshold, the content consistency score of all documents in the selected document set exceeds a preset consistency score threshold, there is no semantic contradiction between any two documents in the selected document set, and the corresponding document number set is composed of the n document numbers of the selected document set that are finally confirmed and output; the specified number n is a positive integer; The retrieval enhancement generation instruction c rag The configurable instruction parameters include the problem parameter p q , the document set parameter p D ; The retrieval enhancement generation instruction c rag Used to guide the model M with the document set parameter p D is a hint, and is the problem parameter p q Generate a corresponding answer text and output; The verification mode V includes a first mode and a second mode; The pseudo query configuration W includes five types of configuration switches w 1、 w 2、 w 3、 w 4、 w5, the value of each configuration switch is 0 or 1.

3. The retrieval iterative method using a generative large language model according to claim 2, characterized in that: The performing pseudo query generation processing based on the pseudo query configuration W, the instruction set C, and the model M and resetting the query Q based on the processing result specifically includes: Extract the corresponding configuration switch w from the pseudo query configuration W 1、 w 2、 w 3、 w 4、 w5; And identify the configuration switch w1; if the configuration switch w1 is 1, the pseudo query generation instruction c pse-que The problem parameter p q , the document set parameter p D Set the corresponding question q and document set D, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-que Input the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 1 If the configuration switch w1 is 0, then set the corresponding pseudo query q 1 is empty; And identify the configuration switch w2; if the configuration switch w2 is 1, the pseudo query generation instruction c pse-pas The problem parameter p q , the document set parameter p D Set the corresponding question q and document set D, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-pas Input the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 2 If the configuration switch w2 is 0, then set the corresponding pseudo query q 2 is empty; And identify the configuration switch w3; if the configuration switch w3 is 1, the pseudo query generation instruction c pse-syn The problem parameter p q Set it as the corresponding question q, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-syn Input the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 3 If the configuration switch w3 is 0, then set the corresponding pseudo query q 3 is empty; And identify the configuration switch w4; if the configuration switch w4 is 1, the pseudo query generation instruction c pse-ctx The problem parameter p q Set it as the corresponding question q, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-ctx Input the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 4 If the configuration switch w4 is 0, then set the corresponding pseudo query q 4 is empty; And identify the configuration switch w5; if the configuration switch w5 is 1, the pseudo query generation instruction c pse-top The problem parameter p q Set it as the corresponding question q, and generate the pseudo query instruction c after completing the instruction parameter configuration pse-top Input the model M for processing and use the pseudo query output from this processing as the corresponding pseudo query q 5 If the configuration switch w5 is 0, then set the corresponding pseudo query q 5 is empty; And the pseudo query q 1 ,q 2 ,q 3 ,q 4 ,q 5 Perform text splicing to obtain a corresponding current spliced text; and reset the query Q based on the current spliced text.

4. The retrieval iterative method using a generative large language model according to claim 2, characterized in that: The retrieval system R searches the document library G according to the query Q and the retrieval threshold N. * Search and get document set D t , specifically including: The retrieval system R compares the query Q with the document library G * The relevance of each document in the , and the top N documents with the highest relevance are extracted to form the corresponding document set D t .

5. The retrieval iterative method using a generative large language model according to claim 2, characterized in that: At each step, the current subset D t,sub , the instruction set C and the model M refresh the document set D, specifically including: Based on the current subset D t,sub The current temporary document set D corresponding to the document set D * =D∪D t,sub ; and sort the document instructions c sor The problem parameter p q , the document set parameter p D Let the corresponding question q and the current temporary document set D be * ; and complete the document sorting instruction c configured with instruction parameters sor Input the model M for processing and use the numbering sequence output by this processing as the corresponding numbering sequence of the current time; and use the temporary document set D of the current time as the corresponding numbering sequence of the current time. * The k documents that meet the current numbering sequence are extracted to form the corresponding current preferred document set; and the document set D is reset based on the current preferred document set.

6. The retrieval iterative method using a generative large language model according to claim 2, characterized in that: Verifying whether the document set D satisfies the question q according to the verification mode V, the instruction set C, and the model M to obtain a corresponding current verification result specifically includes: Identifying the verification mode V; If the verification mode V is the first mode, the retrieval verification instruction c cls The problem parameter p q , the document set parameter p D Set the corresponding question q and the document set D; and complete the retrieval verification instruction c configured with the instruction parameters cls Input the model M for processing and use the verification result output by this processing as the corresponding first result; and identify the first result; if the first result is yes, set the corresponding current verification result to pass; if the first result is no, set the corresponding current verification result to fail; If the verification mode V is the second mode, the retrieval verification instruction c sco The problem parameter p q , the document set parameter p D Set the corresponding question q and the document set D; and complete the retrieval verification instruction c configured with the instruction parameters sco The model M is input for processing and the score output by this processing is used as the corresponding first score; and whether the first score exceeds the preset verification score threshold is identified; if it exceeds, the corresponding current verification result is set to pass; if it does not exceed, the corresponding current verification result is set to fail.

7. The retrieval iterative method using a generative large language model according to claim 2, characterized in that: Based on the document library G M , the instruction set C and the model M to the document library G * Expand multiple information sources, including: Step 71, initialize an empty first document set; and send the document selection instruction c sel The problem parameter p q Let be the corresponding problem q; Step 72: The document selection instruction c that has completed the instruction parameter configuration is sel Input the model M for processing and use the document number set output by this processing as the corresponding first number set; Step 73: the document library G M The n documents that meet the first number set are extracted and added to the first document set; and the document library G in the current first document set is * The non-intersection documents of the current independent document set are composed of the corresponding current independent document set; and whether the number of documents in the current independent document set is less than n is identified; if so, return to step 72; if not, the current independent document set is sent to the document library G * Add in.

8. The retrieval iterative method using a generative large language model according to claim 2, characterized in that: The answer text generation process based on the question q, the document set D, the instruction set C, and the model M is performed to obtain a corresponding answer text and to feed back to the current user, specifically including: The search enhancement generation instruction c rag The problem parameter p q , the document set parameter p D Set the corresponding question q and the document set D; and complete the search enhancement generation instruction c after configuring the instruction parameters. rag The model M is input for processing and the answer text output by this processing is fed back to the current user as the corresponding current answer text.

9. The retrieval iterative method using a generative large language model according to claim 1, characterized in that: The method further comprises: Start timing when the counter t is set to 1 for the first time; and during this round of iteration from step 3 to step 10, monitor in real time whether the current timing duration exceeds the preset iteration duration threshold; and when it is confirmed that the current timing duration exceeds the iteration duration threshold, exit this round of iteration and jump to step 11.

10. The retrieval iterative method using a generative large language model according to claim 1, characterized in that: The method further comprises: After feeding back the answer text to the current user, receiving the user's satisfaction feedback on the answer text; and when the satisfaction feedback is unsatisfactory, optimizing the current document library G based on the preset document library optimization scheme. * Optimize, or optimize the current iteration threshold T, the retrieval threshold N, the subpackaging threshold L, the verification mode V and the pseudo query configuration W based on the preset parameter optimization scheme, and return to step 3 at the end of this optimization to perform another round of iteration and generate a new answer text through this round of iteration to feedback to the current user.

11. A device for executing the iterative retrieval method using a generative large language model according to any one of claims 1 to 10, characterized in that: The device includes: a model and parameter preparation module, an iteration preparation module, an iteration initialization module, a document set judgment module, a query refresh module, a document set refresh module, a retrieval verification module, a verification judgment module, an iteration loop module, a document library expansion module, and an answer generation module; The model and parameter preparation module is used to select a generative large language model that has completed pre-training, natural language task training and prompt engineering fine-tuning for the retrieval system R as the corresponding model M; and record the instruction set obtained by the model M through the prompt engineering as the instruction set C; and record the information resource library of the retrieval system R as the document library G R , the information resource library used by the model M during training and fine-tuning is recorded as document library G M ; And set the corresponding iteration threshold T, retrieval threshold N, subpackaging threshold L, verification mode V, and pseudo query configuration W; The iteration preparation module is used to take the question text input by the user as question q; and * Set as the document library G B ; The iterative initialization module is used to initialize the counter t to 1, the document set D to be empty, and the query Q to the question q; The document set judgment module is used to identify whether the document set D is not empty; if so, it turns to the query refresh module; if not, it turns to the document set refresh module; The query refresh module is used to perform pseudo query generation processing based on the pseudo query configuration W, the instruction set C and the model M and reset the query Q based on the processing result; The document set refreshing module is used for the retrieval system R to update the document library G according to the query Q and the retrieval threshold N. * Search and get document set D t ; and divide the document set D into t Divide into multiple subsets D t,sub ; and from the first to the last of said subset D t,sub Slide step by step, and at each step based on the current subset D t,sub , the instruction set C and the model M refresh the document set D; the first to the second to last subset D t,sub The number of documents is L, and the last subset D t,sub The number of documents is less than or equal to L; The retrieval verification module is used to verify whether the document set D meets the question q according to the verification mode V, the instruction set C and the model M to obtain a corresponding current verification result; the current verification result includes pass and fail; The verification judgment module is used to identify whether the current verification result is failed; if so, it increases the counter t by 1 and switches to the iterative loop module; if not, it switches to the answer generation module; The iterative loop module is used to identify whether the counter t exceeds the iteration threshold T; if so, it switches to the document library expansion module; if not, it returns to the document set determination module; The document library expansion module is used to expand the document library G M , the instruction set C and the model M to the document library G * Perform multi-information source expansion; and return to the iterative initialization module after completing the expansion; The answer generation module is used to generate an answer text based on the question q, the document set D, the instruction set C and the model M to obtain a corresponding answer text and feed it back to the current user.

12. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method according to any one of claims 1 to 10; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is caused to execute the method according to any one of claims 1 to 10.

Images