A retrieval iteration method and device with the aid of a generative large language model

Abstract

The embodiment of the application relates to a retrieval iteration method and device by means of a generative large language model, the method comprising: selecting a generative large language model as a model M for a retrieval system R; recording an instruction set of the model M as an instruction set C; recording an information resource base of the retrieval system R and the model M as a document base G R , G M ; setting iteration parameters; taking a question text input by a user as a question q; performing a round of retrieval iteration by the retrieval system R and the model M to obtain a document set D; and generating an answer text based on the question q, the document set D, the instruction set C and the model M. The retrieval quality and the answer generation quality can be improved by the application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a retrieval iteration method and device based on a generative large language model. BACKGROUND

[0002] In the process of handling intelligent question and answer tasks based on large language models (LLMs), the retrieval augmented generation (RAG) framework is a technical framework for improving the quality of generated answers. Under the RAG framework, a retriever is configured for LLMs, which is used to perform document retrieval on a preset information base with the question text as a query, and LLMs are used to generate corresponding answer text for the question text with the retrieved document as a reference. In principle, the RAG framework can enhance the understanding, reasoning and generation ability of LLMs for input text by using the retrieved document output by the retriever as supplementary knowledge. However, in real applications, some retrievers are limited by their retrieval performance, especially those based on sparse retrieval algorithms (such as bag-of-words algorithm, TF-IDF algorithm, BM25 algorithm, etc.), and the output of the retrieved document cannot fully support answer generation; in this case, the input of the retrieved document to LLMs not only does not play the role of information enhancement, but also may produce useless information interference, resulting in a decline in the quality of the answer text output by the model.

[0003] The natural language task processing ability of LLMs is very powerful, and the prompt engineering based on LLMs can also build a rich set of customized instructions. If LLMs with instruction sets are applied to the retrieval process in reverse, operations such as identifying whether the retrieved document can meet the requirements of answer generation, sorting the relevance of the retrieved document and the question text, generating pseudo queries (such as question rewriting, missing item explanation, synonym replacement, background elaboration, topic expansion, etc.) for the question text to further expand the retrieval range, and expanding the retrieval information base through pre-training knowledge and specific domain knowledge injected by fine-tuning can certainly contribute positively to improving the retrieval quality. However, how to use LLMs and their instruction sets to achieve this is also the technical problem that the present application needs to solve. SUMMARY

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a retrieval iterative method, apparatus, electronic device, and computer-readable storage medium utilizing a generative large language model. This invention selects a generative large language model, M, that has already undergone pre-training, natural language task training, and prompting engineering fine-tuning for the retrieval system R; and denotes the instruction set obtained from model M through prompting engineering as instruction set C; and utilizes model M and its instruction set C to construct a retrieval iterative process for optimizing retrieval quality. In this retrieval iterative process, the retrieval system R first uses the question q as query Q against the document database G. * The document set D was obtained by performing a search. t Then, model M uses document sorting instructions from instruction set C based on the current document set D. t The document set D is refreshed, and then model M uses the two types of retrieval verification instructions provided by instruction set C to verify whether document set D satisfies question q. If the verification passes, the iteration stops, and model M uses the retrieval enhancement generation instructions provided by instruction set C to generate the answer text based on question q and document set D. If the verification fails, model M continues to use the five types of pseudo-query generation instructions provided by instruction set C to expand query Q and uses document selection instructions to process the document library G. * The database is expanded, and the retrieval system R searches the latest document database G based on the latest query Q. * A new document set D was obtained by performing a search. t This continues until the latest document set D passes verification. This invention not only improves the retrieval quality of the retrieval system but also enhances the answer generation quality of the large language model itself.

[0005] To achieve the above objectives, a first aspect of the present invention provides a retrieval iterative method using a generative large language model, the method comprising:

[0006] Step 1: Select a generative large language model M that has completed pre-training, natural language task training, and prompting engineering fine-tuning for the retrieval system R; denote the instruction set obtained by the model M through prompting engineering as instruction set C; and denote the information resource database of the retrieval system R as document database G. R Let the information resource library used by the model M during training and fine-tuning be denoted as document library G. M And set the corresponding iteration threshold T, retrieval threshold N, packet threshold L, verification mode V, and pseudo-query configuration W;

[0007] Step 2: Take the user-inputted question text as question q; and add the document library G. * Let the document library G be... B ;

[0008] Step 3, initialize counter t as 1, document set D as empty, query Q as the question q;

[0009] Step 4, identify whether the document set D is empty; if yes, go to Step 5; if no, go to Step 6;

[0010] Step 5, perform pseudo query generation processing based on the pseudo query configuration W, the instruction set C and the model M and reset the query Q based on the processing result;

[0011] Step 6, retrieve the document set D * from the document base G t according to the query Q and the retrieval threshold N by the retrieval system R; divide the document set D t into multiple subsets D t,sub according to the package threshold L; and slide from the first to the last subset D t,sub , and refresh the document set D t,sub based on the current subset D t,sub , the instruction set C and the model M at each step; the number of documents in the first to the second last subset D t,sub is L, and the number of documents in the last subset D M is less than or equal to L;

[0012] Step 7, verify whether the document set D satisfies the question q according to the verification mode V, the instruction set C and the model M to obtain the corresponding current verification result; the current verification result includes pass and fail;

[0013] Step 8, identify whether the current verification result is fail; if yes, add 1 to the counter t and go to Step 9; if no, go to Step 11;

[0014] Step 9, identify whether the counter t exceeds the iteration threshold T; if yes, go to Step 10; if no, return to Step 4;

[0015] Step 10, expand the document base G M based on the document base G * , the instruction set C and the model M; and return to Step 3 after the expansion is completed;

[0016] Step 11, perform answer text generation processing based on the question q, the document set D, the instruction set C and the model M to obtain the corresponding answer text and feed back to the current user.

[0017] Preferably, the retrieval system R at least includes a sparse retriever and a dense retriever; the sparse retriever at least includes a text retriever implemented based on a bag-of-words algorithm, a text retriever implemented based on a TF-IDF algorithm, and a text retriever implemented based on a BM25 algorithm; the dense retriever is a text retriever implemented based on a deep learning model, and at least includes a BERT series model;

[0018] The model M at least includes a GPT series model, a T5 series model, and a BAT series model;

[0019] The document library G R , G M is composed of a plurality of documents, each document corresponding to a unique document number; the documents of the document library G M come from a plurality of information sources;

[0020] The instruction set C includes document sorting instructions c sor , retrieval verification instructions c cls , retrieval verification instructions c sco , pseudo query generation instructions c pse-que , pseudo query generation instructions c pse-pas , pseudo query generation instructions c pse-syn , pseudo query generation instructions c pse-ctx , pseudo query generation instructions c pse-top , document selection instructions c sel , retrieval enhancement generation instructions c rag ; all instructions of the instruction set C are each a formatted natural language prompt template, and each has a set of configurable instruction parameters;

[0021] The configurable instruction parameters of the document sorting instructions c sor include a question parameter p q and a document set parameter p D ; the document sorting instructions c sor are used to guide the model M to sort the document numbers of the k documents with the highest relevance to the question parameter p D in the document set parameter p q in descending order of relevance and generate a corresponding number sorting sequence; the specified number k is a positive integer;

[0022] The configurable instruction parameters of the retrieval verification instructions c cls include the question parameter p q and the document set parameter p D ; the retrieval verification instructions c cls are used to guide the model M to determine whether the document set parameter p D fully satisfies the question parameter pq the answer condition and generate a check result which is specifically yes or no;

[0023] the search check instruction c sco The configurable instruction parameters of the search check instruction c q include the question parameter p D ; the search check instruction c sco is used to guide the model M to score the support of the document set parameter p D for the answer of the question parameter p q and generate the corresponding score value;

[0024] The configurable instruction parameters of the pseudo query generation instruction c pse-que include the question parameter p q , the document set parameter p D ; the pseudo query generation instruction c pse-que is used to guide the model M to infer the missing information required to answer the question parameter p q but not covered by the document set parameter p D , design a corresponding question for the inferred missing information, and output the designed question as a generated pseudo query;

[0025] The configurable instruction parameters of the pseudo query generation instruction c pse-pas include the question parameter p q , the document set parameter p D ; the pseudo query generation instruction c pse-pas is used to guide the model M to infer the missing information required to answer the question parameter p q but not covered by the document set parameter p D , and generate an information description text based on the inferred missing information as a corresponding pseudo query output;

[0026] The configurable instruction parameters of the pseudo query generation instruction c pse-syn include the question parameter p q ; the pseudo query generation instruction c pse-syn is used to guide the model M to generate a set of synonyms or semantically related words for the question parameter p q as a corresponding pseudo query output;

[0027] The configurable instruction parameters of the pseudo query generation instruction c pse-ctx include the question parameter p q ; the pseudo query generation instruction c pse-ctx is used to guide the model M to generate a set of synonyms or semantically related words for the question parameter pq generating a piece of background text as the corresponding pseudo-query output;

[0028] the configurable instruction parameters of the pseudo-query generation instruction c pse-top include the question parameter p q ; the pseudo-query generation instruction c pse-top is used to guide the model M to identify the knowledge topic of the question parameter p q and its belonging knowledge field, and generate a set of topic texts in the same field as the corresponding pseudo-query output for the identified question topic;

[0029] the configurable instruction parameters of the document selection instruction c sel include the question parameter p q ; the document selection instruction c sel is used to guide the model M to select a specified number n of documents from all information sources of the document library G M to form a selected document set, and requires that the selected document set must meet the answer conditions of the question parameter p q , the credibility of all documents in the selected document set exceeds the preset credibility threshold, the content consistency score of all documents in the selected document set exceeds the preset consistency score threshold, there is no semantic contradiction between any two documents in the selected document set, and the corresponding document number set is composed of the n document numbers of the finally confirmed selected document set and output; the specified number n is a positive integer;

[0030] the configurable instruction parameters of the retrieval enhancement generation instruction c rag include the question parameter p q , the document set parameter p D ; the retrieval enhancement generation instruction c rag is used to guide the model M to generate a corresponding answer text for the question parameter p q as a prompt with the document set parameter p D and output;

[0031] the verification mode V includes first and second modes;

[0032] the pseudo-query configuration W includes five types of configuration switches w 1、 w 2、 w 3、 w 4、 w5, each type of configuration switch has a value of 0 or 1.

[0033] Preferably, the pseudo-query generation processing based on the pseudo-query configuration W, the instruction set C and the model M is performed and the query Q is reset based on the processing result, specifically including:

[0034] Extract the corresponding configuration switch w from the pseudo-query configuration W. 1、 w 2、 w 3、 w 4、 w5;

[0035] The configuration switch w1 is identified; if the configuration switch w1 is 1, the pseudo-query generation instruction c is generated. pse-que The problem parameter p q The document set parameter p D Let q be the corresponding question and D be the document set, and let c be the pseudo-query generation instruction with the completed instruction parameters configured. pse-que The input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 1 If the configuration switch w1 is 0, then the corresponding pseudo-query q is set. 1 Empty;

[0036] The configuration switch w2 is identified; if the configuration switch w2 is 1, the pseudo-query generation instruction c is generated. pse-pas The problem parameter p q The document set parameter p D Let q be the corresponding question and D be the document set, and let c be the pseudo-query generation instruction with the completed instruction parameters configured. pse-pas The input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 2 If the configuration switch w2 is 0, then the corresponding pseudo-query q is set. 2 Empty;

[0037] The configuration switch w3 is identified; if the configuration switch w3 is 1, the pseudo-query generation instruction c is generated. pse-syn The problem parameter p q Let q be the corresponding question, and let c be the pseudo-query generation instruction that completes the instruction parameter configuration. pse-syn The input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 3 If the configuration switch w3 is 0, then the corresponding pseudo-query q is set. 3 Empty;

[0038] The configuration switch w4 is identified; if the configuration switch w4 is 1, the pseudo-query generation instruction c is generated. pse-ctx The problem parameter p q Let q be the corresponding question, and let c be the pseudo-query generation instruction that completes the instruction parameter configuration. pse-ctxThe input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 4 If the configuration switch w4 is 0, then the corresponding pseudo-query q is set. 4 Empty;

[0039] The configuration switch w5 is identified; if the configuration switch w5 is 1, the pseudo-query generation instruction c is generated. pse-top The problem parameter p q Let q be the corresponding question, and let c be the pseudo-query generation instruction that completes the instruction parameter configuration. pse-top The input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 5 If the configuration switch w5 is 0, then the corresponding pseudo-query q is set. 5 Empty;

[0040] And the obtained pseudo-query q 1 q 2 q 3 q 4 q 5 Perform text concatenation to obtain the corresponding current concatenated text; and reset the query Q based on the current concatenated text.

[0041] Preferably, the retrieval system R searches the document database G based on the query Q and the retrieval threshold N. * The document set D was obtained by performing a search. t Specifically, it includes:

[0042] The retrieval system R compares the query Q with the document database G. * The relevance of each document is calculated, and the top N documents with the highest relevance are extracted to form the corresponding document set D. t .

[0043] Preferably, the step is based on the current subset D. t,sub The instruction set C and the model M refresh the document set D, specifically including:

[0044] Based on the current subset D t,sub The document set D corresponds to the current temporary document set D. * =D∪D t,sub ; and the document sorting instruction c sor The problem parameter p q The document set parameter p D Let q be the corresponding question and D be the current temporary document set. * ; and will complete the document sorting instruction c with the configured instruction parameters.sor inputting the model M for processing and taking the numbering sequence output by the current processing as the corresponding current numbering sequence; and taking the current temporary document set D * extracting k documents satisfying the current numbering sequence in the current temporary document set D to form a corresponding current preferred document set; and resetting the document set D based on the current preferred document set.

[0045] Preferably, the verification of whether the document set D satisfies the question q according to the verification mode V, the instruction set C and the model M obtains a corresponding current verification result, and specifically includes:

[0046] identifying the verification mode V;

[0047] if the verification mode V is a first mode, setting the question parameter p cls , the document set parameter p q of the retrieval verification instruction c D to the corresponding question q, the document set D; and setting the retrieval verification instruction c cls with completed instruction parameter configuration to input the model M for processing and taking the verification result output by the current processing as a corresponding first result; and identifying the first result; if the first result is yes, setting the corresponding current verification result as passed; if the first result is no, setting the corresponding current verification result as failed;

[0048] if the verification mode V is a second mode, setting the question parameter p sco , the document set parameter p q of the retrieval verification instruction c D to the corresponding question q, the document set D; and setting the retrieval verification instruction c sco with completed instruction parameter configuration to input the model M for processing and taking the score value output by the current processing as a corresponding first score value; and identifying whether the first score value exceeds a preset verification score threshold; if yes, setting the corresponding current verification result as passed; if no, setting the corresponding current verification result as failed.

[0049] Preferably, the multi-information source expansion of the document library G M based on the document library G * , the instruction set C and the model M specifically includes:

[0050] Step 71, initializing an empty first document set; and setting the question parameter p sel of the document selection instruction c q to the corresponding question q;

[0051] Step 72, the document selection instruction c sel Input the model M for processing and output the document number set as the corresponding first number set;

[0052] Step 73, the document library G M The n documents that meet the first number set are extracted to add to the first document set; and the current independent document set is composed of the non-intersection document group of the document library G * at the current first document set; and identify whether the number of documents in the current independent document set is less than n; if yes, return to step 72; if no, add the current independent document set to the document library G * .

[0053] Preferably, the answer text generated based on the question q, the document set D, the instruction set C and the model M is fed back to the current user, specifically including:

[0054] The problem parameter p rag , the document set parameter p q of the retrieval enhancement generation instruction c D is set to the corresponding question q, the document set D; and the retrieval enhancement generation instruction c rag with completed instruction parameter configuration is input into the model M for processing, and the answer text output by this processing is fed back to the current user as the corresponding current answer text.

[0055] Preferably, the method further comprises:

[0056] Start timing when the counter t is set to 1 for the first time; and in the current iteration process of steps 3 to 10, monitor in real time whether the current timing duration exceeds the preset iteration duration threshold; and when it is confirmed that the current timing duration exceeds the iteration duration threshold, exit the current iteration process and jump to step 11.

[0057] Preferably, the method further comprises:

[0058] After the answer text is fed back to the current user, receive the user's satisfaction feedback on the answer text; and when the satisfaction feedback is not satisfied, optimize the current document library G * based on the preset document library optimization scheme, or optimize the iteration threshold T, the retrieval threshold N, the packaging threshold L, the verification mode V and the pseudo query configuration W based on the preset parameter optimization scheme, and return to step 3 to perform a new iteration and generate a new answer text to feed back to the current user when the optimization is completed.

[0059] A second aspect of the embodiments of the present application provides a device for implementing the retrieval iteration method by means of the generative large language model as described in the first aspect above, the device comprising: a model and parameter preparation module, an iteration preparation module, an iteration initialization module, a document set judgment module, a query refreshing module, a document set refreshing module, a retrieval verification module, a verification judgment module, an iteration loop module, a document library expansion module, and an answer generation module.

[0060] The model and parameter preparation module is configured to select a generative large language model that has completed pre-training, natural language task training, and prompt engineering fine-tuning as the corresponding model M for the retrieval system R; record the instruction set obtained by the model M through prompt engineering as the instruction set C; and record the information resource library of the retrieval system R as the document library G R , record the information resource library used by the model M during training and fine-tuning as the document library G M ; and set the corresponding iteration threshold T, retrieval threshold N, sub-pack threshold L, verification mode V, and pseudo-query configuration W;

[0061] The iteration preparation module is configured to take the problem text input by the user as the problem q; and set the document library G * as the document library G B ;

[0062] The iteration initialization module is configured to initialize the counter t as 1, the document set D as empty, and the query Q as the problem q;

[0063] The document set judgment module is configured to identify whether the document set D is empty; if yes, proceed to the query refreshing module; if no, proceed to the document set refreshing module;

[0064] The query refreshing module is configured to perform pseudo-query generation processing based on the pseudo-query configuration W, the instruction set C, and the model M, and reset the query Q based on the processing result;

[0065] The document set refreshing module is configured to perform retrieval on the document library G * based on the query Q and the retrieval threshold N by the retrieval system R to obtain the document set D t ; divide the document set D t into multiple subsets D t,sub based on the sub-pack threshold L; and slide from the first to the last subset D t,sub , and at each step, refresh the document set D based on the current subset D t,sub , the instruction set C, and the model M; and refresh the document set D based on the first to the second last subset D t,subThe number of documents is L, and the last subset D is... t,sub The number of documents is less than or equal to L;

[0066] The retrieval and verification module is used to verify whether the document set D satisfies the question q according to the verification mode V, the instruction set C, and the model M, and obtain the corresponding current verification result; the current verification result includes pass and fail.

[0067] The verification judgment module is used to identify whether the current verification result is a failure; if yes, the counter t is incremented by 1 and the process is transferred to the iteration loop module; if no, the process is transferred to the answer generation module.

[0068] The iterative loop module is used to identify whether the counter t exceeds the iteration threshold T; if yes, it proceeds to the document library expansion module; if no, it returns to the document set judgment module.

[0069] The document library expansion module is used to expand based on the document library G. M The instruction set C and the model M are related to the document library G. * Perform multi-information source expansion; and return to the iterative initialization module after the expansion is completed;

[0070] The answer generation module is used to generate answer text based on the question q, the document set D, the instruction set C, and the model M to obtain the corresponding answer text and provide feedback to the current user.

[0071] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0072] The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above;

[0073] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0074] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0075] The embodiment of the present application provides a retrieval iteration method, device, electronic equipment and computer readable storage medium by means of a generative large language model. According to the above content, the embodiment of the present application selects a generative large language model which has completed pre-training, natural language task training and prompt engineering fine-tuning as the corresponding model M for the retrieval system R; and records the instruction set obtained by the model M through the prompt engineering as the instruction set C; and uses the model M and the instruction set C to construct a retrieval iteration process for optimizing retrieval quality. In the retrieval iteration process, the retrieval system R first retrieves the document set D by taking the problem q as the query Q to the document library G * t The model M refreshes the document set D based on the current document set D t using the document sorting instruction in the instruction set C, and then checks whether the document set D meets the problem q by using two types of retrieval verification instructions provided by the model M, if the verification is passed, stops iteration and generates an answer text according to the problem q and the document set D by using the retrieval enhancement generation instruction provided by the model M, if the verification is not passed, continues to expand the query Q by using five types of pseudo query generation instructions provided by the model M, expands the document library G * by using the document selection instruction, and retrieves the latest document set D based on the latest query Q to the latest document library G * by the retrieval system R. t until the latest document set D is verified. Through the embodiment of the present application, the retrieval quality of the retrieval system is improved, and the answer generation quality of the large language model itself is also improved. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 A retrieval iteration method by means of a generative large language model provided by the embodiment of the present application;

[0077] Figure 2 A module structure diagram of a retrieval iteration device by means of a generative large language model provided by the embodiment two of the present application;

[0078] Figure 3 A structure schematic diagram of an electronic equipment provided by the embodiment three of the present application. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0080] ​The embodiment one of the present application provides a retrieval iteration method by means of a generative large language model, like Figure 1 The retrieval iteration method by means of a generative large language model provided for the embodiment one of the present application is shown in a schematic diagram, and the method mainly comprises the following steps:

[0081] Step 1, select a generative large language model that has completed pre-training, natural language task training and prompt engineering fine-tuning as the corresponding model M for the retrieval system R; and record the instruction set obtained by the model M through the prompt engineering as instruction set C; and record the information resource library of the retrieval system R as document library G R , record the information resource library used by the model M during training and fine-tuning as document library G M ; and set the corresponding iteration threshold T, retrieval threshold N, package threshold L, verification mode V and pseudo query configuration W.

[0082] Here, the retrieval system R of the embodiment of the present application at least comprises a sparse retriever and a dense retriever; wherein the sparse retriever at least comprises a text retriever based on the bag-of-words algorithm, a text retriever based on the TF-IDF algorithm and a text retriever based on the BM25 algorithm; the dense retriever is a kind of text retriever based on a deep learning model, at least comprising a BERT series model.

[0083] The model M of the embodiment of the present application at least comprises a GPT series model, a T5 series model and a BAT series model.

[0084] The document library G R of the embodiment of the present application M is composed of a plurality of documents, each document corresponds to a unique document number; the documents of the document library G M come from multiple information sources. From the training mechanism of the large language model, we know that the document library G M will be injected into the friction parameters of the model M in the form of training / knowledge injection and be directly called by the model.

[0085] The instruction set C of the embodiment of the present application is an instruction set generated in advance through prompt engineering fine-tuning, the corresponding prompt engineering fine-tuning process does not belong to the technical content of the embodiment of the present application, and the embodiment of the present application only directly quotes the instruction set. The instruction set C at least includes the following 10 types of customized instructions: document sorting instruction c sor , retrieval verification instruction c cls , retrieval verification instruction c sco , pseudo query generation instruction c pse-que , pseudo query generation instruction c pse-pas , pseudo query generation instruction c pse-syn , pseudo query generation instruction c pse-ctx , pseudo query generation instruction cpse-top Document selection command c sel Search enhancement generation command c rag All instructions in the instruction set C are formatted natural language prompt templates and each has a set of configurable instruction parameters.

[0086] The functions of the above 10 types of custom commands are as follows:

[0087] 1) Document sorting command c sor The configurable command parameters include the problem parameter p. q Document set parameter p D Document sorting command c sor The document set parameter p is used to guide model M. D In the problem parameter p q The document numbers of the k most relevant documents are sorted in descending order of relevance, and a corresponding sorted sequence of numbers is generated; the specified number k is a positive integer;

[0088] 2) Retrieve verification command c cls The configurable command parameters include the problem parameter p. q Document set parameter p D ; Search verification command c cls Used to guide model M on document set parameters p D Does the problem parameter p fully satisfy the requirements? q The solution conditions are validated and a specific yes or no validation result is generated.

[0089] 3) Retrieve verification command c sco The configurable command parameters include the problem parameter p. q Document set parameter p D ; Search verification command c sco Used to guide model M to apply the document set parameter p according to the set score range. D Regarding the problem parameter p q The answers are scored based on their support level, and corresponding score values ​​are generated.

[0090] 4) Pseudo-query generation instruction c pse-que The configurable command parameters include the problem parameter p. q Document set parameter p D ; Pseudo-query generation instruction c pse-que Used to guide model M in answering the question parameter p q Required but document set parameter p D Inferring missing information that is not covered, designing corresponding questions for the inferred missing information, and using the designed questions as the output of the generated pseudo-query;

[0091] 5) Pseudo-query generation instruction c pse-pas The configurable command parameters include the problem parameter p. q Document set parameter p D ; Pseudo-query generation instruction c pse-pas Used to guide model M in answering the question parameter p q Required but document set parameter p D Inferring missing information that is not covered, and generating a descriptive text based on the inferred missing information as the corresponding pseudo-query output;

[0092] 6) Pseudo-query generation instruction c pse-syn The configurable command parameters include the problem parameter p. q ; Pseudo-query generation instruction c pse-syn Used to guide model M on problem parameters p q Generate a set of synonyms or semantically related words as the corresponding pseudo-query output;

[0093] 7) Pseudo-query generation instruction c pse-ctx The configurable command parameters include the problem parameter p. q ; Pseudo-query generation instruction c pse-ctx Used to guide the model M, where the problem parameter p is... q Generate a background description text as the corresponding pseudo-query output;

[0094] 8) Pseudo-query generation instruction c pse-top The configurable command parameters include the problem parameter p. q ; Pseudo-query generation instruction c pse-top Used to guide model M on problem parameters p q The system identifies the knowledge topics and their respective knowledge domains, and generates a set of topic texts in the same domain as corresponding pseudo-query outputs for the identified question topics.

[0095] 9) Document selection command c sel The configurable command parameters include the problem parameter p. q Document selection command c sel Used to guide model M from document library G M All information sources are selected to form a document set of a specified number n, and the document set must satisfy the problem parameter p. q The solution conditions are as follows: the credibility of all documents in the selected document set exceeds the preset credibility threshold; the content consistency score of all documents in the selected document set exceeds the preset consistency score threshold; there is no semantic contradiction between any two documents in the selected document set; and the corresponding document number set is composed of the n document numbers of the finally confirmed selected document set and output; the specified quantity n is a positive integer; the credibility threshold and consistency score threshold are two preset threshold parameters.

[0096] 10) Retrieval Enhancement Generation Command c rag The configurable command parameters include the problem parameter p. q Document set parameter p D ; Search enhancement generation command c rag Used to guide model M with document set parameters p D For hints, the question parameter p q Generate and output a corresponding answer text.

[0097] The verification mode V in this embodiment of the invention includes a first mode and a second mode; when V is the first mode, it indicates that model M should be based on the retrieval verification instruction c. cls The document set is checked to ensure it fully meets the question-answering criteria; when V is in the second mode, it indicates that model M should be based on the retrieval verification instruction c. sco Rate the support for the answers to questions in the document set.

[0098] The pseudo-query configuration W in this embodiment of the invention includes five types of configuration switches. 1、 w 2、 w 3、 w 4、 w5, each configuration switch can be either 0 or 1. If configuration switch w1 is 1, it means that model M needs to use the pseudo-query generation instruction c. pse-que Generate a question to address the missing information in the document set, thus generating a pseudo-query. If the configuration switch w2 is set to 1, it means that model M needs to use the pseudo-query generation instruction c. pse-pas This generates a descriptive text for missing information in the document set and thus creates a pseudo-query. If configuration switch w3 is set to 1, it means that model M needs to use the pseudo-query generation instruction c. pse-syn Generate a set of synonyms or semantically related words for the question text, thus generating a pseudo-query. If the configuration switch w4 is set to 1, it means that model M needs to use the pseudo-query generation instruction c. pse-ctx Generate a background description for the question text and thus generate a pseudo-query. If configuration switch w5 is set to 1, it means that model M needs to use the pseudo-query generation instruction c. pse-top Generate a set of topic texts in the same domain for the question text, thus generating a pseudo-query. (If five configuration switches w) 1、 w 2、 w 3、 w 4、 If two or more switches w are simultaneously set to 1 in w5, then the two or more pseudo-queries obtained can be concatenated into a fused pseudo-query containing richer semantic information.

[0099] Step 2: Take the user-inputted question text as question q; and add the document library G. * Set as document library G B .

[0100] Here, step 2 is a preparatory step before the retrieval iteration process in this embodiment of the invention. The retrieval iteration process in this embodiment of the invention consists of steps 3-10 below.

[0101] Step 3: Initialize counter t to 1, document set D to empty, and query Q to question q.

[0102] Step 4: Determine whether document set D is not empty; if yes, proceed to step 5; otherwise, proceed to step 6.

[0103] Step 5: Perform pseudo-query generation processing based on pseudo-query configuration W, instruction set C, and model M, and reset query Q based on the processing result;

[0104] Specifically, this includes: Step 51, extracting the corresponding configuration switch w from the pseudo-query configuration W. 1、 w 2、 w 3、 w 4、 w5;

[0105] Step 52, and identify the configuration switch w1; if the configuration switch w1 is 1, then generate the pseudo-query instruction c. pse-que Problem parameter p q Document set parameter p D Let q be the corresponding question and D be the document set, and let c be the pseudo-query generation instruction that completes the instruction parameter configuration. pse-que The input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 1 If the configuration switch w1 is set to 0, then the corresponding pseudo-query q is set. 1 Empty;

[0106] Step 53, and identify the configuration switch w2; if the configuration switch w2 is 1, then generate the pseudo-query instruction c. pse-pas Problem parameter p q Document set parameter p D Let q be the corresponding question and D be the document set, and let c be the pseudo-query generation instruction that completes the instruction parameter configuration. pse-pas The input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 2 If the configuration switch w2 is set to 0, then the corresponding pseudo-query q is set. 2 Empty;

[0107] Step 54, and identify the configuration switch w3; if the configuration switch w3 is 1, then generate the pseudo-query instruction c. pse-syn Problem parameter p q Let q be the corresponding question, and let c be the pseudo-query generation instruction that completes the instruction parameter configuration. pse-synThe input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 3 If the configuration switch w3 is set to 0, then the corresponding pseudo-query q is set. 3 Empty;

[0108] Step 55, and identify the configuration switch w4; if the configuration switch w4 is 1, then generate the pseudo-query instruction c. pse-ctx Problem parameter p q Let q be the corresponding question, and let c be the pseudo-query generation instruction that completes the instruction parameter configuration. pse-ctx The input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 4 If the configuration switch w4 is set to 0, then the corresponding pseudo-query q is set. 4 Empty;

[0109] Step 56, and identify the configuration switch w5; if the configuration switch w5 is 1, then generate the pseudo-query instruction c. pse-top Problem parameter p q Let q be the corresponding question, and let c be the pseudo-query generation instruction that completes the instruction parameter configuration. pse-top The input model M is processed, and the pseudo-query output from this processing is used as the corresponding pseudo-query q. 5 If the configuration switch w5 is set to 0, then the corresponding pseudo-query q is set. 5 Empty;

[0110] Step 57, and the obtained pseudo-query q 1 q 2 q 3 q 4 q 5 Perform text concatenation to obtain the corresponding current concatenated text; and reset query Q based on the current concatenated text.

[0111] Step 6: The retrieval system R searches the document database G based on query Q and retrieval threshold N. * The document set D was obtained by performing a search. t And based on the sub-packet threshold L, the document set D is divided into... t Divided into multiple subsets D t,sub ; and from the first to the last subset D t,sub Slide gradually, and at each step, based on the current subset D. t,sub Instruction set C and model M refresh document set D once;

[0112] Specifically, this includes: Step 61, where the retrieval system R searches the document database G based on query Q and retrieval threshold N. * The document set D was obtained by performing a search. t ;

[0113] Specifically, this includes: the retrieval system R matching query Q with the document database G. * The relevance of each document is calculated, and the top N documents with the highest relevance are extracted to form the corresponding document set D. t ;

[0114] Step 62, and divide the document set D according to the packet segmentation threshold L. t Divided into multiple subsets D t,sub ;

[0115] Here, the first to the second-to-last subset D t,sub The number of documents is L, and the last subset is D. t,sub The number of documents is less than or equal to L;

[0116] Step 63, and from the first to the last subset D t,sub Slide gradually, and at each step, based on the current subset D. t,sub Instruction set C and model M refresh document set D once;

[0117] Specifically, this includes: based on the current subset D t,sub The document set D corresponds to the current temporary document set D. * =D∪D t,sub ; and the document sorting instruction c sor Problem parameter p q Document set parameter p D Let's set the corresponding question q and the current temporary document set D as... * ; and will complete the document sorting instruction c with the configured command parameters. sor The input model M is processed, and the sorted sequence of numbers output from this processing is used as the corresponding current number sequence; and the current temporary document set D is... * The k documents that satisfy the current number sequence are extracted to form the corresponding current preferred document set; and the document set D is reset based on the current preferred document set.

[0118] Step 7: Verify whether document set D satisfies question q based on verification mode V, instruction set C, and model M to obtain the corresponding current verification result;

[0119] The current verification result includes "passed" and "failed";

[0120] Specifically, this includes: Step 71, identifying the verification pattern V;

[0121] Step 72: If the verification mode V is the first mode, then retrieve the verification command c. cls Problem parameter p q Document set parameter p DSet the corresponding question q and document set D; and complete the retrieval and verification command c with the configured command parameters. cls Input the model M for processing and take the output verification result as the corresponding first result; identify the first result; if the first result is yes, set the corresponding current verification result as pass; if the first result is no, set the corresponding current verification result as fail.

[0122] Step 73: If the verification mode V is the second mode, then retrieve the verification command c. sco Problem parameter p q Document set parameter p D Set the corresponding question q and document set D; and complete the retrieval and verification command c with the configured command parameters. sco The input model M is processed and the score output from this processing is used as the corresponding first score; it is then determined whether the first score exceeds the preset verification score threshold; if it does, the corresponding current verification result is set to pass; if it does not exceed, the corresponding current verification result is set to fail.

[0123] Here, the verification scoring threshold is a pre-set threshold parameter.

[0124] Step 8: Identify whether the current verification result is "failed"; if yes, increment the counter t by 1 and proceed to step 9; if no, proceed to step 11.

[0125] Step 9: Check whether the counter t exceeds the iteration threshold T; if yes, proceed to step 10; if no, return to step 4.

[0126] Step 10, based on document library G M Instruction set C and model M for document library G * Expand the information source by multiple sources; and return to step 3 after the expansion is complete.

[0127] Specifically, this includes: Step 101, initializing an empty first document set; and setting the document selection command c sel Problem parameter p q Let's set it as the corresponding question q;

[0128] Step 102, select the document using the completed command parameter configuration command c. sel Input the model M for processing and use the document number set output from this processing as the corresponding first number set;

[0129] Step 103, transfer the document library G M Extract the n documents that satisfy the first number set and add them to the first document set; and use the document library G in the current first document set. *The non-intersecting documents form the corresponding current independent document set; and it is determined whether the number of documents in the current independent document set is less than n; if yes, return to step 102; if no, the current independent document set is sent to the document library G. * Add to it, and return to step 3 after adding.

[0130] Step 11: Based on question q, document set D, instruction set C, and model M, generate the answer text and provide the corresponding answer text to the current user.

[0131] Specifically, this includes: enhancing the generation of search instructions c rag Problem parameter p q Document set parameter p D Set the corresponding question q and document set D; and complete the retrieval enhancement generation command c with the configured command parameters. rag The input model M is processed, and the output answer text is used as the current answer text to provide feedback to the current user.

[0132] It should also be noted that the method in this embodiment of the invention further includes: starting timing when the counter t is first set to 1; and during the current iteration process from steps 3 to 10, monitoring in real time whether the current timing duration exceeds a preset iteration duration threshold; and when it is confirmed that the current timing duration exceeds the iteration duration threshold, exiting the current iteration process and jumping to step 11. Here, the iteration duration threshold is a preset threshold parameter.

[0133] It should also be noted that the method in this embodiment of the invention further includes: after providing the answer text to the current user, receiving the user's satisfaction feedback on the answer text; and if the satisfaction feedback is unsatisfactory, optimizing the current document library G based on a preset document library optimization scheme. * Optimize the current iteration threshold T, retrieval threshold N, sub-packet threshold L, verification mode V, and pseudo-query configuration W based on a preset parameter optimization scheme. When the current optimization ends, return to step 3 to perform another round of iteration and generate a new answer text to provide feedback to the current user.

[0134] Here, the document library optimization scheme in this embodiment of the invention is a pre-set document library optimization rule / strategy, which can be customized based on actual application needs; for example, the document library G corresponding to model M. M Documents in the same domain as the current problem q are loaded into the document library G. *The parameter optimization scheme of this invention is a pre-set iterative process parameter optimization rule / strategy, which can be customized based on actual application needs. For example, when the iteration time is too long, the iteration threshold T is lowered, the retrieval threshold N and the packet segmentation threshold L are raised, the verification mode V is set to the first mode, and the number of switches set to 1 in the pseudo-query configuration W is increased. When the iteration process consumes too much computing resources, the iteration threshold T is lowered, the retrieval threshold N and the packet segmentation threshold L are lowered, the verification mode V is set to the first mode, and the number of switches set to 1 in the pseudo-query configuration W is reduced. When it is necessary to further improve the retrieval quality, the iteration threshold T is increased, the retrieval threshold N and the packet segmentation threshold L are raised, the verification mode V is set to the second mode, and the number of switches set to 1 in the pseudo-query configuration W is increased.

[0135] Figure 2 This is a module structure diagram of a retrieval iterative device using a generative large language model provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiments, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiments. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 2 As shown, the device includes: a model and parameter preparation module 201, an iteration preparation module 202, an iteration initialization module 203, a document set judgment module 204, a query refresh module 205, a document set refresh module 206, a retrieval verification module 207, a verification judgment module 208, an iteration loop module 209, a document library expansion module 210, and an answer generation module 211.

[0136] The model and parameter preparation module 201 is used to select a generative large language model that has completed pre-training, natural language task training, and prompting engineering fine-tuning for the retrieval system R as the corresponding model M; and to denote the instruction set obtained by model M through prompting engineering as instruction set C; and to denote the information resource database of the retrieval system R as document database G. R Let the information resource library used by model M during training and fine-tuning be denoted as document library G. M And set the corresponding iteration threshold T, retrieval threshold N, sub-packet threshold L, verification mode V, and pseudo-query configuration W.

[0137] Iteration preparation module 202 is used to take the user-input question text as question q; and to prepare the document library G. * Set as document library G B .

[0138] The iterative initialization module 203 is used to initialize the counter t to 1, the document set D to be empty, and the query Q to be the question q.

[0139] The document set judgment module 204 is used to identify whether the document set D is not empty; if yes, it is transferred to the query refresh module 205; if no, it is transferred to the document set refresh module 206.

[0140] The query refresh module 205 is used to generate pseudo queries based on the pseudo query configuration W, instruction set C and model M, and reset the query Q based on the processing results.

[0141] The document set refresh module 206 is used by the retrieval system R to refresh the document database G based on the query Q and the retrieval threshold N. * The document set D was obtained by performing a search. t And based on the sub-packet threshold L, the document set D is divided into... t Divided into multiple subsets D t,sub ; and from the first to the last subset D t,sub Slide gradually, and at each step, based on the current subset D. t,sub Instruction set C and model M refresh document set D once; from the first to the second-to-last subset D. t,sub The number of documents is L, and the last subset is D. t,sub The number of documents is less than or equal to L.

[0142] The retrieval and verification module 207 is used to verify whether the document set D satisfies question q according to the verification mode V, instruction set C and model M to obtain the corresponding current verification result; the current verification result includes pass and fail.

[0143] The verification and judgment module 208 is used to identify whether the current verification result is unsuccessful; if so, the counter t is incremented by 1 and the process is transferred to the iteration loop module 209; if not, the process is transferred to the answer generation module 211.

[0144] The iteration loop module 209 is used to identify whether the counter t exceeds the iteration threshold T; if yes, it proceeds to the document library expansion module 210; if no, it returns to the document set judgment module 204.

[0145] Document library extension module 210 is used for document library G M Instruction set C and model M for document library G * Perform multi-source expansion; and return to the iterative initialization module 203 after the expansion is completed.

[0146] The answer generation module 211 is used to generate answer text based on question q, document set D, instruction set C and model M to obtain the corresponding answer text and provide feedback to the current user.

[0147] The retrieval iteration device provided in this embodiment of the invention, which utilizes a generative large language model, can execute the method steps described in the above method embodiments. Its implementation principle and technical effects are similar, and will not be repeated here.

[0148] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the model and parameter preparation module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0149] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0150] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0151] Figure 3 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 3 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0152] exist Figure 3The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0153] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0154] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0155] This invention provides a retrieval iterative method, apparatus, electronic device, and computer-readable storage medium utilizing a generative large language model. As described above, this invention selects a generative large language model M that has undergone pre-training, natural language task training, and prompting engineering fine-tuning for the retrieval system R; the instruction set obtained from prompting engineering of model M is denoted as instruction set C; and a retrieval iterative process for optimizing retrieval quality is constructed using model M and its instruction set C. In this retrieval iterative process, the retrieval system R first uses question q as query Q against the document database G. * The document set D was obtained by performing a search. t Then, model M uses document sorting instructions from instruction set C based on the current document set D. tThe document set D is refreshed, and then model M uses the two types of retrieval verification instructions provided by instruction set C to verify whether document set D satisfies question q. If the verification passes, the iteration stops, and model M uses the retrieval enhancement generation instructions provided by instruction set C to generate the answer text based on question q and document set D. If the verification fails, model M continues to use the five types of pseudo-query generation instructions provided by instruction set C to expand query Q and uses document selection instructions to process the document library G. * The database is expanded, and the retrieval system R searches the latest document database G based on the latest query Q. * A new document set D was obtained by performing a search. t This continues until the latest document set D passes verification. Through these embodiments, the retrieval quality of the retrieval system is improved, as is the answer generation quality of the large language model itself.

[0156] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0157] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A retrieval iteration method with the help of a generative large language model, characterized in that, The method comprises: Step 1, for the retrieval system R, select a generative large language model that has completed pre-training, natural language task training and prompt engineering fine-tuning as the corresponding model M; and record the instruction set obtained by the prompt engineering of the model M as instruction set C; and record the information resource library of the retrieval system R as document library G R , record the information resource library used by the model M during training and fine-tuning as document library G M ; and set the corresponding iteration threshold T, retrieval threshold N, package threshold L, verification mode V, and pseudo query configuration W; Step 2, take the user input question text as question q; and take the document library G * Set as the document library G B ; Step 3, initializing a counter t as 1, a document set D as empty, and a query Q as the question q; Step 4, identifying whether the document set D is empty; if yes, proceeding to step 5; if no, proceeding to step 6; Step 5, performing pseudo query generation processing based on the pseudo query configuration W, the instruction set C, and the model M, and resetting the query Q based on the processing result; Step 6: The retrieval system R searches the document database G based on the query Q and the retrieval threshold N. * The document set D was obtained by performing a search. t ; and according to the packet segmentation threshold L, the document set D is divided t Divided into multiple subsets D t,sub ; and the subset D described from the first to the last t,sub Slide gradually, and at each step based on the currently stated subset D. t,sub The instruction set C and the model M refresh the document set D once; the first to the second to last subset D t,sub The number of documents is L, and the last subset D is... t,sub The number of documents is less than or equal to L; Step 7, verifying whether the document set D satisfies the question q according to the verification mode V, the instruction set C, and the model M to obtain a corresponding current verification result; the current verification result comprises pass and fail; Step 8, identifying whether the current verification result is fail; if yes, adding 1 to the counter t and proceeding to step 9; if no, proceeding to step 11; Step 9, identifying whether the counter t exceeds the iteration threshold T; if yes, proceeding to step 10; if no, returning to step 4; Step 10, based on the document base G M , the instruction set C and the model M to the document base G * perform multi-information source expansion; and return to step 3 after the expansion is completed; Step 11, performing answer text generation processing based on the question q, the document set D, the instruction set C, and the model M to obtain a corresponding answer text and feeding back the answer text to a current user.

2. The retrieval iteration method by means of a generative large language model according to claim 1, characterized in that: the retrieval system R at least comprises a sparse retriever and a dense retriever; the sparse retriever at least comprises a text retriever based on a bag-of-words algorithm, a text retriever based on a TF-IDF algorithm, and a text retriever based on a BM25 algorithm; the dense retriever is a text retriever based on a deep learning model, at least comprising a BERT series model; the model M at least comprises a GPT series model, a T5 series model, and a BAT series model; The document library G R , G M Each document corresponds to a unique document number; the document library G M The documents of the document library G come from multiple information sources; The instruction set C includes document ordering instruction c sor , search verification instruction c cls , search verification instruction c sco , pseudo query generation instruction c pse-que , pseudo query generation instruction c pse-pas , pseudo query generation instruction c pse-syn , pseudo query generation instruction c pse-ctx , pseudo query generation instruction c pse-top , document selection instruction c sel , search enhancement generation instruction c rag All instructions of the instruction set C are each a formatted natural language prompt template and each has a set of configurable instruction parameters; The document sorting instruction c sor The configurable instruction parameters of the document sorting instruction c q include a question parameter p D ; the document sorting instruction c sor is used for guiding the model M to sort the document numbers of the k documents with the highest relevance to the question parameter p D in the document set parameter p q in descending order of relevance and generate a corresponding number sorting sequence; the specified number k is a positive integer; The search verification instruction c cls The configurable instruction parameters of the search verification instruction c q include the problem parameter p D The document set parameter p cls The search verification instruction c D is used to guide the model M to verify whether the solution condition of the problem parameter p q is completely satisfied by the document set parameter p q and generate a verification result which is specifically yes or no. The retrieval verification command c sco The configurable instruction parameters include the problem parameter p. q The document set parameter p D The retrieval verification command c sco This is used to guide the model M to apply the document set parameter p according to the set score range. D Regarding the problem parameter p q The answers are scored based on their support level, and corresponding score values ​​are generated. The pseudo-query generation instruction c pse-que The configurable instruction parameters of the pseudo-query generation instruction c q include the question parameter p D , the document set parameter p pse-que ; the pseudo-query generation instruction c pse-que is used to guide the model M to infer the missing information required for answering the question parameter p q but not covered by the document set parameter p D , design a corresponding question problem for the inferred missing information, and output the designed question problem as a generated pseudo-query; The pseudo-query generation instruction c pse-pas The configurable instruction parameters include the problem parameter p. q The document set parameter p D The pseudo-query generation instruction c pse-pas Used to guide the model M in answering the question parameter p q The required document set parameter p D Inferring missing information that is not covered, and generating a descriptive text based on the inferred missing information as the corresponding pseudo-query output; The pseudo-query generation instruction c pse-syn The configurable instruction parameters of the pseudo-query generation instruction c q The pseudo-query generation instruction c pse-syn The pseudo-query generation instruction c q generates a set of synonyms or semantically related words as corresponding pseudo-query outputs; The pseudo-query generation instruction c pse-ctx The configurable instruction parameter of the pseudo-query generation instruction c q includes the question parameter p pse-ctx ; The pseudo-query generation instruction c q is used to guide the model M to generate a piece of background description text as the corresponding pseudo-query output for the question parameter p The pseudo-query generation instruction c pse-top The configurable instruction parameters of the pseudo-query generation instruction c q include the question parameter p pse-top ; The pseudo-query generation instruction c q is used to guide the model M to identify the knowledge topic of the question parameter p q and the knowledge field to which it belongs, and generate a set of same-field topic texts as the corresponding pseudo-query output for the identified question topic. The document selection instruction c sel The configurable instruction parameter of the problem parameter p q ; The document selection instruction c sel For guiding the model M to select a specified number n of documents from all information sources of the document library G M Form a selection document set and require that the selection document set must meet the solution conditions of the problem parameter p q , the credibility of all documents of the selection document set exceeds the preset credibility threshold, the content consistency score of all documents of the selection document set exceeds the preset consistency score threshold, there is no semantic contradiction between each two documents of the selection document set, and the corresponding document number set is composed of the n document numbers of the finally confirmed selection document set and output; The specified number n is a positive integer; The search enhancement generation instruction c rag The configurable instruction parameters of the search enhancement generation instruction c q The problem parameter p D The document set parameter p The search enhancement generation instruction c rag For guiding the model M with the document set parameter p D For prompting, the question parameter p q Generate a corresponding answer text and output; the verification mode V comprises a first mode and a second mode; The pseudo-query configuration W includes five types of configuration switches w 1、 w 2、 w 3、 w 4、 w5, each of which is 0 or 1.

3. The retrieval iteration method with a generative large language model according to claim 2, characterized in that, the pseudo query generation processing based on the pseudo query configuration W, the instruction set C, and the model M and the resetting of the query Q based on the processing result specifically comprise: extracting a corresponding configuration switch w from the pseudo query configuration W 1、 w 2、 w 3、 w 4、 w5; and identifying the configuration switch w1; if the configuration switch w1 is 1, setting the pseudo query generation instruction c pse-que with the problem parameter p q , the document set parameter p D corresponding to the problem q and the document set D, and setting the pseudo query generation instruction c pse-que with the completion instruction parameter configuration; inputting the model M for processing and taking the pseudo query output by this processing as the corresponding pseudo query q 1 ; if the configuration switch w1 is 0, setting the corresponding pseudo query q 1 as empty. and identifying the configuration switch w2; if the configuration switch w2 is 1, setting the pseudo query generation instruction c pse-pas with the problem parameter p q , the document set parameter p D corresponding to the problem q and the document set D, and setting the pseudo query generation instruction c pse-pas with the completion instruction parameter configuration; inputting the model M for processing and outputting the pseudo query generated by this processing as the corresponding pseudo query q 2 ; if the configuration switch w2 is 0, setting the corresponding pseudo query q 2 as empty. and the configuration switch w3 is identified; if the configuration switch w3 is 1, the pseudo query generation instruction c pse-syn is set as the question parameter p q corresponding to the question q, and the pseudo query generation instruction c pse-syn with the completed instruction parameter configuration is input into the model M for processing, and the pseudo query output by this processing is taken as the corresponding pseudo query q 3 ; if the configuration switch w3 is 0, the corresponding pseudo query q 3 is set as empty. and the configuration switch w4 is identified; if the configuration switch w4 is 1, the pseudo query generation instruction c pse-ctx is set as the question parameter p q corresponding to the question q, and the pseudo query generation instruction c pse-ctx with the completed instruction parameter configuration is input into the model M for processing, and the pseudo query output by this processing is taken as the corresponding pseudo query q 4 ; if the configuration switch w4 is 0, the corresponding pseudo query q 4 is set as empty. and the configuration switch w5 is identified; if the configuration switch w5 is 1, the pseudo query generation instruction c pse-top is set as the question parameter p q corresponding to the question q, and the pseudo query generation instruction c pse-top with the completed instruction parameter configuration is input into the model M for processing, and the pseudo query output by this processing is taken as the corresponding pseudo query q 5 ; if the configuration switch w5 is 0, the corresponding pseudo query q 5 is set as empty. And the obtained pseudo query q 1 , q 2 , q 3 , q 4 , q 5 Text splicing is performed to obtain the corresponding current spliced text; and the query Q is reset based on the current spliced text.

4. The retrieval iteration method with a generative large language model according to claim 2, characterized in that, said retrieving documents D from said document repository G by said retrieval system R according to said query Q and said retrieval threshold N * t , specifically comprising:​ The relevance of the query Q to each document in the document library G is calculated by the retrieval system R, and the top N documents with the highest relevance are extracted to form the corresponding document set D * t .​ 5. The retrieval iteration method with a generative large language model according to claim 2, characterized in that, said at each step being based on the current said subset D t,sub , said instruction set C and said model M performing one refresh of said document set D, in particular comprising: based on the current subset D t,sub and the document set D, set up a corresponding temporary document set D * t t,sub ; and set the problem parameter p sor of the document sorting instruction c q , the document set parameter p D to the corresponding problem q, the temporary document set D * t ; and input the document sorting instruction c sor configured with the completion instruction parameter into the model M for processing and take the numbering sequence output by this processing as the corresponding temporary numbering sequence; and extract the k documents in the temporary document set D * t that meet the temporary numbering sequence to form a corresponding temporary preferred document set; and reset the document set D based on the temporary preferred document set.

6. The retrieval iteration method with a generative large language model according to claim 2, characterized in that, the verification of whether the document set D satisfies the question q according to the verification mode V, the instruction set C, and the model M to obtain a corresponding current verification result specifically comprises: identifying the verification mode V; If the verification mode V is the first mode, the search verification instruction c cls The problem parameter p q , the document set parameter p D Corresponding to the problem q, the document set D; and the search verification instruction c cls Input the model M for processing and output the verification result of this processing as the corresponding first result; and identify the first result; if the first result is yes, set the corresponding current verification result as passed; if the first result is no, set the corresponding current verification result as failed; If the verification mode V is the second mode, the search verification instruction c sco is set as the corresponding problem parameter p q , the document set parameter p D corresponding to the problem q and the document set D; and the search verification instruction c sco is input into the model M for processing, and the score value output by the processing is taken as the corresponding first score value; and it is identified whether the first score value exceeds a preset verification score threshold; if yes, the corresponding current verification result is set as passed; if no, the corresponding current verification result is set as failed.

7. The retrieval iteration method with a generative large language model according to claim 2, characterized in that, The document library G is based on the document library G M , the instruction set C and the model M to the document library G * Carry out multi-information source expansion, specifically including: Step 71, initialize an empty first document set; and set the document selection instruction c sel the problem parameter p q to the corresponding problem q; Step 72, the document selection instruction c is selected to complete the instruction parameter configuration sel Input the model M for processing and output the document number set of this processing as the corresponding first number set; Step 73, extract n documents from the document library G M that satisfy the first number set to add to the first document set; and the current first document set is composed of the non-intersection document set of the document library G * , and identify whether the number of documents in the current independent document set is less than n; if yes, return to step 72; if no, add the current independent document set to the document library G * .

8. The retrieval iteration method with a generative large language model according to claim 2, characterized in that, the answer text generation processing based on the question q, the document set D, the instruction set C, and the model M to obtain a corresponding answer text and feeding back the answer text to a current user specifically comprises: The retrieval enhancement generation instruction c rag The problem parameter p q The document set parameter p D Corresponding to the problem q, the document set D; and the retrieval enhancement generation instruction c rag Input the model M for processing and output the answer text of this processing as the corresponding current answer text to the current user feedback.

9. The retrieval iteration method with a generative large language model according to claim 1, characterized in that, The method further comprises: starting timing when the counter t is set to 1 for the first time; and monitoring in real time whether a current timing duration exceeds a preset iteration duration threshold during the current iteration process from step 3 to step 10; and when it is confirmed that the current timing duration exceeds the iteration duration threshold, exiting the current iteration process and jumping to step 11.

10. The retrieval iteration method with a generative large language model according to claim 1, characterized in that, The method further comprises: After the answer text is fed back to the current user, the satisfaction feedback of the user to the answer text is received; and when the satisfaction feedback is not satisfied, the current document library G is optimized based on a preset document library optimization scheme * optimization is performed, or the current iteration threshold T, the retrieval threshold N, the packaging threshold L, the verification mode V and the pseudo query configuration W are optimized based on a preset parameter optimization scheme, and step 3 is returned to perform a new round of iteration and generate new answer text to feed back to the current user through the new round of iteration.

11. An apparatus for performing the retrieval iteration method by means of a generative large language model according to any one of claims 1-10, characterized in that, The device comprises a model and parameter preparation module, an iteration preparation module, an iteration initialization module, a document set judgment module, a query refreshing module, a document set refreshing module, a retrieval verification module, a verification judgment module, an iteration loop module, a document library expansion module, and an answer generation module. The model and parameter preparation module is used for selecting a generative large language model that has completed pre-training, natural language task training and prompt engineering fine-tuning as a corresponding model M for a retrieval system R; and recording an instruction set obtained by the model M through prompt engineering as an instruction set C; and recording an information resource base of the retrieval system R as a document base G R , recording an information resource base used by the model M during training and fine-tuning as a document base G M ; and setting a corresponding iteration threshold T, a retrieval threshold N, a subpackage threshold L, a verification mode V and a pseudo query configuration W; The iteration preparation module is configured to take the user input question text as a question q; and take the document library G * is set as the document library G B ; The iteration initialization module is configured to initialize a counter t as 1, a document set D as empty, and a query Q as the question q; The document set judgment module is configured to identify whether the document set D is empty; if not, the query refreshing module is entered; if yes, the document set refreshing module is entered; The query refreshing module is configured to perform pseudo query generation processing based on the pseudo query configuration W, the instruction set C, and the model M, and reset the query Q based on the processing result; The document set refresh module is used by the retrieval system R to refresh the document library G based on the query Q and the retrieval threshold N. * The document set D was obtained by performing a search. t ; and according to the packet segmentation threshold L, the document set D is divided t Divided into multiple subsets D t,sub ; and the subset D described from the first to the last t,sub Slide gradually, and at each step based on the currently stated subset D. t,sub The instruction set C and the model M refresh the document set D once; the first to the second to last subset D t,sub The number of documents is L, and the last subset D is... t,sub The number of documents is less than or equal to L; The retrieval verification module is configured to verify whether the document set D satisfies the question q based on the verification mode V, the instruction set C, and the model M, to obtain a corresponding current verification result; the current verification result includes pass and fail; The verification judgment module is configured to identify whether the current verification result is fail; if yes, the counter t is incremented by 1 and the iteration loop module is entered; if no, the answer generation module is entered; The iteration loop module is configured to identify whether the counter t exceeds the iteration threshold T; if yes, the document library expansion module is entered; if no, the document set judgment module is returned to; The document library expansion module is used for expanding the document library G M based on the instruction set C and the model M * ; and returning the iteration initialization module after completing the expansion. The answer generation module is configured to generate an answer text based on the question q, the document set D, the instruction set C, and the model M, to obtain a corresponding answer text, and feed back the answer text to a current user.

12. An electronic device, comprising: Comprise: a memory, a processor, and a transceiver; The processor is configured to be coupled with the memory, read and execute instructions in the memory, to realize the method in any one of claims 1-10; The transceiver is coupled with the processor, and the transceiver is controlled by the processor to perform message transceiving.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, when the computer instructions are executed by a computer, the computer executes the method in any one of claims 1-10. The computer readable storage medium stores computer instructions, when the computer instructions are executed by a computer, the computer executes the method in any one of claims 1-10.

Images