A dynamic prompt word example recall method and system based on reinforcement learning

Abstract

The present invention belongs to the technical field of large language models, and discloses a dynamic prompt word example recall method and system based on reinforcement learning. The method comprises: constructing a prompt word example selector and a prompt word quality assessor at the same time; introducing a reinforcement learning algorithm to iteratively train the prompt word example selector with the prompt word quality assessor as an evaluator until the expected number of iterations is reached to obtain an optimal prompt word example selector; obtaining a generation task to dynamically select prompt word examples from an example library based on the optimal prompt word example selector; wherein any prompt word example in the example library includes two parts: historical context and expected output; obtaining a number of prompt word examples from the example library in sequence until the probability output by the optimal prompt word example selector is less than a preset probability threshold. The present invention ensures the rationality of the model output by improving the accuracy of prompt word example selection.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to a method and system for recalling dynamic prompt word examples based on reinforcement learning. Background Art

[0002] The large language model is a machine model that generates content based on input prompt words. In order to ensure the accuracy of the model generation results and model output effects, the model will algorithmically process several prompt word examples input by the user to generate the final prompt word for the model's content generation.

[0003] However, the inventors found in the actual application of large language models that even if there is an algorithm to process the prompt word examples to obtain prompt words that the final model can recognize, the output content of the model often cannot meet the output expectations. The reason is that the prompt word examples of existing large language models are mostly constructed subjectively by humans, and thus conform to human language habits. However, large language models are machine models and do not necessarily follow human language laws. Therefore, new methods are needed to construct prompt word examples that are more in line with the model distribution to improve the performance of the model. Summary of the invention

[0004] The present invention aims to provide a method and system for recalling dynamic prompt word examples based on reinforcement learning, so as to improve the technical problem that the output accuracy in the current large language model application cannot be guaranteed due to the application of prompt word examples.

[0005] To achieve the above object, the present invention proposes the following technical solutions:

[0006] In the first aspect, the technical solution provides a dynamic prompt word example recall method based on reinforcement learning. It includes:

[0007] At the same time, a prompt word example selector and a prompt word quality assessor are constructed;

[0008] The prompt word example selector is a neural network, and the corresponding expression is: ;in, S (P i, E i+1 ) indicates that when there is a prompt word P obtained for the i-th time i When , should we continue to select the i+1th prompt word example E i+1 The probability of joining the example set, h0 is to continue to select the i+1th prompt word example E i+1 The probability of h1 not continuing to select the i+1th prompt word example E i+1 The probability of ; where h0+h1=1;

[0009] The prompt word quality assessor is a neural network, and the corresponding expression is: ; Among them, | D Data |For the dataset D Data The number of data in LLM(P i ) is the prompt word P obtained for the i-th time i The actual output of the corresponding large language model, Y i is the prompt word P obtained for the i-th time i The expected output of the corresponding large language model, L (LLM(P i ),Y i ) is used to measure whether the actual output of the large model matches the expected output. L (LLM(P i ),Y i ) has a value of 1, if it does not match L (LLM(P i ),Y i ) is 0;

[0010] Introducing a reinforcement learning algorithm to iteratively train the prompt word example selector using the prompt word quality evaluator as an evaluator until the expected number of iterations is reached to obtain an optimal prompt word example selector;

[0011] A generation task is obtained to dynamically select a prompt word example from an example library based on the optimal prompt word example selector; wherein any prompt word example in the example library includes two parts: historical context and expected output; and a number of prompt word examples are obtained from the example library in sequence until the probability output by the optimal prompt word example selector is less than a preset probability threshold.

[0012] Furthermore, the acquisition generation task includes:

[0013] A final example set is formed based on the finally selected several prompt word examples;

[0014] Inputting the final example set into a pre-built prompt word template to generate a final prompt word;

[0015] The prompt word template is a dynamic template, and the corresponding expression is: i = P (e i ), where e i is the sample set obtained for the i-th time.

[0016] Furthermore, after inputting the final example set into a pre-built prompt word template to generate a final prompt word, the method further includes:

[0017] Inputting the final prompt word into a large language model to generate generated content corresponding to the generation task;

[0018] When the technician determines that the generated content does not meet the expected needs, the optimal prompt word example selector is optimized again based on the reinforcement learning algorithm.

[0019] Furthermore, the acquisition generation task includes:

[0020] A final example set is formed based on the finally selected several prompt word examples;

[0021] Based on user-defined requirements, the final example set is updated to obtain a target example set;

[0022] A target prompt word is generated based on the target example set.

[0023] Furthermore, the acquisition generation task includes:

[0024] Based on the finally selected several prompt word examples, a final example set is formed and stored;

[0025] When it is determined that there is a newly generated task that is similar to the generated task, a first query request is sent to the client front end to inquire whether to call the final example set for reference; or when it is determined that there is a newly generated task that is the same as the generated task, a second query request is sent to the client front end to inquire whether to call the final example set for final prompt word generation.

[0026] In the second aspect, this technical solution provides a dynamic prompt word example recall system based on reinforcement learning. It includes:

[0027] A first building module, used to simultaneously build a cue word example selector and a cue word quality assessor;

[0028] The prompt word example selector is a neural network, and the corresponding expression is: ;in, S (P i, E i+1 ) indicates that when there is a prompt word P obtained for the i-th time i When , should we continue to select the i+1th prompt word example E i+1 The probability of adding h0 is to continue to select the i+1th prompt word example E i+1 The probability of h1 not continuing to select the i+1th prompt word example Ei+1 The probability of ; where h0+h1=1;

[0029] The prompt word quality assessor is a neural network, and the corresponding expression is: ; Among them, | D Data |For the dataset D Data The number of data in LLM(P i ) is the prompt word P obtained for the i-th time i The actual output of the corresponding large language model, Y i is the prompt word P obtained for the i-th time i The expected output of the corresponding large language model, L (LLM(P i ),Y i ) is used to measure whether the actual output of the large model matches the expected output. L (LLM(P i ),Y i ) has a value of 1, if it does not match L (LLM(P i ),Y i ) is 0;

[0030] A first optimization module is used to introduce a reinforcement learning algorithm to iteratively train the prompt word example selector using the prompt word quality evaluator as an evaluator until a desired number of iterations is reached to obtain an optimal prompt word example selector;

[0031] A selection module is used to obtain a generation task to dynamically select a prompt word example from an example library based on the optimal prompt word example selector; wherein any prompt word example in the example library includes two parts: historical context and expected output; and a number of prompt word examples are obtained from the example library in sequence until the probability output by the optimal prompt word example selector is less than a preset probability threshold.

[0032] Further, including:

[0033] A second building module is used to form a final example set based on the finally selected several prompt word examples;

[0034] A first generating module, configured to input the final example set into a pre-built prompt word template to generate a final prompt word;

[0035] The prompt word template is a dynamic template, and the corresponding expression is: i = P (e i ), where e i is the sample set obtained for the i-th time.

[0036] Further, including:

[0037] A second generation module, used for inputting the final prompt word into a large language model to generate generation content corresponding to the generation task;

[0038] The second optimization module is used to optimize the optimal prompt word example selector again based on a reinforcement learning algorithm when the technician determines that the generated content does not meet the expected requirements.

[0039] In a third aspect, the technical solution provides an electronic device, characterized in that it includes at least one processor, the processor is coupled to a memory, the memory stores a computer program, and the computer program is configured to execute the method when executed by the processor.

[0040] In a fourth aspect, the present technical solution provides a computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program implements the described method when executed by a computer.

[0041] Beneficial effects:

[0042] It can be seen from the above technical solutions that the technical solution of the present invention provides a dynamic prompt word example recall method based on reinforcement learning to improve the technical problem that the output accuracy in the current large language model application cannot be guaranteed by the prompt word example application.

[0043] The method includes: first, constructing a prompt word example selector and a prompt word quality assessor. Secondly, introducing a reinforcement learning algorithm to iteratively train the prompt word example selector with the prompt word quality assessor as an evaluator until the expected number of iterations is reached to obtain an optimal prompt word example selector. Then, obtaining a generation task to dynamically select prompt word examples from the example library based on the optimal prompt word example selector; wherein any prompt word example in the example library includes two parts: historical context and expected output; obtaining a number of prompt word examples from the example library in sequence until the probability output by the optimal prompt word example selector is less than a preset probability threshold. At this point, the technical solution will use the prompt word example selector as an intelligent agent and the prompt word quality assessor as an evaluator, and use the reinforcement learning algorithm to select prompt word examples that are more in line with the distribution of the large language model from the huge example library, thereby optimizing the final generated prompt words and improving the task completion effect of the large language model.

[0044] It should be appreciated that all combinations of the foregoing concepts, as well as additional concepts described in greater detail below, may be considered to be part of the inventive subject matter of the present disclosure, provided such concepts are not mutually inconsistent.

[0045] The foregoing and other aspects, embodiments and features of the present invention can be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the present invention, such as the features and / or beneficial effects of the exemplary embodiments, will be apparent from the following description or learned from the practice of the specific embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in various figures may be represented by the same reference numeral. For clarity, not every component is labeled in every figure. Embodiments of various aspects of the present invention will now be described by way of example and with reference to the accompanying drawings, in which:

[0047] Figure 1 is a flow chart of the dynamic prompt word example recall method based on reinforcement learning described in this embodiment;

[0048] Figure 2 Flowchart for final prompt word generation;

[0049] Figure 3 Flowchart for re-optimization of the optimal cue word example selector;

[0050] Figure 4 Flowchart for final prompt word optimization;

[0051] Figure 5 A flowchart generated for a new generation task;

[0052] Figure 6 is a structural block diagram of the dynamic prompt word example recall system based on reinforcement learning described in this embodiment;

[0053] Figure 7 is a structural block diagram of the electronic device described in this embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meaning understood by people with general skills in the field to which the present invention belongs.

[0055] The words "first", "second" and similar words used in the specification and claims of this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular form of "a", "an" or "the" and other similar words do not indicate a quantitative limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" and the like mean that the elements or objects appearing before "include" or "comprise" cover the features, wholes, steps, operations, elements and / or components listed after "include" or "comprise", and do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0056] With the development of large language models (LLMs), prompts play a vital role in guiding models to generate content. Specifically, the prompt examples (demonstrations) of generating prompt words are an important factor affecting the output effect of the model. However, traditional prompt examples and prompt words are mostly constructed subjectively by humans and conform to human language habits. However, large language models do not necessarily follow the language laws of humans, so new methods are needed to construct prompt words that are more in line with the model distribution to improve the performance of the model. Based on this, this embodiment aims to propose a dynamic prompt example recall method based on reinforcement learning to improve the above technical defects.

[0057] The following is a detailed introduction to the dynamic prompt word example recall method based on reinforcement learning described in this embodiment in conjunction with the accompanying drawings.

[0058] like Figure 1 As shown, the method includes:

[0059] Step S102: construct a prompt word example selector and a prompt word quality assessor at the same time.

[0060] Specifically, the prompt word example selector is a neural network, and the corresponding expression is: ;

[0061] in, S (P i, E i+1 ) indicates that when there is a prompt word P obtained for the i-th time i When , should we continue to select the i+1th prompt word example E i+1 The probability of joining the example set. h0 is to continue to select the i+1th prompt word example E i+1 The probability of h1 not continuing to select the i+1th prompt word example Ei+1 The probability of. Among them, h0+h1=1.

[0062] The prompt word quality assessor is a neural network, and the corresponding expression is: ;

[0063] Among them, | D Data |For the dataset D Data The number of data in LLM(P i ) is the prompt word P obtained for the i-th time i The actual output of the corresponding large language model, Y i is the prompt word P obtained for the i-th time i The expected output of the corresponding large language model. L (LLM(P i ),Y i ) is used to measure whether the actual output of the large model matches the expected output. L (LLM(P i ),Y i ) has a value of 1, if it does not match L (LLM(P i ),Y i ) is 0. In specific implementation, the criterion for judging whether the actual output of the large model matches the expected output is whether the difference between the two does not exceed a preset threshold.

[0064] Step S104: introducing a reinforcement learning algorithm to iteratively train the prompt word example selector using the prompt word quality evaluator as an evaluator until the expected number of iterations is reached to obtain an optimal prompt word example selector.

[0065] Step S106: Obtain a generation task to dynamically select prompt word examples from the example library based on the optimal prompt word example selector.

[0066] In specific implementation, any prompt word example in the example library includes two parts: historical context and expected output. Several prompt word examples are sequentially obtained from the example library until the probability output by the optimal prompt word example selector is less than a preset probability threshold.

[0067] As a specific implementation method, Figure 2 As shown in the figure, after the prompt word is dynamically selected, the final prompt word for model generation is obtained through the following steps:

[0068] Step S10802: forming a final example set based on the finally selected prompt word examples.

[0069] Step S10804: input the final example set into a pre-built prompt word template to generate a final prompt word.

[0070] The prompt word template is a dynamic template, and the corresponding expression is: i = P (e i ), where e i is the sample set obtained for the i-th time.

[0071] Based on steps S10802 to S10804, it can be seen that in this embodiment, the process of generating the final prompt word is also consistent with the process of obtaining the final example set, which is a dynamic process in time sequence. Therefore, in the process of model generation, in addition to simply optimizing the prompt word examples, any intermediate prompt word example set in the process can also be used to generate intermediate prompt words according to user needs to input the model for content generation, thereby simultaneously verifying the rationality of the prompt word examples and prompt words, as well as the accuracy of the final result.

[0072] As a further embodiment, in combination Figure 3 As shown, the above optimal prompt word example selector can also be iteratively optimized through the following process:

[0073] Step S11002: input the final prompt word into a large language model to generate generated content corresponding to the generation task.

[0074] Step S11004: When the technical staff determines that the generated content does not meet the expected demand, the optimal prompt word example selector is optimized again based on the reinforcement learning algorithm.

[0075] Since the final output result of the large language model needs to be user-oriented, the expected needs of the passers-by are used as a reference in combination with steps S11002 to S11004, and the optimal prompt word example selector is further optimized so that the final generation result can better meet user needs, rather than being simply based on the machine and guided by machine cognition.

[0076] Similarly, combined Figure 4 As shown, after step S106, the following steps are also included:

[0077] Step S10822: forming a final example set based on the finally selected prompt word examples.

[0078] Step S10824: Update the final example set based on user-defined requirements to obtain a target example set.

[0079] Step S10826: Generate target prompt words based on the target example set.

[0080] At this time, based on steps S10822 to S10826, the user preference is introduced into the final example set obtained by the optimal prompt word example selector, so that the target prompt word finally generated can better meet the user's targeted needs.

[0081] As a preferred implementation, considering the generation rate when facing the same generation task or similar generation task, combined with Figure 5 As shown, after step S106, the following steps are also included:

[0082] Step S10862: A final example set is formed based on the finally selected prompt word examples and stored.

[0083] Step S10864: When it is determined that there is a newly generated task that is similar to the generated task, a first query request is sent to the client front end to inquire whether to call the final example set for reference; or when it is determined that there is a newly generated task that is the same as the generated task, a second query request is sent to the client front end to inquire whether to call the final example set for final prompt word generation.

[0084] At this time, in combination with steps S10862 to S10864, ​​the existing final example set can be introduced into the new generation task to form a reference for it, or content generation can be performed directly; thereby improving the generation rate.

[0085] In summary, the method described in this embodiment uses reinforcement learning technology to select prompt word examples that are more consistent with the distribution of the large language model from a huge example library to form optimized prompt words, thereby improving the task completion effect of the large language model. In addition, considering the match between the output results of the large language model and the user's preferences, a corresponding solution is designed to optimize and update the optimal prompt word example selector and the final prompt word set. At the same time, the existing prompt word example set is introduced into the new generation task to increase the generation rate, thereby further improving the practicality of the method.

[0086] The above program can be run in the processor, or it can be stored in the memory (or computer-readable storage medium), which includes permanent and non-permanent, removable and non-removable media. Information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include temporary computer-readable media, such as modulated data signals and carrier waves.

[0087] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more blocks can be implemented by different modules corresponding to different steps.

[0088] Based on the above method, this embodiment also provides a dynamic prompt word example recall system based on reinforcement learning. Figure 6 As shown, the system includes the following functional modules:

[0089] The first building module is used to simultaneously build a cue word example selector and a cue word quality assessor.

[0090] The prompt word example selector is a neural network, and the corresponding expression is: ;in, S (P i, E i+1 ) indicates that when there is a prompt word P obtained for the i-th time i When , should we continue to select the i+1th prompt word example E i+1 The probability of joining the example set, h0 is to continue to select the i+1th prompt word example E i+1 The probability of h1 not continuing to select the i+1th prompt word example E i+1 The probability of h0+h1=1.

[0091] The prompt word quality assessor is a neural network, and the corresponding expression is: ; Among them, | D Data |For the dataset D Data The number of data in LLM(P i ) is the prompt word P obtained for the i-th time i The actual output of the corresponding large language model, Y i is the prompt word P obtained for the i-th time i The expected output of the corresponding large language model, L (LLM(P i ),Y i ) is used to measure whether the actual output of the large model matches the expected output. L (LLM(P i ),Y i ) has a value of 1, if it does not match L (LLM(P i ),Y i ) has a value of 0.

[0092] The first optimization module is used to introduce a reinforcement learning algorithm to iteratively train the prompt word example selector using the prompt word quality evaluator as an evaluator until an expected number of iterations is reached to obtain an optimal prompt word example selector.

[0093] A selection module is used to obtain a generation task to dynamically select a prompt word example from an example library based on the optimal prompt word example selector; wherein any prompt word example in the example library includes two parts: historical context and expected output; and a number of prompt word examples are obtained from the example library in sequence until the probability output by the optimal prompt word example selector is less than a preset probability threshold.

[0094] Since the system is used to implement the steps of the above method, those that have been described have not been repeated here.

[0095] For example, the system further comprises:

[0096] The second building module is used to form a final example set based on the finally selected several prompt word examples.

[0097] The first generation module is used to input the final example set into a pre-built prompt word template to generate a final prompt word; wherein the prompt word template is a dynamic template, and the corresponding expression is: i = P (e i ), where e iis the sample set obtained for the i-th time.

[0098] For example, the system further comprises:

[0099] The second generation module is used to input the final prompt word into a large language model to generate generation content corresponding to the generation task.

[0100] The second optimization module is used to optimize the optimal prompt word example selector again based on a reinforcement learning algorithm when the technician determines that the generated content does not meet the expected requirements.

[0101] Combination Figure 7 As shown, this embodiment further provides an electronic device, comprising at least one processor, the processor being coupled to a memory, the memory storing a computer program, and the computer program being configured to execute the method when being executed by the processor.

[0102] Similarly, this embodiment also provides a computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program implements the method when executed by a computer.

[0103] Since the system, the electronic device and the computer-readable storage medium are all built based on the method, they also have the technical advantage of improving the accuracy of the final large language model output in practical applications.

[0104] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. A dynamic prompt word example recall method based on reinforcement learning, characterized in that: include: At the same time, a prompt word example selector and a prompt word quality assessor are constructed; The prompt word example selector is a neural network, and the corresponding expression is: ;in, S (P i, E i+1 ) indicates that when there is a prompt word P obtained for the i-th time i When , should we continue to select the i+1th prompt word example E i+1 The probability of joining the example set, h0 is to continue to select the i+1th prompt word example E i+1 The probability of h1 not continuing to select the i+1th prompt word example E i+1 The probability of ; where h0+h1=1; The prompt word quality assessor is a neural network, and the corresponding expression is: ; Among them, | D Data |For the dataset D Data The number of data in LLM(P i ) is the prompt word P obtained for the i-th time i The actual output of the corresponding large language model, Y i is the prompt word P obtained for the i-th time i The expected output of the corresponding large language model, L (LLM(P i ),Y i ) is used to measure whether the actual output of the large model matches the expected output. L (LLM(P i ),Y i ) has a value of 1, if it does not match L (LLM(P i ),Y i ) is 0; Introducing a reinforcement learning algorithm to iteratively train the prompt word example selector using the prompt word quality evaluator as an evaluator until the expected number of iterations is reached to obtain an optimal prompt word example selector; A generation task is obtained to dynamically select a prompt word example from an example library based on the optimal prompt word example selector; wherein any prompt word example in the example library includes two parts: historical context and expected output; and a number of prompt word examples are obtained from the example library in sequence until the probability output by the optimal prompt word example selector is less than a preset probability threshold.

2. The method for recalling dynamic prompt words based on reinforcement learning according to claim 1, characterized in that: The acquisition generation task is to dynamically select prompt word examples from the example library based on the optimal prompt word example selector; comprising: A final example set is formed based on the finally selected several prompt word examples; Inputting the final example set into a pre-built prompt word template to generate a final prompt word; The prompt word template is a dynamic template, and the corresponding expression is: i = P (e i ), where e i is the sample set obtained for the i-th time.

3. The method for recalling dynamic prompt words based on reinforcement learning according to claim 2, characterized in that: After inputting the final example set into a pre-built prompt word template to generate a final prompt word; further comprising: Inputting the final prompt word into a large language model to generate generated content corresponding to the generation task; When the technician determines that the generated content does not meet the expected needs, the optimal prompt word example selector is optimized again based on the reinforcement learning algorithm.

4. The method for recalling dynamic prompt words based on reinforcement learning according to claim 1, characterized in that: The acquisition generation task is to dynamically select prompt word examples from the example library based on the optimal prompt word example selector; comprising: A final example set is formed based on the finally selected several prompt word examples; Based on user-defined requirements, the final example set is updated to obtain a target example set; A target prompt word is generated based on the target example set.

5. The method for recalling dynamic prompt words based on reinforcement learning according to claim 1, characterized in that: The acquisition generation task is to dynamically select prompt word examples from the example library based on the optimal prompt word example selector; comprising: Based on the finally selected several prompt word examples, a final example set is formed and stored; When it is determined that there is a newly generated task that is similar to the generated task, a first query request is sent to the client front end to inquire whether to call the final example set for reference; or when it is determined that there is a newly generated task that is the same as the generated task, a second query request is sent to the client front end to inquire whether to call the final example set for final prompt word generation.

6. A dynamic prompt word example recall system based on reinforcement learning, characterized in that: include: A first building module, used to simultaneously build a cue word example selector and a cue word quality assessor; The prompt word example selector is a neural network, and the corresponding expression is: ;in, S (P i, E i+1 ) indicates that when there is a prompt word P obtained for the i-th time i When , should we continue to select the i+1th prompt word example E i+1 The probability of joining the example set, h0 is to continue to select the i+1th prompt word example E i+1 The probability of h1 not continuing to select the i+1th prompt word example E i+1 The probability of ; where h0+h1=1; The prompt word quality assessor is a neural network, and the corresponding expression is: ; Among them, | D Data |For the dataset D Data The number of data in LLM(P i ) is the prompt word P obtained for the i-th time i The actual output of the corresponding large language model, Y i is the prompt word P obtained for the i-th time i The expected output of the corresponding large language model, L (LLM(P i ),Y i ) is used to measure whether the actual output of the large model matches the expected output. L (LLM(P i ),Y i ) has a value of 1, if it does not match L (LLM(P i ),Y i ) is 0; A first optimization module is used to introduce a reinforcement learning algorithm to iteratively train the prompt word example selector using the prompt word quality evaluator as an evaluator until a desired number of iterations is reached to obtain an optimal prompt word example selector; A selection module is used to obtain a generation task to dynamically select a prompt word example from an example library based on the optimal prompt word example selector; wherein any prompt word example in the example library includes two parts: historical context and expected output; and a number of prompt word examples are obtained from the example library in sequence until the probability output by the optimal prompt word example selector is less than a preset probability threshold.

7. The dynamic prompt word example recall system based on reinforcement learning according to claim 6, characterized in that: include: A second building module is used to form a final example set based on the finally selected several prompt word examples; A first generating module, configured to input the final example set into a pre-built prompt word template to generate a final prompt word; The prompt word template is a dynamic template, and the corresponding expression is: i = P (e i ), where e i is the sample set obtained for the i-th time.

8. The dynamic prompt word example recall system based on reinforcement learning according to claim 7, characterized in that: include: A second generation module, used for inputting the final prompt word into a large language model to generate generation content corresponding to the generation task; The second optimization module is used to optimize the optimal prompt word example selector again based on a reinforcement learning algorithm when the technician determines that the generated content does not meet the expected requirements.

9. An electronic device, characterized in that: The method comprises at least one processor, wherein the processor is coupled to a memory, wherein a computer program is stored in the memory, and wherein the computer program is configured to execute the method according to any one of claims 1 to 5 when executed by the processor.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 5 is implemented.

Images