A large language model strategy generation method and system based on retrieval enhancement

Abstract

The present invention discloses a large language model strategy generation method and system based on retrieval enhancement, comprising the following steps: step 1, grouping multiple tasks into a group, and constructing a database in an initial round; step 2, taking the i-th task as the current task, obtaining a graph model based on the observation features in the observation image through the large language model and encoding it; step 3, extracting the corresponding macro action sequence from the database, obtaining the macro action after interaction with the environment, and temporarily storing it together with the corresponding encoded graph model in step 2 as a temporary feature sequence; step 4, setting i=i+1, repeating steps 2 to 3, obtaining temporary feature sequences corresponding to all tasks, and updating the database based on all temporary feature sequences; the large language model strategy generation method establishes a retrieval database of scene interaction trajectories, and obtains a decision strategy with customized decision-making capabilities.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for generating a large language model strategy based on retrieval enhancement. Background Art

[0002] Large models have made significant progress in fields such as natural language processing and image recognition. However, in embodied intelligence applications, such as indoor robot movement and robotic arm manipulation, traditional large language models often suffer from insufficient generalization and an inability to generate specialized strategies for specific scenarios. These tasks typically require models to have scenario-specific decision-making capabilities and the ability to autonomously learn and adapt in complex environments.

[0003] Common policy generation frameworks can be roughly divided into two categories. One is a technical framework that learns specific policies for specific scenarios through reinforcement learning, and the other is a technical framework that directly infers policies based on a general large language model.

[0004] While the reinforcement learning (RL) technology framework has performed well in many embodied intelligence application scenarios, it also has some problems. First, RL algorithms require a large amount of training data and time to find the optimal strategy, which can lead to inefficiency and high costs in practical applications. Second, RL methods often require trial and error during training, which can cause damage to the agent or even lead to danger during actual operation. Furthermore, model training often relies on accurate environmental modeling. If the environment changes, the model may need to be retrained, reducing the system's flexibility and adaptability.

[0005] Although the technical framework for direct policy reasoning based on large language models (LLMs) can generate policies using pre-trained models and reduce the amount of training required for specific scenarios, it also presents some challenges. First, the policies generated by large language models may lack the accuracy and feasibility for specific scenarios, leading to errors or undesirable results during execution. Second, when generating policies, large language models are unable to make customized judgments and decisions based on the specificity of the scenario. In addition, the policy generation of large language models relies on the natural language description of the input. If the input description is not clear or accurate enough, the generated policy may also be inaccurate, affecting the actual effect. Summary of the Invention

[0006] Based on the technical problems existing in the background technology, the present invention proposes a large language model strategy generation method and system based on retrieval enhancement, establishes a retrieval database of scene interaction trajectories, and obtains a decision strategy with customized decision-making capabilities.

[0007] The present invention proposes a method for generating a large language model strategy based on retrieval enhancement, comprising the following steps:

[0008] Step 1: Group multiple tasks together. In the initial round, obtain trajectory data from the interaction between the agent and the environment for each task. Separate different structured data series from the trajectory data. The structured data series includes observation sequences and macro-action sequences. Organize the observation sequence of each task into a graphical model. Encode each graphical model to obtain a fixed-length feature vector and store it in the database along with the macro-action sequence.

[0009] Step 2: Take the i-th task as the current task. In the current round, obtain the observation picture given by the environment under the current task. The observation picture includes observation features and action features. Convert the observation features into label text and encode them into a query feature vector. Perform cosine similarity matching on the query feature vector and all feature vectors in the database and sort them in descending order. Take the feature vectors corresponding to the top k similarities from the database and input the extracted feature vectors into the prompt of the large language model. Output the graph model corresponding to the extracted feature vector and encode it. i∈[1,N], where N is the total number of tasks in a group.

[0010] Step 3: The macro-action sequence corresponding to the extracted feature vector is retrieved from the database. The large language model uses the set prompt word and the selected macro-action sequence as an index to select one macro-action from the set macro-action library. Each macro-action is converted into an executable micro-action according to the pre-set principle and input into the environment in sequence for interaction. The macro-action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence.

[0011] Step 4: Let i = i + 1, repeat steps 2 to 3 to obtain temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences;

[0012] Step 5: Use the updated database as the database for the next set of interactions between the agent and the environment under multiple tasks, and iterate steps 2 to 4 until the database is finally iteratively updated.

[0013] Furthermore, in step 1, each graph model is encoded to obtain a feature vector of fixed length, specifically including:

[0014] In the graph model, the basic object is the root node and other objects are leaf nodes;

[0015] The root node and leaf node are converted into string type, and the string type is output as a fixed-length feature vector through the sentence feature extraction module.

[0016] Furthermore, in step 2, the observed features are converted into label text and encoded into a query feature vector, specifically:

[0017] Convert observation features into labeled text by using MaskRCNN or TextWorld tools;

[0018] Convert the label text into a graph model, where the basic object is the root node and other objects are leaf nodes;

[0019] The root node and leaf nodes are converted into string types, and the string types are output as query feature vectors through the sentence feature extraction module.

[0020] Furthermore, in step three, each macro action follows the principle of "action does not involve specific parameters but only focuses on objects".

[0021] A large language model strategy generation system based on retrieval enhancement includes an initial construction module, an observation matching module, an action matching module, a current group database update module and a next group database update module;

[0022] The initial construction module is used to group multiple tasks. In the initial round, the trajectory data obtained from the interaction between the agent and the environment for each task is obtained. Different structural data series are separated from the trajectory data. The structural data series include observation sequences and macro-action sequences. The observation sequence of each task is combined into a graphical model. Each graphical model is encoded to obtain a fixed-length feature vector and stored in the database along with the macro-action sequence.

[0023] The observation matching module is used to take the i-th task as the current task. In the current round, it obtains the observation picture given by the environment under the current task. The observation picture includes observation features and action features. The observation features are converted into label text and encoded into a query feature vector. The query feature vector is matched with all feature vectors in the database by cosine similarity and sorted in descending order. The feature vectors corresponding to the top k similarities are taken from the database and input into the prompt of the large language model. The graphical model corresponding to the extracted feature vector is output and encoded. i∈[1,N], where N is the total number of tasks in a group;

[0024] The action matching module is used to retrieve the macro-action sequence corresponding to the extracted feature vector from the database. The large language model uses the set prompt word and the selected macro-action sequence as an index to select one macro-action from the set macro-action library. The selected macro-action is converted into micro-actions to be executed according to the pre-set principle and is sequentially input into the environment for interaction. The macro-action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence.

[0025] The database update module under the current group is used to set i=i+1, repeat the observation matching module to the action matching module, obtain the temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences;

[0026] The next group of database update modules is used to use the updated database as the database for the next group of multi-task agent-environment interaction, iterating the observation matching module to the current group of database update modules until the final iterative update of the database is completed.

[0027] Furthermore, in the initial construction module, each graph model is encoded to obtain a fixed-length feature vector, specifically including:

[0028] In the graph model, the basic object is the root node and other objects are leaf nodes;

[0029] The root node and leaf node are converted into string type, and the string type is output as a fixed-length feature vector through the sentence feature extraction module.

[0030] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the large language model strategy generation method as described above when executing the computer program.

[0031] A computer-readable storage medium stores a plurality of classification programs, which are used to be called by a processor and execute the large language model strategy generation method as described above.

[0032] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various media that can store program codes.

[0033] The advantage of the large language model strategy generation method and system based on retrieval enhancement provided by the present invention is that it combines the advantages of the generation framework of progressive retrieval enhancement and the direct strategy reasoning technology framework based on the large language model (LLM) to solve the limitations of traditional methods in embodied intelligence application scenarios. By combining the progressive retrieval enhancement framework and the large language model, this embodiment has the following beneficial effects: first, the preliminary strategy is generated using the large language model, and then refined and optimized through the progressive retrieval enhancement generation framework, thereby improving the generalization ability and adaptability of the large language model in complex environments. The preliminary strategy foundation provided by the large language model can reduce the demand for training data and time, while the progressive retrieval enhancement generation framework can be customized according to specific scenarios. Secondly, the large language model provides a pre-training strategy, which reduces the large amount of training data and time required by the algorithm in the initial exploration stage, thereby reducing training costs and efficiency issues.

[0034] By combining the strategies generated by a large language model with the precise adjustments of a progressive retrieval enhancement framework, this embodiment generates more accurate and feasible strategies, reducing errors and undesirable results during execution. Furthermore, in this embodiment, the updated database of the current set serves as the database for the robot's interaction with the environment during the next set of multi-tasks. Therefore, when the environment changes, the strategy can be quickly adjusted by searching the retrieval-enhanced database, without the need for complete retraining, enhancing the system's flexibility and autonomous learning capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the process of the present invention;

[0036] Figure 2 Flowchart of the structure of a database enhanced for progressive retrieval;

[0037] Figure 3 It is a structural flow chart of the present invention. DETAILED DESCRIPTION

[0038] The technical solutions of the present invention are described in detail below through specific embodiments. Numerous specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0039] There are currently two types of frameworks. One is a technical framework that learns specific strategies for specific scenarios through reinforcement learning. The other is a technical framework that directly reasons about strategies based on a general large language model, establishing a retrieval database of scene interaction trajectories, so that the large language model can autonomously learn and adapt in complex environments.

[0040] 1. A technical framework for learning specific strategies for specific scenarios through reinforcement learning

[0041] Reinforcement learning (RL) is a machine learning method that optimizes the behavior of an agent through interaction with its environment. This method uses a reward and penalty mechanism to guide the agent's learning process, enabling it to gradually develop a strategy that maximizes cumulative rewards. In embodied intelligence applications, the RL technology framework typically includes the following core components:

[0042] · State Space: describes all possible states of the agent in the environment.

[0043] · Action Space: All possible actions that the agent can perform in each state.

[0044] · Reward Function: Evaluates the agent's performance on each state-action pair and provides feedback to the agent through rewards or penalties.

[0045] · Policy: The rule or function by which an agent chooses an action in each state.

[0046] In specific applications, such as indoor robot movement and robotic arm manipulation, RL technology frameworks require extensive training for specific scenarios to learn optimal strategies. For example, when a robot moves within a specific room, it needs to continuously explore and try different paths to find the optimal navigation strategy. When a robotic arm performs a specific task, it needs to determine the optimal operation sequence and force through repeated trials.

[0047] Common reinforcement learning strategies include the Actor-Critic (AC) algorithm and the Proximal Policy Optimization (PPO) algorithm.

[0048] The executor-supervisor algorithm uses two modules, the Actor and the Critic, for training and reasoning. The Actor is responsible for learning the policy π(a|s;θ) and outputting the action a directly from the state s. The Critic estimates the state-value function V(s;w) or the action-value function Q(s,a;w) to evaluate the Actor's policy. The parameters of the Actor and Critic networks are then iteratively updated to perform training and reasoning on different tasks. The proximal policy optimization algorithm maximizes the objective function:

[0049]

[0050] where π θ is the current strategy, It's the old strategy, A t is the advantage function, and ∈ is the clipping threshold used to limit the amplitude of the policy update.

[0051] 2. Technical framework for direct policy reasoning based on a general large language model

[0052] Unlike reinforcement learning, the direct policy inference framework based on a large language model (LLM) leverages pre-trained large-scale neural network models to learn from large amounts of text data. This framework can directly generate operational policies without extensive training for specific scenarios. The core idea of ​​this framework is to leverage the powerful generative and knowledge transfer capabilities of LLMs to generate appropriate policies for specific scenarios.

[0053] Specifically, the LLM-based policy reasoning framework usually includes the following steps:

[0054] · Pre-trained Model: Use massive text data to train a general large language model, giving it rich knowledge and generation capabilities.

[0055] · Scenario Description: Convert the specific scenario and task requirements into natural language description and input it into the model.

[0056] · Policy Generation: The model generates a sequence of movement or manipulation actions that meet the requirements based on the input scene description.

[0057] · Policy Execution and Feedback: The agent executes the generated policy and continuously optimizes the model's generation capabilities through a feedback mechanism.

[0058] In practical applications, LLM can leverage its powerful language understanding and generation capabilities to generate specific operational instructions for indoor robot movement and manipulator manipulation tasks. For example, a robot can understand the room layout and generate a reasonable navigation path based on a natural language input description; a manipulator can infer the appropriate movement position and manipulation sequence based on the task description.

[0059] This embodiment addresses the problems of inefficient reinforcement learning training, the inability of large language models to make customized decisions, and the inability of large language models to make customized judgments and decisions based on the specificity of the scenario when generating strategies using pre-trained models, as well as the problem of relying on natural language descriptions of inputs. By establishing a retrieval database of scenario interaction trajectories, a decision-making system with customized decision-making capabilities is constructed using a retrieval-enhanced large language model. Figure 2 As shown, this embodiment uses a database with progressive search enhancement, specifically:

[0060] 1) Feature Information Extraction and Storage: Separate the structured data pairs (action, observation) from the trajectory data, convert each observation into text using the TextWorld tool, and encode it into fixed-length features using the MiniLM-v6-large text encoding model.

[0061] 2) Information retrieval structure: The data observed in the environment is converted into feature vectors using TextWorld and MiniLM-v6-large. The features obtained in the previous round are compared in the database. The cosine similarity is used to obtain a similarity score, and the three trajectories with the highest output are retrieved.

[0062] 3) Large model reasoning: Retrieved similar trajectories are used as examples and fed into the large language model. The output is macro-actions for specific environments. The macro-actions output by the large language model are decomposed into micro-actions using a predefined method. After one round of interaction, the macro-actions are encoded into feature vectors using the method in step 1) and updated into the database. Then, steps 1) to 3) are repeated to complete database iteration.

[0063] 1) to 3) combine the advantages of the progressive retrieval enhancement framework technology framework and the direct policy reasoning technology framework based on the large language model (LLM) to address the limitations of traditional methods in embodied intelligence application scenarios. The following uses the intelligent agent as a robot to specifically illustrate the large language model policy generation method based on this retrieval enhancement framework.

[0064] like Figures 1 to 3 As shown, the present invention proposes a large language model strategy generation method based on retrieval enhancement, including the following steps:

[0065] Step 1: Group multiple tasks together. In the initial round, obtain trajectory data from the robot's interaction with the environment for each task. Separate different structural data series from the trajectory data. The structural data series includes observation sequences and macro-action sequences. The observation sequence of each task is combined into a graphical model. Each graphical model is encoded to obtain a fixed-length feature vector and stored in the database along with the macro-action sequence.

[0066] In the initial stage of the robot's interaction with the environment, because there is no database for this scenario, in the initial round, it is first necessary to obtain the trajectory data of the robot's interaction with the environment, separate different structural data sequences from the trajectory data text, organize the observation sequence into a graphical model, and encode the graphical model. The specific encoding process is as follows: in the graphical model, the basic object is used as the root node and other objects are used as leaf nodes; the root node and leaf nodes are converted into string types, and the string type is output as a fixed-length feature vector through the sentence feature extraction module (MiniLM-v6-Large).

[0067] Step 2: Take the i-th task as the current task. In the current round, obtain the observation picture given by the environment under the current task. The observation picture includes observation features and action features. Convert the observation features into label text and encode them into a query feature vector. Perform cosine similarity matching on the query feature vector and all feature vectors in the database and sort them in descending order. Take the feature vectors corresponding to the top k similarities from the database and input the extracted feature vectors into the prompt of the large language model. Output the graph model corresponding to the extracted feature vector and encode it. i∈[1,N], where N is the total number of tasks in a group.

[0068] Except for the initial round, which does not require retrieval because the database is built based on the trajectory data of environmental interactions, all other rounds need to be retrieved from the database. The specific retrieval process is as follows:

[0069] (a1) Before making an action decision, the current environment will receive an observation image provided by the environment. The observation image includes observation features and action features. The observation features are converted into label text using MaskRCNN or TextWorld tools, and then the query feature vector is converted using MiniLM-v6-Large, the same as in step 1. Specifically, the label text is organized into a graph model with basic objects as root nodes and other objects as leaf nodes; the root node and leaf nodes are converted into string types, and the string types are output as query feature vectors using MiniLM-v6-Large.

[0070] (a2) Perform cosine similarity matching between the query feature vector and all feature vectors in the database:

[0071]

[0072] Among them, Sim represents similarity, Q represents the query feature vector, and K represents the feature vector in the database.

[0073] (a3) Extract the trajectories corresponding to the k (e.g., 3) feature vectors with the highest similarity matching values ​​and input them into the prompt of the large language model to obtain the corresponding graphical model for the current task. For example, if there are 200 tasks in a group, there will be 200 corresponding graphical models. Through similarity matching, the three graphical models most similar to the current task are selected from these 200 graphical models to determine the macro action sequence.

[0074] Therefore, this embodiment uses a large language model to generate a preliminary strategy, which is then refined and optimized through a generation framework enhanced by progressive retrieval, thereby improving the generalization ability and adaptability of the model in complex environments.

[0075] Step 3: The macro-action sequence corresponding to the extracted feature vector is retrieved from the database. The large language model uses the set prompt word and the selected macro-action sequence as an index to select one macro-action from the set macro-action library. The selected macro-action is converted into micro-actions to be executed according to the pre-set principle and is sequentially input into the environment for interaction. The macro-action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence.

[0076] In step 2, three feature vectors are retrieved from the database. These feature vectors correspond to three macro-action sequences, respectively. Using these three selected macro-action sequences and the set prompt word as indexes, a macro-action is selected from the macro-action library. Each macro-action is designed based on the principle of "actions are not concerned with specific parameters, but only with objects." A prompt word is set to allow the large language model to select one of these macro-actions. The macro-action is then converted into each micro-action that is actually executed according to pre-defined principles and then sequentially input into the environment, resulting in the resulting macro-action after interaction. A macro-action is selected based on the prompt word, and this macro-action also corresponds to a graphical model. Therefore, the temporary feature sequence is the selected macro-action and the corresponding encoded graphical model. The macro-action library contains all macro-actions pre-set for the environment.

[0077] For the redundant information that may appear after the large language model is inferred, it is filtered through the pre-set regular matching rules to obtain the macro action that meets the conditions. If there is no macro action that meets the conditions, it will be retried.

[0078] The regular matching rules are as follows:

[0079] Define the macro action set A in the alternative macro action library as:

[0080] A={a1,a2,a3,…,a n}

[0081] Among them, a n Indicates the nth macro action;

[0082] The regular expression pattern is:

[0083] Pattern=a1∣a2∣a3∣…∣a n

[0084] Based on the k (for example, 3) feature vectors inferred from the large language model, the macro action sequence Matches is matched from the database:

[0085] Matches = {m∈T | m∈A}

[0086] Where T is the text corresponding to k (for example, 3) feature vectors, and m is the macro-action sequence to be matched. If there are 3 feature vectors, 3 macro-action sequences will be matched;

[0087] Finally, use Matches and the set prompt word as an index to select a macro action from the macro action library as a macro action that meets the conditions.

[0088] Step 4: Let i = i + 1, repeat steps 2 to 3 to obtain temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences;

[0089] This embodiment treats multiple tasks as a group, and updates the database after a group of data completes the interaction, which can update the database in parallel and speed up the iteration of the database.

[0090] Step 5: Use the updated database as the database for the next set of robot-environment interactions under multiple tasks, and iterate steps 2 to 4 until the database is finally iteratively updated.

[0091] According to steps one to five, this embodiment combines the advantages of the generation framework of progressive retrieval enhancement and the direct policy reasoning technology framework based on the large language model (LLM) to solve the limitations of traditional methods in embodied intelligence application scenarios. By combining the progressive retrieval enhancement framework and the large language model, this embodiment has the following beneficial effects: First, the large language model is used to generate a preliminary strategy, which is then refined and optimized through the progressive retrieval enhancement generation framework (steps two to five), thereby improving the generalization ability and adaptability of the large language model in complex environments. The preliminary strategy foundation provided by the large language model can reduce the demand for training data and time, while the progressive retrieval enhancement generation framework can be customized according to specific scenarios. Secondly, the large language model provides a pre-training strategy, which reduces the large amount of training data and time required by the algorithm in the initial exploration stage, thereby reducing training costs and efficiency issues.

[0092] By combining the strategies generated by a large language model with the precise adjustments of a progressive retrieval enhancement framework, this embodiment generates more accurate and feasible strategies, reducing errors and undesirable results during execution. Furthermore, in this embodiment, the updated database of the current set serves as the database for the robot's interaction with the environment during the next set of multi-tasks. Therefore, when the environment changes, the strategy can be quickly adjusted by searching the retrieval-enhanced database, without the need for complete retraining, enhancing the system's flexibility and autonomous learning capabilities.

[0093] At the same time, this embodiment uses a large language model to provide guidance in the early stages of strategy generation, reducing the need for repeated trials in the reinforcement learning algorithm during training, thereby reducing the risk of damage or danger to the robot during actual operation; that is, using the general knowledge inherent in the large language model can reduce unnecessary exploration by the system, and by defining macro actions, the system can avoid making meaningless or dangerous macro actions.

[0094] Although the strategy generation of large language models depends on the natural language description of the input, by combining the feedback mechanism of the progressive retrieval enhancement framework, the strategy can be continuously optimized and adjusted to make it more in line with the needs of specific scenarios. Even if the input description is not clear or accurate enough, it can be corrected and improved through the progressive retrieval enhancement framework.

[0095] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A large language model strategy generation method based on retrieval enhancement, characterized in that: The steps include: Step 1: Group multiple tasks together. In the initial round, obtain trajectory data from the interaction between the agent and the environment for each task. Separate different structured data series from the trajectory data. The structured data series includes observation sequences and macro-action sequences. Organize the observation sequence of each task into a graphical model. Encode each graphical model to obtain a fixed-length feature vector and store it in the database along with the macro-action sequence. Step 2: Take the i-th task as the current task. In the current round, obtain the observation picture given by the environment under the current task. The observation picture includes observation features and action features. Convert the observation features into label text and encode them into a query feature vector. Perform cosine similarity matching on the query feature vector and all feature vectors in the database and sort them in descending order. Take the feature vectors corresponding to the top k similarities from the database and input the extracted feature vectors into the prompt of the large language model. Output the graph model corresponding to the extracted feature vector and encode it. i∈[1,N], where N is the total number of tasks in a group. Step 3: The macro-action sequence corresponding to the extracted feature vector is retrieved from the database. The large language model uses the set prompt word and the selected macro-action sequence as an index to select one macro-action from the set macro-action library. The selected macro-action is converted into micro-actions to be executed according to the pre-set principle and is sequentially input into the environment for interaction. The macro-action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence. Step 4: Let i = i + 1, repeat steps 2 to 3 to obtain temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences; Step 5: Use the updated database as the database for the next set of interactions between the agent and the environment under multiple tasks, and iterate steps 2 to 4 until the database is finally iteratively updated.

2. The method for generating a large language model strategy based on retrieval enhancement according to claim 1, characterized in that: In step 1, each graph model is encoded to obtain a fixed-length feature vector, specifically including: In the graph model, the basic object is the root node and other objects are leaf nodes; The root node and leaf node are converted into string type, and the string type is output as a fixed-length feature vector through the sentence feature extraction module.

3. The method for generating a large language model strategy based on retrieval enhancement according to claim 1, characterized in that: In step 2, the observed features are converted into label text and encoded into query feature vectors, specifically: Convert observation features into labeled text by using MaskRCNN or TextWorld tools; Convert the label text into a graph model, where the basic object is the root node and other objects are leaf nodes; The root node and leaf nodes are converted into string types, and the string types are output as query feature vectors through the sentence feature extraction module.

4. The method for generating a large language model strategy based on retrieval enhancement according to claim 1, characterized in that: In step three, each macro action follows the principle of "action does not involve specific parameters but only focuses on objects".

5. A large language model strategy generation system based on retrieval enhancement, characterized in that: It includes an initial construction module, an observation matching module, an action matching module, a database update module for the current group, and a database update module for the next group; The initial construction module is used to group multiple tasks. In the initial round, the trajectory data obtained from the interaction between the agent and the environment for each task is obtained. Different structural data series are separated from the trajectory data. The structural data series include observation sequences and macro-action sequences. The observation sequence of each task is combined into a graphical model. Each graphical model is encoded to obtain a fixed-length feature vector and stored in the database along with the macro-action sequence. The observation matching module is used to take the i-th task as the current task. In the current round, it obtains the observation picture given by the environment under the current task. The observation picture includes observation features and action features. The observation features are converted into label text and encoded into a query feature vector. The query feature vector is matched with all feature vectors in the database by cosine similarity and sorted in descending order. The feature vectors corresponding to the top k similarities are taken from the database and input into the prompt of the large language model. The graphical model corresponding to the extracted feature vector is output and encoded. i∈[1,N], where N is the total number of tasks in a group; The action matching module is used to retrieve the macro-action sequence corresponding to the extracted feature vector from the database. The large language model uses the set prompt word and the selected macro-action sequence as an index to select one macro-action from the set macro-action library. The selected macro-action is converted into micro-actions to be executed according to the pre-set principle and is sequentially input into the environment for interaction. The macro-action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence. The database update module under the current group is used to set i=i+1, repeat the observation matching module to the action matching module, obtain the temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences; The next group of database update modules is used to use the updated database as the database for the next group of multi-task agent-environment interaction, iterating the observation matching module to the current group of database update modules until the final iterative update of the database is completed.

6. The system for generating a large language model strategy based on retrieval enhancement according to claim 5, characterized in that: In the initial building block, each graph model is encoded to obtain a fixed-length feature vector, including: In the graph model, the basic object is the root node and other objects are leaf nodes; The root node and leaf node are converted into string type, and the string type is output as a fixed-length feature vector through the sentence feature extraction module.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the large language model strategy generation method according to any one of claims 1 to 4 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of classification programs, which are used to be called by a processor and execute the large language model strategy generation method according to any one of claims 1 to 4.

Images