Large language model strategy generation method and system based on retrieval enhancement

Abstract

The invention discloses a large language model strategy generation method and system based on retrieval enhancement, and the method comprises the following steps: 1, enabling a plurality of tasks to form a group, and constructing a database in an initial round; 2, taking the ith task as a current task, obtaining a graph model through a large language model based on observation features in an observation picture, and encoding the graph model, 3, extracting a corresponding macro action sequence from a database, obtaining a macro action after interaction with the environment, and temporarily storing the macro action and the correspondingly encoded graph model in the step 2 as a temporary feature sequence; 4, letting i = i + 1, repeating the steps 2 to 3 to obtain temporary feature sequences corresponding to all tasks, and updating the database based on all the temporary feature sequences; according to the large language model strategy generation method, a retrieval database of scene interaction tracks is established, and a decision strategy with customized decision ability is obtained.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large language model strategy generation method and system based on retrieval enhancement. Background Art

[0002] Large models have made significant progress in natural language processing, image recognition and other fields. However, in embodied intelligence application scenarios, such as indoor robot movement and robotic arm operation, traditional large language models often have problems such as insufficient generalization and inability to generate dedicated strategies for specific scenarios. These tasks usually require the model to have scenario-customized decision-making capabilities and be able to autonomously learn and adapt in complex environments.

[0003] Common policy generation frameworks can be roughly divided into two categories. One is a technical framework that learns specific policies for specific scenarios through reinforcement learning, and the other is a technical framework that directly infers policies based on a general large language model.

[0004] Although the reinforcement learning (RL) technology framework has performed well in many embodied intelligence application scenarios, it also has some problems. First, the reinforcement learning algorithm requires a lot of training data and time to find the optimal strategy, which may lead to problems such as inefficiency and high cost in practical applications. Secondly, reinforcement learning methods often require repeated trials during the training process, which may cause damage to the intelligent agent or cause danger in actual operation. In addition, the training of the model usually relies on accurate environmental modeling. If the environment changes, the model may need to be retrained, which reduces the flexibility and adaptability of the system.

[0005] Although the technical framework of direct policy reasoning based on large language models (LLM) can generate policies using pre-trained models and reduce a lot of training for specific scenarios, there are also some challenges. First, the policies generated by large language models may lack accuracy and feasibility for specific scenarios, resulting in errors or undesirable results during execution. Secondly, when generating policies, large language models are also unable to make customized judgments and decisions based on the specificity of the scenario. In addition, the policy generation of large language models depends on the natural language description of the input. If the input description is not clear or accurate enough, the generated policy may also be inaccurate, affecting the actual effect. Summary of the invention

[0006] Based on the technical problems existing in the background technology, the present invention proposes a large language model strategy generation method and system based on retrieval enhancement, establishes a retrieval database of scene interaction trajectories, and obtains a decision strategy with customized decision-making capabilities.

[0007] The present invention proposes a large language model strategy generation method based on retrieval enhancement, comprising the following steps:

[0008] Step 1: Group multiple tasks into a group. In the initial round, obtain the trajectory data obtained by the interaction between the agent and the environment under multiple tasks respectively, separate different structural data series from the trajectory data, and the structural data series includes observation sequence and macro action sequence. Organize the observation sequence of each task into a graph model, encode each graph model to obtain a fixed-length feature vector and store it in the database together with the macro action sequence;

[0009] Step 2: Take the i-th task as the current task. In the current round, obtain the observation picture given by the environment under the current task. The observation picture includes observation features and action features. Convert the observation features into label text and encode them into query feature vectors. Perform cosine similarity matching between the query feature vector and all feature vectors in the database and arrange them in descending order. Take the feature vectors corresponding to the top k similarities from the database, input the extracted feature vectors into the prompt of the large language model, output the graph model corresponding to the extracted feature vector and encode it, i∈[1,N], N is the total number of tasks in a group;

[0010] Step 3: The macro action sequence corresponding to the extracted feature vector is extracted from the database. The large language model uses the set prompt word and the selected macro action sequence as an index to select one of the macro actions from the set macro action library. Each macro action is converted into an executed micro action according to a pre-set principle and input into the environment in sequence for interaction. The macro action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence.

[0011] Step 4: let i=i+1, repeat steps 2 to 3, obtain temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences;

[0012] Step 5: Use the updated database as the database for the next set of interactions between the agent and the environment under multiple tasks, and iterate steps 2 to 4 until the database is finally iteratively updated.

[0013] Furthermore, in step 1, each graph model is encoded to obtain a feature vector of fixed length, specifically including:

[0014] In the graph model, the basic object is the root node and other objects are leaf nodes;

[0015] The root node and leaf node are converted into string type, and the string type is output as a fixed-length feature vector through the sentence feature extraction module.

[0016] Furthermore, in step 2, the observed features are converted into label text and encoded into a query feature vector, specifically:

[0017] Convert observation features into labeled text by using MaskRCNN or TextWorld tools;

[0018] Convert the label text into a graph model, in which the basic object is the root node and other objects are leaf nodes;

[0019] The root node and leaf nodes are converted into string types, and the string types are output as query feature vectors through the sentence feature extraction module.

[0020] Furthermore, in step three, each macro action follows the principle of "the action does not involve specific parameters but only focuses on the object".

[0021] A large language model strategy generation system based on retrieval enhancement, comprising an initial construction module, an observation matching module, an action matching module, a current group database update module and a next group database update module;

[0022] The initial construction module is used to group multiple tasks together. In the initial round, the trajectory data obtained by the interaction between the agent and the environment under multiple tasks are obtained respectively, and different structural data series are separated from the trajectory data. The structural data series includes observation sequences and macro-action sequences. The observation sequence of each task is composed into a graph model. Each graph model is encoded to obtain a feature vector of fixed length and stored in the database together with the macro-action sequence.

[0023] The observation matching module is used to take the i-th task as the current task. In the current round, the observation picture given by the environment under the current task is obtained. The observation picture includes observation features and action features. The observation features are converted into label texts and encoded into query feature vectors. The query feature vectors and all feature vectors in the database are matched by cosine similarity and arranged in descending order. The feature vectors corresponding to the top k similarities are taken from the database, and the taken feature vectors are input into the prompt of the large language model. The graph model corresponding to the taken feature vector is output and encoded, i∈[1,N], where N is the total number of tasks in a group;

[0024] The action matching module is used to retrieve the macro action sequence corresponding to the extracted feature vector from the database. The large language model uses the set prompt word and the selected macro action sequence as an index to select one of the macro actions from the set macro action library. The selected macro action is converted into micro actions to be executed according to the pre-set principle and is input into the environment in sequence for interaction. The macro action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence.

[0025] The database update module under the current group is used to set i=i+1, repeat the observation matching module to the action matching module, obtain the temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences;

[0026] The next group of database update modules is used to use the updated database as the database for the next group of multi-task agents to interact with the environment, iterating the observation matching module to the database update module of the current group until the final iterative update of the database is completed.

[0027] Furthermore, in the initial construction module, each graph model is encoded to obtain a feature vector of fixed length, specifically including:

[0028] In the graph model, the basic object is the root node and other objects are leaf nodes;

[0029] The root node and leaf node are converted into string type, and the string type is output as a fixed-length feature vector through the sentence feature extraction module.

[0030] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the large language model strategy generation method as described above when executing the computer program.

[0031] A computer-readable storage medium stores a plurality of classification programs, wherein the plurality of classification programs are used to be called by a processor and execute the large language model strategy generation method as described above.

[0032] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0033] The advantage of the large language model strategy generation method and system based on retrieval enhancement provided by the present invention is that it combines the advantages of the generation framework of progressive retrieval enhancement and the direct strategy reasoning technology framework based on the large language model (LLM) to solve the limitations of traditional methods in embodied intelligence application scenarios. By combining the progressive retrieval enhancement framework and the large language model, this embodiment has the following beneficial effects: first, the preliminary strategy is generated using the large language model, and then refined and optimized through the progressive retrieval enhancement generation framework, thereby improving the generalization ability and adaptability of the large language model in complex environments. The preliminary strategy foundation provided by the large language model can reduce the demand for training data and time, while the progressive retrieval enhancement generation framework can be customized according to specific scenarios. Secondly, the large language model provides a pre-training strategy, which reduces the large amount of training data and time required by the algorithm in the initial exploration stage, thereby reducing training costs and efficiency issues.

[0034] By combining the strategy generated by the large language model with the precise adjustment of the progressive retrieval enhancement framework, this embodiment can generate more precise and feasible strategies, reducing errors and undesirable results in the execution process. In addition, the database update of this embodiment is that the database after the current group of updates is used as the database for the robot to interact with the environment under the next group of multi-tasks. Therefore, when the environment changes, the strategy can be quickly adjusted by searching in the retrieval-enhanced database without completely retraining, which enhances the flexibility and autonomous learning ability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the process of the present invention;

[0036] Figure 2 Flowchart of the structure of a database enhanced for progressive retrieval;

[0037] Figure 3 It is a structural flow chart of the present invention. DETAILED DESCRIPTION

[0038] Below, the technical solution of the present invention is described in detail through specific embodiments. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

[0039] There are two types of frameworks. One is a technical framework that learns specific strategies for specific scenarios through reinforcement learning. The other is a technical framework that directly reasons about strategies based on a general large language model and establishes a retrieval database of scene interaction trajectories, so that the large language model can autonomously learn and adapt in complex environments.

[0040] 1. A technical framework for learning specific strategies for specific scenarios through reinforcement learning

[0041] Reinforcement Learning (RL) is a machine learning method that optimizes the behavior of an agent by interacting with the environment. This method guides the learning process of the agent through a reward and punishment mechanism, so that it gradually forms a strategy that can maximize the cumulative reward. In the application scenario of embodied intelligence, the RL technology framework usually includes the following core components:

[0042] · State Space: describes all possible states of the agent in the environment.

[0043] · Action Space: All possible actions that the agent can perform in each state.

[0044] · Reward Function: Evaluates the agent’s performance on each state-action pair and provides feedback to the agent through rewards or penalties.

[0045] · Policy: The rule or function by which the agent chooses an action in each state.

[0046] In specific applications, such as indoor robot movement and robotic arm operation tasks, the RL technology framework requires a lot of training for specific scenarios to learn the optimal strategy. For example, when a robot moves in a specific room, it needs to constantly explore and try different paths to find the optimal navigation strategy; when a robotic arm performs a specific task, it needs to determine the best operation sequence and strength through repeated trials.

[0047] Common reinforcement learning strategies include the Actor-Critic (AC) algorithm and the Proximal Policy Optimization (PPO) algorithm.

[0048] The executor-supervisor algorithm uses two modules, Actor and Critic, for training and reasoning. Actor: responsible for learning the strategy π(a|s; θ), directly outputting the action a from the state s; Critic estimates the state value function V(s; w) or the action value function Q(s, a; w), which is used to evaluate the Actor's strategy. Then, by iteratively updating the parameters of the Actor and Critic networks, training and reasoning are performed on different tasks. The proximal policy optimization algorithm maximizes the objective function:

[0049]

[0050] where π θ is the current strategy, It's the old strategy. t is the advantage function, ∈ is the clipping threshold, which is used to limit the amplitude of the policy update.

[0051] 2. Technical framework for direct policy reasoning based on a general large language model

[0052] Different from reinforcement learning, the technical framework based on direct policy reasoning of large language models (LLM) uses pre-trained large-scale neural network models to learn through a large amount of text data, and can directly generate operation strategies without a lot of training for specific scenarios. The core idea of ​​this technical framework is to use the powerful generation and knowledge transfer capabilities of LLM to generate appropriate strategies for specific scenarios.

[0053] Specifically, the LLM-based policy reasoning framework usually includes the following steps:

[0054] · Pre-trained Model: Use massive text data to train a general large language model so that it has rich knowledge and generation capabilities.

[0055] · Scenario Description: Convert the specific scenario and task requirements into natural language description and input it into the model.

[0056] · Policy Generation: The model generates a sequence of movement or manipulation actions that meet the requirements based on the input scene description.

[0057] · Policy Execution and Feedback: The agent executes the generated policy and continuously optimizes the model’s generation capabilities through a feedback mechanism.

[0058] In practical applications, LLM can use its powerful language understanding and generation capabilities to generate specific operation instructions for indoor robot movement and manipulator operation tasks. For example, the robot can understand the room layout and generate a reasonable navigation path based on the input natural language description; the manipulator can infer the appropriate movement position and operation sequence based on the task description.

[0059] This embodiment aims to solve the problems of inefficient reinforcement learning training, the inability of large language models to make customized decisions, and the inability of large language models to make customized judgments and decisions based on the specificity of scenarios when using pre-trained models to generate strategies, as well as the problem of relying on natural language descriptions of inputs. A retrieval database of scenario interaction trajectories is established, and a decision-making system with customized decision-making capabilities is constructed using a retrieval-enhanced large language model. Figure 2 As shown, this embodiment adopts a database with progressive search enhancement, specifically:

[0060] 1) Feature information extraction and storage: Separate structured data pairs (action, observation) from the trajectory data, convert each observation into text using the TextWorld tool, and encode it into fixed-length features using the MiniLM-v6-large text encoding model.

[0061] 2) Information retrieval structure: The data observed in the environment is also converted into feature vectors using TextWorld and MiniLM-v6-large, and then compared with the features obtained in the previous round in the database. The cosine similarity is used to compare and obtain the similarity score, and the top three trajectories are retrieved and output.

[0062] 3) Large model reasoning: The retrieved similar trajectories are input as examples to the large language model, and the macro-actions for the specific environment are output. The macro-actions output by the large language model are decomposed into micro-actions in a predefined way. After a round of interaction, they are encoded into feature vectors in the way of step 1) and updated to the database. Then, steps 1) to 3) are repeated to complete database iteration.

[0063] 1) to 3) combine the advantages of the progressive retrieval enhancement framework technology framework and the large language model (LLM) based direct policy reasoning technology framework to address the limitations of traditional methods in embodied intelligence application scenarios. The following uses the intelligent agent as a robot to specifically illustrate the large language model strategy generation method based on the retrieval enhancement framework.

[0064] like Figures 1 to 3 As shown, the present invention proposes a large language model strategy generation method based on retrieval enhancement, comprising the following steps:

[0065] Step 1: Group multiple tasks into a group. In the initial round, obtain the trajectory data obtained by the robot interacting with the environment under multiple tasks respectively, separate different structural data series from the trajectory data, and the structural data series includes observation sequence and macro action sequence. The observation sequence of each task is combined into a graph model, and each graph model is encoded to obtain a feature vector of fixed length and stored in the database together with the macro action sequence;

[0066] In the initial stage of the interaction between the robot and the environment, because there is no database for this scenario, in the initial round, we first need to obtain the trajectory data of the interaction between the robot and the environment, separate different structural data sequences from the trajectory data text, organize the observation sequence into a graph model, and encode the graph model. The specific encoding process is: in the graph model, the basic objects are used as root nodes and other objects are used as leaf nodes; the root nodes and leaf nodes are converted into string types, and the string types are output as fixed-length feature vectors through the sentence feature extraction module (MiniLM-v6-Large).

[0067] Step 2: Take the i-th task as the current task. In the current round, obtain the observation picture given by the environment under the current task. The observation picture includes observation features and action features. Convert the observation features into label text and encode them into query feature vectors. Perform cosine similarity matching between the query feature vector and all feature vectors in the database and arrange them in descending order. Take the feature vectors corresponding to the top k similarities from the database, input the extracted feature vectors into the prompt of the large language model, output the graph model corresponding to the extracted feature vector and encode it, i∈[1,N], N is the total number of tasks in a group;

[0068] Except for the initial round, which is a database built based on the trajectory data of environmental interactions and does not need to be retrieved, other rounds need to be retrieved from the database. The specific retrieval process is as follows:

[0069] (a1) Before making an action decision, the current environment will obtain an observation picture given by the environment. The observation picture includes observation features and action features. The observation features are converted into label text by using MaskRCNN or TextWorld tools, and then the query feature vector is converted through MiniLM-v6-Large as in step 1. Specifically, the label text is organized into a graph model, in which the basic objects are the root nodes and other objects are the leaf nodes; the root nodes and leaf nodes are converted into string types, and the string types are output as query feature vectors through MiniLM-v6-Large.

[0070] (a2) Perform cosine similarity matching between the query feature vector and all feature vectors in the database:

[0071]

[0072] Among them, Sim represents similarity, Q represents the query feature vector, and K represents the feature vector in the database.

[0073] (a3) Take out the trajectories corresponding to the k (for example, 3) feature vectors with the highest similarity matching values ​​and input them into the prompt of the large language model to obtain the corresponding graph model for the current task. For example, if there are 200 tasks in a group, there are 200 corresponding graph models. Through similarity matching, select the 3 graph models that are most similar to the current task from the 200 graph models to determine the macro action sequence.

[0074] Therefore, this embodiment uses a large language model to generate a preliminary strategy, which is then refined and optimized through a generation framework enhanced by progressive retrieval, thereby improving the generalization ability and adaptability of the model in complex environments.

[0075] Step 3: The macro action sequence corresponding to the extracted feature vector is extracted from the database. The large language model selects one of the macro actions from the set macro action library with the set prompt word and the selected macro action sequence as indexes. The selected macro action is converted into micro actions to be executed according to the pre-set principle and input into the environment in sequence for interaction. The macro action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence.

[0076] In step 2, three feature vectors are taken from the database. These three feature vectors correspond to three macro action sequences respectively. The selected three macro action sequences and the set prompt words are used as indexes to select one of the macro actions from the macro action library. The design of each macro action follows the principle of "action does not involve specific parameters but only focuses on objects". The prompt words are set to let the large language model select one of the macro actions. The macro actions are converted into each micro action that is actually executed according to the pre-set principle and input into the environment in sequence, so as to obtain the macro action after interaction. A macro action is selected according to the prompt words, and the macro action corresponds to a graph model at the same time. Therefore, the temporary feature sequence is the selected macro action and the corresponding encoded graph model; in addition, the macro action library is a set of all macro actions set in advance according to the environment.

[0077] For the redundant information that may appear after the large language model is inferred, it is filtered through the preset regular matching rules to obtain the macro actions that meet the conditions. If there are none, it will be retried.

[0078] The regular matching rules are as follows:

[0079] Define the macro action set A in the alternative macro action library as:

[0080] A={a 1 ,a 2 ,a 3 ,…,a n}

[0081] Among them, a n Indicates the nth macro action;

[0082] The regular expression pattern is:

[0083] Pattern = a 1 ∣a 2 ∣a 3 ∣…∣a n

[0084] Based on the k (for example, 3) feature vectors inferred from the large language model, the macro action sequence Matches is matched from the database:

[0085] Matches={m∈T|m∈A}

[0086] Wherein, T is the text corresponding to k (for example, 3) feature vectors, m is the macro action sequence to be matched, and if there are 3 feature vectors, 3 macro action sequences will be matched;

[0087] Finally, use Matches and the set prompt word as indexes to select a macro action from the macro action library as a macro action that meets the conditions.

[0088] Step 4: let i=i+1, repeat steps 2 to 3, obtain temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences;

[0089] This embodiment treats multiple tasks as a group, and updates the database after a group of data completes the interaction, so that the database can be updated in parallel, thereby accelerating the iteration of the database.

[0090] Step 5: Use the updated database as the database for the next set of multi-task interactions between the robot and the environment, and iterate steps 2 to 4 until the database is finally iteratively updated.

[0091] According to steps one to five, this embodiment combines the advantages of the generation framework of progressive retrieval enhancement and the direct policy reasoning technology framework based on the large language model (LLM) to solve the limitations of traditional methods in embodied intelligence application scenarios. By combining the progressive retrieval enhancement framework and the large language model, this embodiment has the following beneficial effects: First, the preliminary strategy is generated using the large language model, and then refined and optimized through the progressive retrieval enhancement generation framework (steps two to five), which improves the generalization ability and adaptability of the large language model in complex environments. The preliminary strategy foundation provided by the large language model can reduce the demand for training data and time, while the progressive retrieval enhancement generation framework can be customized according to specific scenarios. Secondly, the large language model provides a pre-training strategy, which reduces the large amount of training data and time required by the algorithm in the initial exploration stage, thereby reducing training costs and efficiency issues.

[0092] By combining the strategy generated by the large language model with the precise adjustment of the progressive retrieval enhancement framework, this embodiment can generate more precise and feasible strategies, reducing errors and undesirable results in the execution process. In addition, the database update of this embodiment is that the database after the current group of updates is used as the database for the robot to interact with the environment under the next group of multi-tasks. Therefore, when the environment changes, the strategy can be quickly adjusted by searching in the retrieval-enhanced database without completely retraining, which enhances the flexibility and autonomous learning ability of the system.

[0093] At the same time, this embodiment uses a large language model to provide guidance in the early stage of strategy generation, reducing the need for repeated trials of the reinforcement learning algorithm during the training process, thereby reducing the risk of damage to the robot or danger in actual operation; that is, using the general knowledge of the large language model itself can reduce unnecessary exploration by the system, and by defining macro actions, the system can avoid making meaningless or dangerous macro actions.

[0094] Although the strategy generation of the large language model depends on the natural language description of the input, by combining the feedback mechanism of the progressive retrieval enhancement framework, the strategy can be continuously optimized and adjusted to make it more in line with the needs of specific scenarios. Even if the input description is not clear or accurate enough, it can be corrected and improved through the progressive retrieval enhancement framework.

[0095] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A large language model strategy generation method based on retrieval enhancement, characterized in that: The steps include: Step 1: Group multiple tasks into a group. In the initial round, obtain the trajectory data obtained by the interaction between the agent and the environment under multiple tasks respectively, separate different structural data series from the trajectory data, and the structural data series includes observation sequence and macro action sequence. Organize the observation sequence of each task into a graph model, encode each graph model to obtain a fixed-length feature vector and store it in the database together with the macro action sequence; Step 2: Take the i-th task as the current task. In the current round, obtain the observation picture given by the environment under the current task. The observation picture includes observation features and action features. Convert the observation features into label text and encode them into query feature vectors. Perform cosine similarity matching between the query feature vector and all feature vectors in the database and arrange them in descending order. Take the feature vectors corresponding to the top k similarities from the database, input the extracted feature vectors into the prompt of the large language model, output the graph model corresponding to the extracted feature vector and encode it, i∈[1,N], N is the total number of tasks in a group; Step 3: The macro-action sequence corresponding to the extracted feature vector is extracted from the database. The large language model uses the set prompt word and the selected macro-action sequence as an index to select one of the macro-actions from the set macro-action library. The selected macro-action is converted into micro-actions to be executed according to the pre-set principle and input into the environment in sequence for interaction. The macro-action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence. Step 4: let i=i+1, repeat steps 2 to 3, obtain temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences; Step 5: Use the updated database as the database for the next set of interactions between the agent and the environment under multiple tasks, and iterate steps 2 to 4 until the database is finally iteratively updated.

2. The method for generating a large language model strategy based on retrieval enhancement according to claim 1, characterized in that: In step 1, each graph model is encoded to obtain a feature vector of fixed length, which specifically includes: In the graph model, the basic object is the root node and other objects are leaf nodes; The root node and leaf node are converted into string type, and the string type is output as a fixed-length feature vector through the sentence feature extraction module.

3. The method for generating a large language model strategy based on retrieval enhancement according to claim 1, characterized in that: In step 2, the observed features are converted into label text and encoded into a query feature vector, specifically: Convert observation features into labeled text by using MaskRCNN or TextWorld tools; Convert the label text into a graph model, in which the basic object is the root node and other objects are leaf nodes; The root node and leaf nodes are converted into string types, and the string types are output as query feature vectors through the sentence feature extraction module.

4. The method for generating a large language model strategy based on retrieval enhancement according to claim 1, characterized in that: In step three, each macro action follows the principle of "action does not involve specific parameters but only focuses on objects".

5. A large language model strategy generation system based on retrieval enhancement, characterized in that: It includes an initial construction module, an observation matching module, an action matching module, a current group database update module and a next group database update module; The initial construction module is used to group multiple tasks together. In the initial round, the trajectory data obtained by the interaction between the agent and the environment under multiple tasks are obtained respectively, and different structural data series are separated from the trajectory data. The structural data series includes observation sequences and macro-action sequences. The observation sequence of each task is composed into a graph model. Each graph model is encoded to obtain a feature vector of fixed length and stored in the database together with the macro-action sequence. The observation matching module is used to take the i-th task as the current task. In the current round, the observation picture given by the environment under the current task is obtained. The observation picture includes observation features and action features. The observation features are converted into label texts and encoded into query feature vectors. The query feature vectors and all feature vectors in the database are matched by cosine similarity and arranged in descending order. The feature vectors corresponding to the top k similarities are taken from the database, and the taken feature vectors are input into the prompt of the large language model. The graph model corresponding to the taken feature vector is output and encoded, i∈[1,N], where N is the total number of tasks in a group; The action matching module is used to retrieve the macro action sequence corresponding to the extracted feature vector from the database. The large language model uses the set prompt word and the selected macro action sequence as an index to select one of the macro actions from the set macro action library. The selected macro action is converted into micro actions to be executed according to the pre-set principle and is input into the environment in sequence for interaction. The macro action after interaction and the corresponding encoded graph model in step 2 are temporarily stored as a temporary feature sequence. The database update module under the current group is used to set i=i+1, repeat the observation matching module to the action matching module, obtain the temporary feature sequences corresponding to all tasks, and update the database based on all temporary feature sequences; The next group of database update modules is used to use the updated database as the database for the next group of multi-task agents to interact with the environment, iterating the observation matching module to the database update module of the current group until the final iterative update of the database is completed.

6. The method for generating a large language model strategy based on retrieval enhancement according to claim 5, characterized in that: In the initial building block, each graph model is encoded to obtain a fixed-length feature vector, including: In the graph model, the basic object is the root node and other objects are leaf nodes; The root node and leaf node are converted into string type, and the string type is output as a fixed-length feature vector through the sentence feature extraction module.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the large language model strategy generation method as described in any one of claims 1-4 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of classification programs, and the plurality of classification programs are used to be called by a processor and execute the large language model strategy generation method as described in any one of claims 1-4.

Images