Robot control strategy optimization system based on large language model and reinforcement learning
By employing a dual-channel enhancement framework combining a large language model and reinforcement learning, the robot task is decomposed and dense reward signals are generated, solving the sparse reward problem in long-term complex robot tasks and achieving efficient optimization of robot control strategies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2026-05-28
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies suffer from problems such as low learning efficiency due to sparse rewards, high computational costs, and a disconnect between high-level planning and low-level control in long-term complex tasks for robots.
A high-level task decomposition module and a low-level progress reward module based on a large language model are adopted to generate a sub-task planning list and progress reward signals. Combined with a reinforcement learning module, dual-channel enhancement is performed to achieve efficient optimization of robot control strategy.
It improves the learning efficiency and success rate of robots in handling long-term complex tasks, reduces training time and computing resource consumption, and avoids blind exploration in traditional sparse reward environments.
Smart Images
Figure CN122480967A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and robot control, and specifically to a robot control strategy optimization system based on large language models and reinforcement learning. Background Technology
[0002] With the development of artificial intelligence technology, reinforcement learning has shown great potential in robot sequential decision-making tasks. However, in real-world robot control scenarios, tasks are often long-term and complex, and the reward signals from environmental feedback are typically very sparse. For example, a robot may need to complete a series of actions—"grasping, rotating, and placing"—before receiving a successful reward, while the rewards obtained in the intermediate stages are zero. This sparse reward makes it difficult for the agent to obtain effective gradient signals, resulting in extremely low learning efficiency or failure to converge.
[0003] To address the aforementioned issues, existing technologies are mainly divided into two paradigms: The first is to use a Large Language Model (LLM) as a planner to decompose complex tasks into subtasks. However, this approach often neglects the underlying reward design and lacks dense feedback signals. The second is to use an LLM as a rewarder to directly generate reward functions. However, without task decomposition, it is extremely difficult to require an LLM to directly construct reward functions for a globally complex task, and it often requires a large amount of manual feedback or expensive search algorithms.
[0004] In summary, existing technologies either focus on planning but lack reward guidance, or focus on reward generation but lack task decomposition. This leads to problems such as low learning efficiency, high computational cost, and difficulty in end-to-end training when dealing with long-term sparse reward tasks. Therefore, how to unify high-level planning and low-level reward generation to achieve efficient, end-to-end control of complex robot tasks, avoid blind exploration in traditional sparse reward environments, reduce training time and computational resource consumption, and enable robots to adapt to new operational tasks more quickly remains a technical challenge that urgently needs to be solved in this field. Summary of the Invention
[0005] The purpose of this invention is to provide a robot control policy optimization system and method based on large language models and reinforcement learning, aiming to solve the technical problems of sparse rewards, low learning efficiency, and the disconnect between high-level planning and low-level control faced by robots in handling long-term complex tasks in existing technologies. The system of this invention employs a unified, dual-channel enhancement framework. At the high level, it utilizes LLM to decompose the task and provides "sub-task priors" for the policy to enhance the state space. At the low level, it uses the "progress function" generated by LLM to construct a guaranteed "progress reward model" to enhance the reward function, thereby improving the training convergence speed and learning efficiency of the robot control policy. The dense signal generated by the low-level progress reward module in this invention provides clear guidance for the robot's joint control commands at each step, avoiding blind exploration in traditional sparse reward environments, significantly reducing training time and computational resource consumption, and enabling the robot to adapt to new operational tasks more quickly.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0007] In a first aspect, the present invention provides a robot control policy optimization system based on a large language model and reinforcement learning. The system includes: a high-level task decomposition module, a low-level progress reward module, and a reinforcement learning module, wherein:
[0008] The high-level task decomposition module is used to receive robot task descriptions, generate a sub-task planning list and a task progress function to evaluate task completion using a large language model.
[0009] The underlying progress reward module is used to receive the task progress function, robot state and original sparse success reward, use the task progress function as potential energy function, calculate the time difference of the function based on the underlying progress reward model, and generate dense reward signal.
[0010] The reinforcement learning module enhances the Markov decision process through two parallel channels, uses the sub-task planning list as a state prior to enhance the high-level state, and uses the dense reward signal to drive the policy network for updates, outputting the robot's control policy.
[0011] Optionally, the high-level task decomposition module adopts a program-assisted thought chain reasoning approach, instructing the large language model to output the reasoning process and results in executable Python code format, and generating a sub-task planning list and progress evaluation function at once.
[0012] Optionally, the subtask planning list is a Python list, whose elements are natural language strings describing each subtask, serving as a source of subtask priors for high-level state enhancement.
[0013] Optionally, the task progress function is an executable, fine-grained Python function that evaluates the overall completion of the robot task in the current state by integrating the progress of all subtasks.
[0014] Optionally, in the underlying progress reward module, the calculation formula for the underlying progress reward model is as follows:
[0015]
[0016] in: Is Dense reward signals generated at all times; It is a discount factor; express Current state express Current state; Representing state Task progress, Representing state Task progress; It is an indicator function, when the state The value is 1 if the task success condition is met, and 0 otherwise. This represents the original sparse success reward value.
[0017] Optionally, the reinforcement learning module includes:
[0018] The state enhancement unit is used to encode the subtask planning list and concatenate it into the robot's current state vector to form an enhanced state containing high-level semantic priors.
[0019] The policy optimization unit employs either a soft actor-critic (SAC) algorithm based on the maximum entropy reinforcement learning framework or a proximal policy optimization (PPO) algorithm, and is trained based on the augmented state and reward signals to update the policy.
[0020] Optionally, the robot task description received by the high-level task decomposition module includes: environmental information, state space, and a description of the task objective.
[0021] Secondly, this invention also provides a robot control strategy optimization method based on a large language model and reinforcement learning. This method applies the aforementioned system to optimize robot control strategies and includes the following steps:
[0022] Step 1: Receive the robot task description through the high-level task decomposition module, and use the large language model to generate a sub-task planning list and a task progress function to evaluate the task completion rate.
[0023] Step 2: Receive the task progress function, robot status, and original sparse success reward through the underlying progress reward module. Calculate the time difference of the function based on the underlying progress reward model to generate a dense reward signal.
[0024] Step 3: Through the reinforcement learning module, the Markov decision process is enhanced using two parallel channels. The sub-task planning list is used as a state prior to enhance the high-level state, and the dense reward signal is used to drive the policy network for updating, outputting an optimized robot control policy.
[0025] Compared with the prior art, the present invention has at least the following beneficial effects:
[0026] This invention provides a robot control strategy optimization system based on large language models and reinforcement learning. It adopts a novel framework that unifies and integrates high-level task planning and low-level reward shaping, effectively bridging the gap between high-level semantic planning and low-level sparse rewards. This is beneficial to improving the learning efficiency and success rate of robots in handling long-term complex tasks, and helps to achieve efficient, end-to-end control of complex robot tasks. It avoids blind exploration in traditional sparse reward environments, reduces training time and computational resource consumption, and enables robots to adapt to new operational tasks more quickly.
[0027] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.
[0028] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0029] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.
[0030] Figure 1 This is a schematic diagram of the overall system architecture provided by the present invention.
[0031] Figure 2 This is a pseudocode diagram illustrating the subtask planning prior provided by the present invention.
[0032] Figure 3 A pseudocode diagram illustrating the task progress function provided by this invention. Detailed Implementation
[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The accompanying drawings are intended to exemplarily illustrate the system architecture, interface layout, data processing flow, and core analysis functions involved in the present invention. Those skilled in the art can understand the essence of the present invention by referring to the accompanying drawings and specific embodiments.
[0034] In the description of this invention, it should be noted that some processes described in this application specification and accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may be performed in any order or in parallel. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention.
[0035] See Figure 1 As shown, this invention discloses a robot control strategy optimization system based on a large language model and reinforcement learning. The system includes: a high-level task decomposition module, a low-level progress reward module, and a reinforcement learning module, wherein:
[0036] The high-level task decomposition module is used to receive robot task descriptions, generate a sub-task planning list and a task progress function to evaluate task completion using a large language model;
[0037] The underlying progress reward module is used to receive the task progress function, robot status and the original sparse success reward, and generate a dense reward signal based on the time difference of the calculation function based on the underlying progress reward model.
[0038] The reinforcement learning module enhances the Markov decision process through two parallel channels. It uses the sub-task planning list as a state prior to enhance the high-level state and uses dense reward signals to drive the policy network for updates, outputting the robot's control policy.
[0039] In this invention, the high-level task decomposition module is the "planning brain" of the system. It utilizes the reasoning capabilities of large language models (LLMs) to generate all the components required for high-level planning at once before training begins.
[0040] The input to this module is a contextual description of the robot task, including environmental information, state space, and task objective description.
[0041] The processing mechanism of this module: This invention employs a program-assisted thought chain (prompt) LLM. The LLM instructs thought and output in executable Python code format.
[0042] The output of this module (a set of key components):
[0043] 1. Subtask planning list (prior): A natural language list that breaks down a long-term task into a short sequence of subtasks;
[0044] 2. Task progress function ( ): An executable, fine-grained function that evaluates the overall task completion status in the current state by integrating the progress of all subtasks.
[0045] In this invention, the underlying progress reward module is the system's "reward engine," which is responsible for transforming the "progress function" generated by the high-level task decomposition module into a dense reward signal.
[0046] The input to this module is the progress function generated by the high-level task decomposition module. Current state Next state And the original sparse success reward .
[0047] The processing mechanism of this module is based on potential energy-based reward shaping. The core innovation of this invention lies in not directly using a progress function. As a reward. Instead, the present invention will... It is considered as a potential energy function. The reward of the progress reward model of this invention... It is constructed by calculating the time difference of the potential energy function.
[0048] The module's output is a dense, efficient reward signal. .
[0049] In this invention, the reinforcement learning module is the "executor" of the system, which simultaneously feeds the outputs of the high-level task decomposition module (planning) and the low-level progress reward module (reward) to a standard reinforcement learning (RL) algorithm.
[0050] The inputs to this module are: the original Markov Decision Process (MDP), the output of the high-level task decomposition module (prior), and the output of the low-level progress reward module. ).
[0051] The processing mechanism of this module: This invention enhances the standard MDP through two parallel channels.
[0052] Channel 1: High-level state enhancement: The prior generated by the high-level task decomposition module is concatenated into the state representation.
[0053] Channel Two: Enhanced Underlying Rewards: Learning is guided by the reward functions generated by the underlying progress reward module.
[0054] The output of this module is the trained policy.
[0055] Therefore, it can be seen that the system framework of this invention innovatively unifies high-level planning and low-level reward generation; wherein:
[0056] In terms of high-level planning: This invention utilizes LLM to decompose complex tasks into a series of manageable subtasks and provides subtask priors for the strategy.
[0057] At the underlying reward level: This invention does not allow the LLM to directly generate rewards, but rather, inspired by the potential energy-based reward shaping theory, instructs the LLM to construct a fine-grained "progress function". Subsequently, this invention utilizes the time difference form of the progress function to construct a progress reward model.
[0058] The present invention achieves the following effects through the above-described means:
[0059] Effect 1 (High Performance): This invention effectively bridges the gap between high-level planning and low-level control.
[0060] Effect 2 (High Efficiency): The dense signal provided by the progress reward model significantly improves the convergence speed. Simultaneously, through task decomposition, this invention simplifies the reward generation process, avoiding reliance on extensive manual feedback or expensive searches.
[0061] The specific embodiments and working principles of the present invention are described in detail below:
[0062] Overall Architecture: This invention provides a unified, dual-channel augmentation reinforcement learning framework, the overall architecture of which is as follows: Figure 1 As shown. This framework aims to bridge the technological gap existing in the prior art. The architecture of this invention consists of three core modules:
[0063] Module 1: High-level Task Decomposition Module;
[0064] Module 2: Underlying Progress Reward Module;
[0065] Module 3: Reinforcement Learning Module.
[0066] Overall working principle: The working principle of this invention can be summarized as: "One-time high-level planning, full-process dual-channel enhancement":
[0067] (1) High-level planning (offline): Before training begins, the operator provides Module 1 with a high-level hint. Module 1 uses a "program-assisted thought chain" to generate key executable Python components in one go: a subtask planning list (prior) and a task progress function. .
[0068] (2) Dual-channel enhancement throughout: In each step t of RL training:
[0069] Channel 1 (High-Level State Enhancement): Module 3 retrieves the corresponding "subtask prior" from the prior generated by Module 1. This prior is encoded and concatenated into the current state. middle.
[0070] Channel Two (Enhanced Underlying Rewards): Module Two calls the progress function generated by Module One. And based on the current state and the next state A dense reward is calculated using the core reward formula of this invention. .
[0071] (3) Strategy optimization: Module 3 replaces the original sparse reward with The enhanced state and reward are then fed together to a standard RL optimization algorithm (such as SAC or PPO) for training.
[0072] Functionality Achieved: This invention implements an end-to-end system capable of autonomously transforming a complex, sparse-reward, long-term task into an easily solvable, enhanced Markov decision process with dense rewards and high-level context guidance. The functions of each module are as follows:
[0073] Module 1: High-Level Task Decomposition Module
[0074] This module is the system's "planning brain." It utilizes the reasoning capabilities of a large language model (LLM) to generate all the components required for high-level planning in one go before training begins.
[0075] 1. Input: The input to this module is a "Pythonic hint". This hint contains rich contextual information about the task, such as:
[0076] State space: describes all the state variables that the agent can observe.
[0077] Environment description: Natural language description of the environment (such as robots, desktops, objects).
[0078] Task description: The overall objective that needs to be achieved.
[0079] 2. Module working principle: This invention adopts a program-assisted reasoning method based on a thought chain.
[0080] Step 1: Instruct the LLM to generate comments in Python for its reasoning process (e.g., using # syntax).
[0081] Step 2: Instruct the LLM to generate executable Python statements from its intermediate steps and final response (i.e., the output of this module).
[0082] In this way, the present invention can generate high-level plans and all necessary low-level executable functions simultaneously with only one LLM interaction.
[0083] 3. Module output:
[0084] (1) Subtask planning (prior):
[0085] Composition: A Python list whose elements are natural language strings describing each subtask; for example, ['Move to object', 'Grasp object', 'Rotate object'].
[0086] Purpose: As a source of "subtask priors" for high-level state enhancement in Module 3.
[0087] (2) Task progress function :
[0088] Composition: An executable Python function that takes the current state s as input and outputs a scalar value representing the overall progress of the task.
[0089] Purpose: This function is the core input of Module 2. It is not a reward in itself, but is used as a potential energy function.
[0090] Module Two: Underlying Progress Reward Module (Underlying Progress Reward Model):
[0091] This module is responsible for processing the "progress function" generated by module one. "Transformed into a highly efficient and dense reward signal."
[0092] 1. Input: The progress function generated by Module 1 Current state Next state And the original sparse success reward .
[0093] 2. Working Principle: Reward shaping based on potential energy. The core innovation of this invention lies in not directly using a progress function. As a reward. Instead, the present invention will... It is considered as a potential energy function. The progress reward model of this invention rewards... It is constructed by calculating the time difference of the potential energy function, as defined below:
[0094]
[0095] in: This is the dense reward signal generated by the present invention at time t; It is a discount factor, consistent with the one defined in MDP; It is an indicator function, when the state The value is 1 if the task success condition is met, and 0 otherwise. This represents the original sparse success reward value. The original sparse reward of the environment; The potential energy shaping term constructed for this invention.
[0096] 3. Output: A dense and efficient reward signal .
[0097] Module 3: Reinforcement Learning Module
[0098] This module is the system's "executor," feeding the outputs of Module 1 (planning) and Module 2 (rewards) simultaneously into a standard RL algorithm.
[0099] 1. Input: Original MDP, output of Module 1 (prior), output of Module 2 ( ).
[0100] 2. Processing Mechanism: This invention enhances the standard MDP through two parallel channels.
[0101] Channel 1: High-level state enhancement: Concatenate the prior generated by Module 1 into the state representation.
[0102] Channel Two: Enhanced Underlying Rewards: Learning is guided by the reward function generated in Module Two.
[0103] 3. Output: The trained policy.
[0104] The unified framework of this invention simultaneously bridges two technological gaps in the prior art. High performance is achieved through efficient function generation in Module 1, theoretically guaranteed rewards in Module 2, and dual-channel integration in Module 3.
[0105] The following example, using a multi-stage robot operation task, illustrates the specific workflow of the system of this invention:
[0106] Example: Multi-stage robot operation task:
[0107] Example Background: This example selects a typical robot operation task in the Metaworld simulation environment.
[0108] Task objective: A Sawyer robot needs to grab a blue object (Object) on a table (Grasp), rotate it (Rotate) to a specific angle, and then move it (Move) to a designated target (Goal) location.
[0109] (1) Input: Before training begins, the operator calls the LLM (e.g., GPT-4o) only once. The input is a "Pythonic hint" (PROMPT) containing environmental information, such as a state space description and a task objective description.
[0110] (2) LLM processing (program-assisted thought chain).
[0111] (3) The LLM output generates the following two components at once:
[0112] Component 1: prior, pseudocode as follows Figure 2 As shown;
[0113] Component 2: Progress function; pseudocode as follows Figure 3 As shown.
[0114] (4) Dual-channel enhanced reinforcement learning training:
[0115] By concatenating the prior in the state, the reward is replaced with the underlying progress reward model.
[0116] (5) Obtain robot control strategies after reinforcement learning training.
[0117] In summary, this invention discloses a robot control policy optimization system based on a large language model and reinforcement learning. This system includes a high-level task decomposition module, a low-level progress reward module, and a reinforcement learning module. Specifically: the high-level task decomposition module uses a large language model to decompose complex tasks offline into a sequence of sub-tasks and generates corresponding task progress functions; the low-level progress reward module, based on potential energy reward shaping theory, constructs a dense progress reward signal using the time difference of the task progress function; and the reinforcement learning module drives the robot policy network through a dual-channel mechanism of state enhancement and reward enhancement to perform end-to-end training, thereby obtaining the optimal robot control policy. This invention, through a unified, dual-channel enhanced reinforcement learning framework, effectively bridges the gap between high-level semantic planning and low-level sparse rewards, which is beneficial for improving the learning efficiency and success rate of robots handling long-term complex tasks.
[0118] This invention proposes for the first time a novel framework that unifies and integrates high-level task planning with low-level reward shaping. The advantage of this invention lies in the fact that it not only solves the "technology gap" of existing technologies through synergistic effects, but also ensures the convergence efficiency of reward signals by introducing potential energy-based rewards.
[0119] This invention proposes a unified, dual-channel enhancement framework. At the high level, it utilizes LLM to decompose tasks and provides "subtask priors" for the policy to enhance the state space; at the low level, it uses the "progress function" generated by LLM to construct a guaranteed "progress reward model" to enhance the reward function. This unified framework simultaneously bridges the "technology gap" between the two existing technologies, achieving a "1+1>2" effect.
[0120] The core technological innovation of this invention lies in not directly using the progress generated by LLM. As a reward. Instead, inspired by the reward-shaping theory based on potential energy, it will... This can be considered as a potential energy function. The progress reward model constructed in this invention is built by calculating the time difference of this potential energy function.
[0121] This invention solves the high computational overhead caused by "calling LLM at every step" in existing LLM-assisted methods during training, achieving the advantages of "lightweight and high efficiency". This invention designs a highly efficient architecture of "generation once, reuse throughout", where LLM is only called once before training begins.
[0122] Furthermore, this invention also provides a robot control strategy optimization method based on a large language model and reinforcement learning. This method applies the above-mentioned system to optimize the robot control strategy and includes the following steps:
[0123] Step 1: Receive the robot task description through the high-level task decomposition module, and use the large language model to generate a sub-task planning list and a task progress function to evaluate the task completion rate.
[0124] Step 2: Receive the task progress function, robot status, and original sparse success reward through the underlying progress reward module. Calculate the time difference of the function based on the underlying progress reward model to generate a dense reward signal.
[0125] Step 3: Through the reinforcement learning module, the Markov decision process is enhanced using two parallel channels. The sub-task planning list is used as a state prior to enhance the high-level state, and the dense reward signal is used to drive the policy network for updating, outputting an optimized robot control policy.
[0126] The method provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned system embodiment. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the aforementioned system embodiment, and will not be repeated here.
[0127] Those skilled in the art will understand that embodiments of the present invention can be provided as systems, methods, devices, or computer program products, etc. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.
[0128] It should be noted that the word "comprising" does not exclude the presence of components or steps not listed in the claims. The words "a" or "an" preceding a component do not exclude the presence of a plurality of such components. This invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer.
[0129] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0130] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A robot control policy optimization system based on large language models and reinforcement learning, characterized in that, The system includes: a high-level task decomposition module, a low-level progress reward module, and a reinforcement learning module, among which: The high-level task decomposition module is used to receive robot task descriptions, generate a sub-task planning list and a task progress function to evaluate task completion using a large language model. The underlying progress reward module is used to receive the task progress function, robot state and original sparse success reward, use the task progress function as potential energy function, calculate the time difference of the function based on the underlying progress reward model, and generate dense reward signal. The reinforcement learning module enhances the Markov decision process through two parallel channels, uses the sub-task planning list as a state prior to enhance the high-level state, and uses the dense reward signal to drive the policy network for updates, outputting the robot's control policy.
2. The system according to claim 1, characterized in that, The high-level task decomposition module adopts a program-assisted thought chain reasoning approach, instructing the large language model to output the reasoning process and results in executable Python code format, and generating a sub-task planning list and progress evaluation function at once.
3. The system according to claim 1, characterized in that, The subtask planning list is a Python list, whose elements are natural language strings describing each subtask, serving as a source of subtask priors for high-level state enhancement.
4. The system according to claim 1, characterized in that, The task progress function is an executable, fine-grained Python function that evaluates the overall completion of the robot task in the current state by integrating the progress of all subtasks.
5. The system according to claim 1, characterized in that, In the underlying progress reward module, the calculation formula for the underlying progress reward model is as follows: in: Is Dense reward signals generated at any given time; It is a discount factor; express Current state express Current state; Representing state Task progress, Representing state Task progress; It is an indicator function, when the state The value is 1 if the task success condition is met, and 0 otherwise. This represents the original sparse success reward value.
6. The system according to claim 1, characterized in that, The reinforcement learning module includes: The state enhancement unit is used to encode the subtask planning list and concatenate it into the robot's current state vector to form an enhanced state containing high-level semantic priors. The policy optimization unit uses the SAC or PPO algorithm to train and update the policy based on the augmented state and reward signals.
7. The system according to claim 1, characterized in that, The robot task description received by the high-level task decomposition module includes: environmental information, state space, and a description of the task objective.
8. A method for optimizing robot control strategies based on large language models and reinforcement learning, characterized in that, This method applies the system as described in any one of claims 1–7 to optimize robot control strategies.