Robot motion control optimization method, device, equipment and medium

Abstract

The invention discloses a robot motion control optimization method, device, equipment and medium, and the method comprises the steps: inputting task description in a natural language form, robot state information, a historical execution track and environment feedback information into a preset large language model; through the large language model, selecting a target motion strategy of the robot according to the task description; when it is detected that the task of the robot fails or the action of the robot is abnormal, a reward function used for controlling a deep reinforcement learning controller of the robot is reconstructed; and when the state of the robot is abnormal, parameters of the deep reinforcement learning controller are adjusted and optimized, the motion stability, the task completion efficiency and the strategy adaptability of the robot in a high-risk, high-temperature and high-complexity scene are remarkably improved, and the self-adaptive adjusting and optimizing capability and the abnormity recovery capability of the robot in a dynamic environment are enhanced.

Description

Technical Field

[0001] This application relates to the field of robot control technology, specifically to a robot motion control optimization method, apparatus, equipment, and medium. Background Technology

[0002] Robots, with their exceptional mobility and strong environmental adaptability, have been widely used in various fields such as industrial inspection, disaster relief, and high-risk operations. Deep reinforcement learning (DRL), as an advanced control method, has been successfully applied to improve robots' balance, path planning, and motion coordination, greatly enhancing their autonomous movement capabilities in complex environments.

[0003] Existing deep reinforcement learning-based robot control systems still face several unresolved issues that limit their efficient operation in practical applications. Firstly, current deep reinforcement learning methods are highly dependent on specific environmental settings during training. This over-reliance severely restricts the robot's generalization ability, making it difficult to maintain stable and reliable performance in diverse real-world application scenarios. Secondly, the design of the reward function is crucial in deep reinforcement learning, directly impacting the quality of the agent's learned strategy. However, current reward function designs often rely on the experience of domain experts, lacking systematicity and flexibility, and are prone to leading to local optima. Thirdly, in practical applications, robots often face multi-tasking and dynamically changing environments. When a robot needs to switch between different tasks, existing methods often struggle to adjust strategies and parameters in real time, resulting in low task execution efficiency and even potential task failure. This poor multi-tasking adaptability severely restricts the application value of robots in complex industrial scenarios.

[0004] With the rapid development of Large Language Models (LLMs), these models can process and understand complex natural language instructions, generate meaningful feedback, and learn and optimize based on large amounts of historical data. This provides new ideas and methods for solving the problems existing in reinforcement learning control systems. Therefore, how to effectively integrate Large Language Models with deep reinforcement learning algorithms to achieve autonomous and optimized control of robots is a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides a robot motion control optimization method, apparatus, equipment, and medium, which can solve the problems of poor generalization ability of robot motion control in complex industrial environments, difficulty in designing reward functions, and insufficient multi-task adaptability in the prior art.

[0006] In a first aspect, embodiments of this application provide a robot motion control optimization method, the robot motion control optimization method comprising: Input the task description, robot state information, historical execution trajectory, and environmental feedback information in natural language form into a pre-defined large language model, and execute the task through the large language model. Select the robot's target motion strategy based on the task description; When the robot's task fails or its actions are abnormal, the reward function of the deep reinforcement learning controller used to control the robot is reconstructed. When the robot's state is abnormal, the parameters of the deep reinforcement learning controller are optimized.

[0007] In conjunction with the first aspect, in one implementation method: The task description includes: task objectives and expected task completion standards; The robot's state information includes: robot posture angles, joint angles, speed, position, and energy consumption; The historical execution trajectory includes: a sequence of state-action-reward feedback data; The environmental feedback information includes: task completion rate, anomaly markers, and action stability indicators.

[0008] In conjunction with the first aspect, in one implementation, selecting a target motion strategy for the robot based on the task description using the large language model includes: The large language model generates semantic cue vectors based on the task description. Based on a preset mapping function and the semantic cue vector, the robot generates the probability of choosing its motion strategy:

[0009]

[0010] in, For a set of strategies, For the robot's motion sub-strategy, The semantic cue vector, For the first One movement sub-strategy With semantic cue vectors Match score, To select the first under the semantic cue vector The probability of each movement sub-strategy; The robot is controlled to select the motion sub-policy with the highest probability as the target motion policy:

[0011] in, Targeted movement strategy.

[0012] In conjunction with the first aspect, in one implementation, the motion sub-strategy corresponds to a specific action control sub-task, including: climbing stairs, rotating in place, or standing still. Each of the aforementioned control subtasks corresponds to a specific sequence of control commands, including joint angles and gait patterns.

[0013] In conjunction with the first aspect, in one implementation, when the robot's task failure or abnormal action is detected, the reward function of the deep reinforcement learning controller used to control the robot is reconstructed, including: When the robot detects a task failure or abnormal action, the task failure and abnormal action information described in natural language, along with the corresponding state-action-reward feedback data sequence, are input into the large language model to obtain the output reward function adjustment suggestion description and the structurally adjusted reward function. The reward function of the deep reinforcement learning controller is adjusted to the structure-adjusted reward function output by the large language model.

[0014] In conjunction with the first aspect, in one implementation, when an abnormal robot state occurs, the parameters of the deep reinforcement learning controller are optimized, including: The task success rate and task failure stages described in natural language are input into the large language model to obtain the output of the task failure reasons and the adjustment parameters of the deep reinforcement learning controller. The adjustment parameters of the deep reinforcement learning controller include the exploration rate and the course learning scheme.

[0015] In conjunction with the first aspect, in one implementation, before tuning the parameters of the deep reinforcement learning controller when an abnormal robot state occurs, the method further includes: When the robot's task success rate is less than a preset success rate threshold, the motion oscillation frequency is greater than a preset frequency threshold, or the fluctuation of the deep reinforcement learning controller's reward value is greater than a preset fluctuation threshold, the robot's state is determined to be abnormal.

[0016] Secondly, embodiments of this application provide a robot motion control optimization device, the robot motion control optimization device comprising: The input module is used to input task descriptions in natural language form, robot state information, historical execution trajectories, and environmental feedback information into a pre-built large language model. The tuning module is used to perform the following through the large language model: select the target motion strategy of the robot according to the task description, reconstruct the reward function of the deep reinforcement learning controller used to control the robot when the robot task is detected to be failed or the action is abnormal, and tune the parameters of the deep reinforcement learning controller when the robot state is abnormal.

[0017] Thirdly, embodiments of this application provide a robot motion control tuning device, which includes a processor, a memory, and a robot motion control tuning program stored in the memory and executable by the processor. When the robot motion control tuning program is executed by the processor, it implements the steps of the robot motion control tuning method described in any of the above claims.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a robot motion control tuning program, wherein when the robot motion control tuning program is executed by a processor, it implements the steps of the robot motion control tuning method as described in any of the preceding claims.

[0019] The beneficial effects of the technical solutions provided in this application include: By inputting a task description in natural language, robot state information, historical execution trajectory, and environmental feedback information into a pre-built large language model, the large language model performs the following operations: selecting a target motion strategy for the robot based on the task description; when the robot task failure or abnormal action is detected, reconstructing the reward function of the deep reinforcement learning controller used to control the robot; and when the robot state is abnormal, optimizing the parameters of the deep reinforcement learning controller. This solves the technical problems in related technologies, such as poor generalization ability of robot motion control in complex industrial environments, reliance on expert experience in reward function design and difficulty in dynamic adjustment, and insufficient multi-task adaptability. It significantly improves the robot's motion stability, task completion efficiency, and strategy adaptability in high-risk, high-temperature, and high-complexity scenarios, and enhances the robot's adaptive optimization ability and anomaly recovery ability in dynamic environments. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating an embodiment of the robot motion control optimization method of this application; Figure 2 This is a functional module diagram of an embodiment of the robot motion control optimization device of this application; Figure 3 This is a schematic diagram of the hardware structure of the robot motion control optimization device involved in the embodiments of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0022] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.

[0023] Large Language Model (LLM): This refers to an artificial intelligence model that has been trained on large-scale data and has powerful natural language understanding and generation capabilities. This model can process and understand natural language text, generate meaningful responses, and can be used for a variety of complex tasks.

[0024] A Deep Reinforcement Learning Controller (DRL Controller) is a system based on deep reinforcement learning algorithms. In this application, the DRL Controller is used to control the motion behavior of a robot, enabling the robot to efficiently complete tasks in complex environments by learning optimal policies.

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0026] In a first aspect, embodiments of this application provide a method for optimizing robot motion control. The robot involved in this embodiment may include a quadruped robot for coal-fired power units.

[0027] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the robot motion control optimization method of this application. Figure 1 As shown, the robot motion control optimization method includes: Step S101: Input the task description, robot state information, historical execution trajectory and environmental feedback information in natural language form into the preset large language model.

[0028] Specifically, the task description in this embodiment includes the task objective and expected task completion criteria in natural language form, such as "inspecting cracks on the outer wall of a boiler"; the robot state information includes sensor data such as the robot's posture angles, joint angles, speed, position, and energy consumption; and the historical execution trajectory includes a state-action-reward feedback data sequence. Environmental feedback information includes: task completion rate, anomaly markers, and action stability indicators.

[0029] Anomaly markers are discrete labels for major malfunctions during robot movement, such as falls, task failures, excessive energy consumption, or collisions with obstacles. When these events are detected, anomaly markers are triggered, and subsequently, a large language model generates linguistic explanations and optimization suggestions, thereby helping the reinforcement learning strategy to be quickly corrected.

[0030] Step S102: Execute through the large language model: Select the target motion strategy of the robot according to the task description; when the robot task is detected to be failed or the action is abnormal, reconstruct the reward function of the deep reinforcement learning controller used to control the robot; when the robot state is abnormal, optimize the parameters of the deep reinforcement learning controller.

[0031] It is worth noting that the Deep Reinforcement Learning (DRL) controller is the core controller for robot motion. It is responsible for the underlying state-action-reward loop and can be trained and executed in the traditional RL framework. For example, controlling a robot to walk or climb stairs, the generation, updating and learning of these actions are all the responsibility of the DRL.

[0032] In this embodiment, the Large Language Model (LLM) can be viewed as an "external intelligence layer." It does not directly replace the DRL but acts as an auxiliary tuner, performing semantic parsing of information such as natural language task descriptions, historical trajectories, and anomaly markers to output tuning signals. The LLM's output data includes: control command sequences, such as joint angles and gait patterns; policy switching commands, such as the selected sub-policy ID; reward function parameter adjustments, such as adding penalty terms and updating weight values; and anomaly diagnosis and tuning reports, such as natural language plus parameter values.

[0033] This embodiment employs a multi-strategy parallel deep reinforcement learning controller framework, with the state space being... The action space is The goal is to learn an optimal robot motion strategy. To maximize long-term expected returns:

[0034] in, Let be the motion policy function, representing the state. Take action below The probability of; This is the discount factor, representing the decay coefficient of future rewards; For at any time Instant rewards received; For movement strategies The objective function is the expected cumulative reward. In this embodiment, the exercise strategy that maximizes the long-term expected return is taken as the optimal exercise strategy.

[0035] To adapt to various scenarios, this embodiment introduces a set of motion strategies:

[0036] in, For a set of strategies, This is the robot's motion sub-strategy.

[0037] For example, each motion sub-strategy corresponds to a specific motion control sub-task, including: climbing stairs, rotating in place, or standing still; each of the control sub-tasks corresponds to a specific sequence of control instructions, including joint angles and gait patterns.

[0038] In one embodiment, selecting a target motion strategy for the robot based on the task description using the large language model includes: The large language model generates semantic cue vectors based on the task description. ; Based on a preset mapping function and the semantic cue vector, the robot generates the probability of choosing its motion strategy:

[0039] in, The semantic cue vector, For the first One movement sub-strategy With semantic cue vectors Match score, To select the first under the semantic cue vector The probability of each movement sub-strategy; The robot is controlled to select the motion sub-policy with the highest probability as the target motion policy:

[0040] in, Targeted movement strategy.

[0041] It is worth noting that this embodiment establishes a semantic prompt-based multi-movement strategy switching mechanism for robots by introducing a Large Language Model (LLM), which can significantly improve the robot's generalization ability in multi-task or new environments. Using the LLM as the upper-level semantic perception and task planning engine, semantic prompts can be automatically generated based on the natural language description of the inspection task. These prompts guide the reinforcement learning system to select the optimal strategy from multiple preset sub-strategies, thereby achieving more efficient task execution and environmental adaptation.

[0042] Furthermore, in one embodiment, when the robot task failure or abnormal action is detected, the reward function of the deep reinforcement learning controller used to control the robot is reconstructed, including: when the robot task failure or abnormal action is detected, inputting the task failure and abnormal action information described in natural language, as well as the corresponding state-action-reward feedback data sequence, into the large language model to obtain the output reward function adjustment suggestion description and the structurally adjusted reward function; adjusting the reward function of the deep reinforcement learning controller to the structurally adjusted reward function output by the large language model.

[0043] As an example, the standard reward function Reward for a deep reinforcement learning controller used to control a robot is:

[0044] in, The robot's forward speed; Energy consumption per unit time; This refers to the foot offset. These are pre-set weights.

[0045] When the system detects a task failure or an abnormal action, it records the status-action-feedback data sequence. And input it into a large language model.

[0046] Among them, state This mainly refers to robot sensor information such as attitude angles, joint angles, velocity, position, and energy consumption, as well as motion. It can be divided into two layers: the bottom layer contains the specific control commands for each joint, and the top layer contains the selection of sub-strategies, such as climbing stairs, rotating in place, and standing still. The state-action-reward feedback data sequence binds perception, execution, and feedback together, which can be used for DRL training and also provided to LLM for analysis in case of anomalies.

[0047] For example, when a robot task failure or abnormal movement is detected, inputting "The robot frequently tipped over while performing boiler perimeter inspection, resulting in high energy consumption and failure to achieve the objective" into the Large Language Model (LLM) will yield the LLM output: "It is recommended to enhance the penalty term for tilt angular velocity and adjust the reward function structure as follows:" . in, Representing robots The roll angle at any given moment; The newly introduced pre-defined penalty weights are used. As an "experience feedback" mechanism, LLM translates behavioral language into a reward function, providing modification suggestions and enabling fine-tuning of the RL policy.

[0048] It is worth noting that by utilizing a large language model (LLM) to analyze historical trajectories and environmental feedback, the system accurately identifies potential flaws in the reward function design. These flaws can then be transformed into specific reward function reconstruction suggestions in natural language, thereby achieving dynamic optimization of the reinforcement learning training process. During reinforcement learning training, the system continuously monitors the policy's performance and uses the language model to generate real-time correction suggestions for the reward function, supporting online or iterative updates and dynamically adjusting the reward structure to achieve adaptive optimization of the control policy.

[0049] Furthermore, in one embodiment, when the robot's state is abnormal, the parameters of the deep reinforcement learning controller are optimized, including: inputting the task success rate and task failure stage described in natural language into the large language model to obtain the output task failure reason and the adjustment parameters of the deep reinforcement learning controller; wherein, the adjustment parameters of the deep reinforcement learning controller include the exploration rate and the course learning scheme.

[0050] Action oscillation is assessed by monitoring high-frequency fluctuations in signals such as posture angle, joint angular velocity, and foot contact force; increased reward fluctuations are detected by analyzing the variance and convergence trend of the reward time series. If the robot completes the objective on time without falling or experiencing abnormal energy consumption, the action is considered successful; otherwise, it is considered a failure. A statistical trigger mechanism is also included: if the success rate in the most recent N tasks falls below a preset threshold, such as 50%, the system is considered to be in an abnormal state, requiring analysis and optimization using a large language model.

[0051] Therefore, before optimizing the parameters of the deep reinforcement learning controller when the robot's state is abnormal, the method further includes: determining that the robot's state is abnormal when the robot's task success rate is less than a preset success rate threshold, the motion oscillation frequency is greater than a preset frequency threshold, or the fluctuation of the deep reinforcement learning controller's reward value is greater than a preset fluctuation threshold.

[0052] As an example, after the robot's abnormal state is triggered, the system calls the Large Language Model (LLM) to generate an explanation of the fault phenomenon and propose optimization suggestions: The input to the large language model is: "The success rate dropped to 48% in the last 20 tasks, with the main failures concentrated in the step-up stage." The corresponding output is: "Possible reasons for the anomaly: The policy converges to the easy-to-operate region, lacking training for high-risk policies. It is recommended to increase the exploration rate or adopt a curriculum learning scheme." This suggestion can be translated into tuning parameters for deep reinforcement learning controllers:

[0053] in, Current exploration rate; The increase recommended by LLM; This is the threshold for the maximum exploration rate.

[0054] It's understandable that adjusting the exploration rate directly modifies the parameter ε, increasing the proportion of exploration actions. Course learning, on the other hand, is a phased training method. Its adjustments extend beyond parameters to include the training environment and reward design. For example, the robot might first learn basic gait in a low-difficulty environment, then gradually increase the terrain difficulty or load; or it might initially emphasize stability rewards, then gradually increase efficiency-related rewards. In this embodiment, the large language model provides such phased training suggestions based on failure modes.

[0055] It is worth noting that this embodiment designs a closed-loop mechanism of "anomaly-language feedback-policy fine-tuning," which significantly enhances the robot's interpretability and self-correction capabilities when faced with action failures. In this closed-loop mechanism, a large language model (LLM) is used to support efficient interaction with human engineers, providing task feedback and parameter adjustment suggestions through natural language to assist in efficient policy fine-tuning, thus enabling the system to possess a certain degree of "self-explanation" and policy self-healing capabilities.

[0056] The robot motion control optimization method provided in this application integrates a Large Language Model (LLM) and Deep Reinforcement Learning (DRL) to achieve adaptive optimization of quadruped robot motion control. This method utilizes the LLM to generate prompts based on natural language task descriptions, guiding policy selection and supporting zero-shot adaptation. This allows for rapid policy switching in new environments, improving generalization ability. Regarding sample efficiency, the LLM generates policy optimization suggestions by summarizing historical execution records, resulting in faster convergence and reduced training costs. In the reward function design, the LLM analyzes failure modes and automatically generates reward structure adjustment formulas to avoid local optima. For anomaly recovery, the LLM generates a closed-loop suggestion of "anomaly-explanation-optimization," with automatic controller updates, improving task continuity. This method is applicable to complex scenarios in industrial environments such as boiler rooms in thermal power plants and high-temperature equipment areas, improving the motion stability, action rationality, and task completion rate of quadruped robots in high-risk environments.

[0057] Secondly, embodiments of this application also provide a robot motion control optimization device.

[0058] In one embodiment, reference is made to Figure 2 , Figure 2 This is a functional module diagram of an embodiment of the robot motion control optimization device of this application. Figure 2As shown, the robot motion control optimization device includes: The input module is used to input task descriptions in natural language form, robot state information, historical execution trajectories, and environmental feedback information into a pre-built large language model. The tuning module is used to perform the following through the large language model: select the target motion strategy of the robot according to the task description, reconstruct the reward function of the deep reinforcement learning controller used to control the robot when the robot task is detected to be failed or the action is abnormal, and tune the parameters of the deep reinforcement learning controller when the robot state is abnormal.

[0059] Furthermore, in one embodiment, the optimization module is also used for: The task description includes: task objectives and expected task completion standards; The robot's state information includes: robot posture angles, joint angles, speed, position, and energy consumption; The historical execution trajectory includes: a sequence of state-action-reward feedback data; The environmental feedback information includes: task completion rate, anomaly markers, and action stability indicators.

[0060] Furthermore, in one embodiment, the optimization module is also used for: The large language model generates semantic cue vectors based on the task description. Based on a preset mapping function and the semantic cue vector, the robot generates the probability of choosing its motion strategy:

[0061]

[0062] in, For a set of strategies, For the robot's motion sub-strategy, The semantic cue vector, For the first One movement sub-strategy With semantic cue vectors Match score, To select the first under the semantic cue vector The probability of each movement sub-strategy; The robot is controlled to select the motion sub-policy with the highest probability as the target motion policy:

[0063] in, Targeted movement strategy.

[0064] Furthermore, in one embodiment, the optimization module is also used for: The motion sub-strategy corresponds to a specific action control sub-task, including: climbing stairs, rotating in place, or standing still. Each of the aforementioned control subtasks corresponds to a specific sequence of control commands, including joint angles and gait patterns.

[0065] Furthermore, in one embodiment, the optimization module is also used for: When the robot detects a task failure or abnormal action, the task failure and abnormal action information described in natural language, along with the corresponding state-action-reward feedback data sequence, are input into the large language model to obtain the output reward function adjustment suggestion description and the structurally adjusted reward function. The reward function of the deep reinforcement learning controller is adjusted to the structure-adjusted reward function output by the large language model.

[0066] Furthermore, in one embodiment, the optimization module is also used for: The task success rate and task failure stages described in natural language are input into the large language model to obtain the output of the task failure reasons and the adjustment parameters of the deep reinforcement learning controller. The adjustment parameters of the deep reinforcement learning controller include the exploration rate and the course learning scheme.

[0067] Furthermore, in one embodiment, the optimization module is also used for: When the robot's task success rate is less than a preset success rate threshold, the motion oscillation frequency is greater than a preset frequency threshold, or the fluctuation of the deep reinforcement learning controller's reward value is greater than a preset fluctuation threshold, the robot's state is determined to be abnormal.

[0068] The functions of each module in the above-mentioned robot motion control optimization device correspond to the steps in the above-mentioned robot motion control optimization method embodiment, and their functions and implementation processes will not be described in detail here.

[0069] Thirdly, embodiments of this application provide a robot motion control optimization device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0070] Reference Figure 3 , Figure 3 This is a schematic diagram of the hardware structure of the robot motion control optimization device involved in the embodiments of this application. In the embodiments of this application, the robot motion control optimization device may include a processor, a memory, a communication interface, and a communication bus.

[0071] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0072] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting components within the robot motion control tuning equipment, as well as interfaces used for interconnecting the robot motion control tuning equipment with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0073] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0074] The processor can be a general-purpose processor, which can call the robot motion control tuning program stored in the memory and execute the robot motion control tuning method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the robot motion control tuning program is called can be referred to in the various embodiments of the robot motion control tuning method of this application, and will not be repeated here.

[0075] Those skilled in the art will understand that Figure 3 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0076] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0077] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0078] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0079] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0080] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0082] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for optimizing robot motion control, characterized in that, The robot motion control optimization method includes: Input the task description, robot state information, historical execution trajectory, and environmental feedback information in natural language form into a pre-defined large language model, and execute the task through the large language model. Select the robot's target motion strategy based on the task description; When the robot's task fails or its actions are abnormal, the reward function of the deep reinforcement learning controller used to control the robot is reconstructed. When the robot's state is abnormal, the parameters of the deep reinforcement learning controller are optimized.

2. The robot motion control optimization method as described in claim 1, characterized in that: The task description includes: task objectives and expected task completion standards; The robot's state information includes: robot posture angles, joint angles, speed, position, and energy consumption; The historical execution trajectory includes: a sequence of state-action-reward feedback data; The environmental feedback information includes: task completion rate, anomaly markers, and action stability indicators.

3. The robot motion control optimization method as described in claim 2, characterized in that, The robot's target motion strategy is selected based on the task description using the large language model, including: The large language model generates semantic cue vectors based on the task description. Based on a preset mapping function and the semantic cue vector, the robot generates the probability of choosing its motion strategy: in, For a set of strategies, For the robot's motion sub-strategy, The semantic cue vector, For the first One movement sub-strategy With semantic cue vectors Match score, To select the first under the semantic cue vector The probability of each movement sub-strategy; The robot is controlled to select the motion sub-policy with the highest probability as the target motion policy: in, Targeted movement strategy.

4. The robot motion control optimization method as described in claim 3, characterized in that: The motion sub-strategy corresponds to a specific action control sub-task, including: climbing stairs, rotating in place, or standing still. Each of the aforementioned control subtasks corresponds to a specific sequence of control commands, including joint angles and gait patterns.

5. The robot motion control optimization method as described in claim 2, characterized in that, When a robot task failure or abnormal action is detected, the reward function of the deep reinforcement learning controller used to control the robot is reconstructed, including: When the robot detects a task failure or abnormal action, the task failure and abnormal action information described in natural language, along with the corresponding state-action-reward feedback data sequence, are input into the large language model to obtain the output reward function adjustment suggestion description and the structurally adjusted reward function. The reward function of the deep reinforcement learning controller is adjusted to the structure-adjusted reward function output by the large language model.

6. The robot motion control optimization method as described in claim 2, characterized in that, When the robot's state is abnormal, the parameters of the deep reinforcement learning controller are tuned, including: The task success rate and task failure stages described in natural language are input into the large language model to obtain the output of the task failure reasons and the adjustment parameters of the deep reinforcement learning controller. The adjustment parameters of the deep reinforcement learning controller include the exploration rate and the course learning scheme.

7. The robot motion control optimization method as described in claim 6, characterized in that, Before tuning the parameters of the deep reinforcement learning controller when the robot's state is abnormal, the following steps are also included: When the robot's task success rate is less than a preset success rate threshold, the motion oscillation frequency is greater than a preset frequency threshold, or the fluctuation of the deep reinforcement learning controller's reward value is greater than a preset fluctuation threshold, the robot's state is determined to be abnormal.

8. A robot motion control optimization device, characterized in that, The robot motion control optimization device includes: The input module is used to input task descriptions in natural language form, robot state information, historical execution trajectories, and environmental feedback information into a pre-built large language model. The tuning module is used to perform the following through the large language model: select the target motion strategy of the robot according to the task description, reconstruct the reward function of the deep reinforcement learning controller used to control the robot when the robot task is detected to be failed or the action is abnormal, and tune the parameters of the deep reinforcement learning controller when the robot state is abnormal.

9. A robot motion control optimization device, characterized in that, The robot motion control tuning device includes a processor, a memory, and a robot motion control tuning program stored in the memory and executable by the processor, wherein when the robot motion control tuning program is executed by the processor, it implements the steps of the robot motion control tuning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a robot motion control tuning program, wherein when the robot motion control tuning program is executed by a processor, it implements the steps of the robot motion control tuning method as described in any one of claims 1 to 7.

Images