Motion control method of robot in multi-terrain environment, motion control model training method, device, equipment, robot and medium

Abstract

The invention provides a motion control method of a robot in a multi-terrain environment, a motion control model training method and device, equipment, the robot and a medium, and relates to the technical field of robots. The method comprises the steps that the target moving speed of the robot is acquired; judging a speed interval of the target moving speed, and selecting a target control strategy network corresponding to the speed interval from a plurality of control strategy networks; outputting a control strategy for controlling the robot to move based on the target control strategy network; wherein the plurality of control strategy networks are obtained based on deep reinforcement learning training of the robot in a specified speed interval. The robot can achieve strategy switching matched with the speed requirement in the multi-terrain environment, and therefore the stability and continuity of action output are kept.

Description

Technical Field

[0001] This disclosure relates to the field of robotics technology, and to a method for motion control of a robot in a multi-terrain environment, a method for training a motion control model of a robot in a multi-terrain environment, an apparatus, an electronic device, a robot, and a computer-readable storage medium. Background Technology

[0002] In related technologies, to improve a robot's adaptability to different terrains, multiple policy networks corresponding to each terrain are typically trained using deep reinforcement learning methods. When the robot determines its current terrain environment through algorithms, it switches to the policy network corresponding to that terrain and uses this network to generate an appropriate control policy to achieve motion control of the robot.

[0003] However, in practical applications, the robot's recognition of terrain types may be inaccurate or delayed, causing deviations or delays in the switching timing of the policy network. In such cases, the terrain recognition-based automatic switching policy network struggles to ensure that the switching timing matches the robot's actual motion state, leading to problems such as unstable movements and discontinuous control in complex, multi-terrain scenarios.

[0004] Therefore, when facing different terrain environments, it is currently necessary to manually switch control strategies, with operators judging the appropriate strategy network based on their experience. However, manual switching relies on the operator's subjective experience, making it difficult to achieve real-time and continuous strategy adjustments. This can easily affect the stability and consistency of the robot's motion control in complex, multi-terrain scenarios, while also reducing the operator's experience.

[0005] Therefore, how to improve the reliability of control strategy switching without manual switching, so that the robot can maintain continuous and stable motion control under various terrain conditions and also provide a good user experience, has become a key technical problem that urgently needs to be solved.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] This disclosure provides a motion control method, motion control model training method, device, equipment, robot, and medium for robots in multi-terrain environments, to overcome the problems in related technologies where the reliability of control strategy switching is insufficient due to manual switching of control strategies, which in turn makes robots prone to unstable movements, discontinuous control, and poor operator experience during multi-terrain transitions.

[0008] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure.

[0009] According to a first aspect of this disclosure, a method for motion control of a robot in a multi-terrain environment is provided, comprising: Obtain the target moving speed of the robot; Determine the target's movement speed range and select the target control strategy network corresponding to the speed range from multiple control strategy networks; Based on the target control strategy network, output the control strategy for controlling the robot's motion; Among them, multiple control policy networks are obtained by training the robot based on deep reinforcement learning within a specified speed range; The multiple control strategy networks include at least a first control strategy network and a second control strategy network. The first control strategy network is used to control the robot's movement in a first terrain, and the second control strategy network is used to control the robot's movement in a second terrain. The ruggedness of the first terrain is higher than that of the second terrain. The first speed range corresponding to the first control strategy network and the second speed range corresponding to the second control strategy network are set continuously, and the upper speed limit of the first speed range is less than or equal to the lower speed limit of the second speed range.

[0010] In one exemplary embodiment of this disclosure, the first control strategy network is a complex terrain control strategy network, and the second control strategy network is a flat terrain control strategy network.

[0011] In one exemplary embodiment of this disclosure, the first speed range is 0.1~1m / s, and the second speed range is greater than or equal to 1m / s.

[0012] In one exemplary embodiment of this disclosure, the plurality of control strategy networks further includes a third control strategy network, wherein the third speed range corresponding to the third control strategy network is continuously set with the first speed range corresponding to the first control strategy network, and the upper speed limit of the third speed range is less than or equal to the lower speed limit of the first speed range.

[0013] In one exemplary embodiment of this disclosure, the third control strategy network is a zero-speed control strategy network, and the third speed range is 0~0.1m / s.

[0014] In one exemplary embodiment of this disclosure, the method further includes: In response to a speed adjustment command, obtain the target acceleration corresponding to the speed adjustment command; Based on the target acceleration, the robot will be adjusted from its current speed to the target moving speed after passing through several speed states; Among them, the robot's motion is controlled based on multiple control strategies under several speed states.

[0015] In one exemplary embodiment of this disclosure, adjusting the robot from its current speed to a target moving speed after passing through several speed states based on the target acceleration includes: If the target acceleration is positive, the current speed will be adjusted to the target's moving speed after passing through several speed states corresponding to the positive acceleration. If the target acceleration is negative, the current speed will be adjusted to the target's moving speed after passing through several speed states corresponding to the negative acceleration.

[0016] In an exemplary embodiment of this disclosure, the plurality of velocity states corresponding to positive acceleration are plurality of velocity states in a first velocity state group, the first velocity state group including: a first stationary state, a first low-speed state, a preparatory high-speed state, and a first high-speed state; The velocity states corresponding to negative acceleration are several velocity states in the second velocity state group, which includes: second high speed state, preparatory low speed state, second low speed state, and second stationary state.

[0017] In one exemplary embodiment of this disclosure, the method further includes: The absolute value of the target acceleration is compared with a preset acceleration threshold. When the absolute value is greater than the preset acceleration threshold, the robot is controlled to skip the pre-high speed state or the pre-low speed state in order to accelerate the speed adjustment. When the absolute value is less than or equal to the preset acceleration threshold, the robot is controlled to sequentially pass through several speed states in the first speed state group or several speed states in the second speed state group to complete the speed adjustment.

[0018] In one exemplary embodiment of this disclosure, obtaining the target acceleration corresponding to the speed adjustment command includes: Obtain the robot's current speed when it receives the speed adjustment command; The target acceleration is calculated based on the speed difference between the target's moving speed and its current speed.

[0019] In one exemplary embodiment of this disclosure, after obtaining the target acceleration corresponding to the speed adjustment command, the method further includes: Limit the target acceleration; Based on the target acceleration of the amplitude limiting process, the robot's speed adjustment step size in each speed state is updated, and the speed adjustment step size is adjusted from the current speed through several speed states to the target moving speed.

[0020] In one exemplary embodiment of this disclosure, after outputting a control strategy for controlling the robot's motion, the method includes: The control strategy is adjusted based on the speed adjustment step size corresponding to the current speed state.

[0021] In one exemplary embodiment of this disclosure, the method further includes: At each speed state, the robot's target moving speed is updated based on the target acceleration to obtain the intermediate moving speed; Determine the speed range of the intermediate movement speed, and update the target control policy network if the speed range of the intermediate movement speed changes relative to the speed range corresponding to the previous speed state.

[0022] According to a second aspect of this disclosure, a method for training a motion control model for a robot in a multi-terrain environment is provided, comprising: In a deep learning training environment, the robot's speed state during acceleration is divided into a first stationary state, a first low-speed state, a preparatory high-speed state, and a first high-speed state. The robot's speed state during deceleration is divided into a second high-speed state, a preparatory low-speed state, a second low-speed state, and a second stationary state. Collect the robot's current state, action output, and environmental feedback information at each speed state to form training data corresponding to each speed state; Based on the training data corresponding to each speed state, and combined with the speed range to which each speed state belongs, the parameters of the control policy network corresponding to the speed range are updated.

[0023] In one exemplary embodiment of this disclosure, the speed range includes: a first speed range, a second speed range, and a third speed range; Based on the training data corresponding to each speed state, and combined with the speed range to which each speed state belongs, the parameters of the control policy network corresponding to the speed range are updated, including: When the robot is in the first low-speed state, the second low-speed state, or the preparatory high-speed state, the parameters of the first control strategy network corresponding to the first speed interval are updated based on the training data corresponding to each speed state and combined with the first speed interval to which each speed state belongs. When the robot is in the first high-speed state, the second high-speed state, or the preparatory low-speed state, based on the training data corresponding to each speed state and combined with the second speed interval to which each speed state belongs, the parameters of the second control strategy network corresponding to the second speed interval are updated. When the robot is in the first stationary state or the second stationary state, based on the training data corresponding to each speed state and combined with the third speed interval to which each speed state belongs, the parameters of the third control strategy network corresponding to the third speed interval are updated.

[0024] In one exemplary embodiment of this disclosure, the parameters of the first control policy network corresponding to the first speed interval are updated based on the training data corresponding to each speed state and in combination with the first speed interval to which each speed state belongs, including: Based on training data, the low-speed motion characteristics of the robot within the first speed range are determined. Based on the characteristics of low-speed motion, a reward function for the low-speed range is constructed. The reward value corresponding to the training data is calculated based on the reward function in the low-speed range, and the parameters of the first control policy network are updated based on the reward value.

[0025] In one exemplary embodiment of this disclosure, the low-speed motion characteristics include terrain undulation response, foot lift height deviation, foot contact change, and low-speed posture change. Based on low-speed motion characteristics, a reward function for the low-speed region is constructed, including: Based on the response to terrain undulations, construct terrain adaptation reward items; Based on the deviation in foot lift height, a gait lift reward item is constructed. Construct a contact stability reward item based on the change in foot contact. Construct a walking posture reward item based on low-speed posture changes; A reward function for low-speed intervals is constructed based on terrain adaptation reward, gait lift reward, contact stability reward, and walking posture reward.

[0026] In one exemplary embodiment of this disclosure, the parameters of the second control policy network corresponding to the second speed interval are updated based on the training data corresponding to each speed state and in combination with the second speed interval to which each speed state belongs, including: Based on training data, the high-speed motion characteristics of the robot in the second velocity range were determined. Based on the characteristics of high-speed motion, a high-speed interval reward function is constructed; The reward value corresponding to the training data is calculated based on the high-speed interval reward function, and the parameters of the second control policy network are updated based on the reward value.

[0027] In one exemplary embodiment of this disclosure, the high-speed motion characteristics include dynamic attitude change, velocity tracking deviation, and foot contact deviation. Based on high-speed motion characteristics, a high-speed interval reward function is constructed, including: Construct a dynamic posture reward item based on the dynamic posture change; Based on the speed tracking deviation, construct a speed tracking reward item; Construct a contact accuracy reward system based on the foot contact deviation. A high-speed interval reward function is constructed based on dynamic attitude reward, velocity tracking reward, and contact accuracy reward.

[0028] In one exemplary embodiment of this disclosure, the parameters of the third control policy network corresponding to the third speed interval are updated based on the training data corresponding to each speed state and in combination with the third speed interval to which each speed state belongs, including: Based on the training data, the static state characteristics of the robot in the third velocity range are determined. Based on the characteristics of the static state, a zero-speed interval reward function is constructed; The reward value corresponding to the training data is calculated based on the zero-speed interval reward function, and the parameters of the third control policy network are updated based on the reward value.

[0029] In one exemplary embodiment of this disclosure, the static state characteristics include static attitude change, static position change, and static velocity change; Based on the characteristics of the static state, a zero-velocity interval reward function is constructed, including: Construct a static attitude reward item based on the static attitude change. Construct a static position reward item based on the change in static position; Construct a static velocity reward item based on the change in static velocity; A zero-speed interval reward function is constructed based on the static attitude reward term, the static position reward term, and the static velocity reward term.

[0030] According to a third aspect of this disclosure, a motion control device for a robot in multi-terrain environments is provided, comprising: The speed acquisition module is used to acquire the target movement speed of the robot; The network selection module is used to determine the speed range of the target's movement speed and select the target control strategy network corresponding to the speed range from multiple control strategy networks. The strategy output module is used to output the control strategy for controlling the robot's motion based on the target control strategy network. Among them, multiple control policy networks are obtained by training the robot based on deep reinforcement learning within a specified speed range; The multiple control strategy networks include at least a first control strategy network and a second control strategy network. The first control strategy network is used to control the robot's movement in a first terrain, and the second control strategy network is used to control the robot's movement in a second terrain. The ruggedness of the first terrain is higher than that of the second terrain. The first speed range corresponding to the first control strategy network and the second speed range corresponding to the second control strategy network are set continuously, and the upper speed limit of the first speed range is less than or equal to the lower speed limit of the second speed range.

[0031] According to a fourth aspect of this disclosure, a motion control model training device for a robot in a multi-terrain environment is provided, comprising: The velocity state division module is used in a deep learning training environment to divide the robot's velocity state during acceleration into a first stationary state, a first low-speed state, a preparatory high-speed state, and a first high-speed state, and to divide the robot's velocity state during deceleration into a second high-speed state, a preparatory low-speed state, a second low-speed state, and a second stationary state. The training data generation module is used to collect the robot's current state, action output, and environmental feedback information at each speed state to form training data corresponding to each speed state. The policy network training module is used to update the parameters of the control policy network corresponding to each speed state based on the training data corresponding to each speed state and the speed range to which each speed state belongs.

[0032] According to a fifth aspect of this disclosure, an electronic device is provided, comprising: Processor; and A memory that stores computer-readable instructions, which, when executed by a processor, implement the method as described in the above embodiments.

[0033] According to a sixth aspect of this disclosure, a robot is provided, comprising: Processor; and A memory that stores computer-readable instructions, which, when executed by a processor, implement the method as described in the above embodiments.

[0034] In one exemplary embodiment of this disclosure, the robot includes any one of a legged robot, a wheeled robot, and a humanoid robot.

[0035] According to a seventh aspect of this disclosure, a computer-readable storage medium is provided that stores computer program code instructions that, when invoked by a processor, cause the processor to perform the method as described in the above embodiments.

[0036] As can be seen from the above technical solution, this disclosure possesses at least one of the following advantages and positive effects: This disclosure uses the robot's target moving speed as the basis for control strategy selection, enabling the robot to automatically match the corresponding control strategy network under different speed requirements, thereby achieving adaptive selection of the control strategy network under different operating conditions.

[0037] In multi-terrain motion control, the robot's motion state typically exhibits continuous changes, and the switching of the control strategy network requires precise timing to avoid adverse effects on motion stability. Therefore, this disclosure establishes a correspondence between the control strategy network and a preset speed range, enabling the control strategy network to switch according to changes in the target's movement speed. This reduces the dependence of the strategy switching process on the accuracy of terrain recognition and the timing of the switching.

[0038] Since speed changes can reflect the different motion requirements of robots under different terrain conditions, when the control strategy is based on the speed range, the control strategy can be adjusted according to the motion state without the need for manual switching of the control strategy. This reduces the impact of manual intervention on the continuity of motion and enables the robot to maintain the stability and consistency of motion output during continuous transitions between multiple terrains.

[0039] In addition, different tasks usually correspond to different target speed requirements. By distinguishing the control modes required for different tasks through speed ranges, multi-task control can be achieved without increasing operational complexity, thereby improving the stability of the overall motion control process and the user experience. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A system architecture diagram is shown that can be applied to the motion control method of the robot in a multi-terrain environment according to the embodiments of this disclosure.

[0042] Figure 2 A flowchart illustrating a motion control method for a robot in a multi-terrain environment according to an embodiment of this disclosure is shown.

[0043] Figure 3 A schematic diagram of a process for obtaining the target moving speed of a robot is shown in an embodiment of this disclosure.

[0044] Figure 4 A schematic diagram of a process for adjusting to a target moving speed is shown in an embodiment of this disclosure.

[0045] Figure 5 The diagram illustrates the speed state groups and control strategy switching of a robot in different speed ranges according to an embodiment of this disclosure.

[0046] Figure 6A flowchart illustrating a method for training a robot's motion control model in a multi-terrain environment, as described in an embodiment of this disclosure, is shown.

[0047] Figure 7 A schematic flowchart illustrating an embodiment of this disclosure is shown for updating parameters of a control strategy network corresponding to a speed range.

[0048] Figure 8 A schematic flowchart illustrating an embodiment of this disclosure of updating parameters of a first control strategy network corresponding to a first speed range is shown.

[0049] Figure 9 A schematic flowchart illustrating an embodiment of this disclosure is shown for updating parameters of a second control strategy network corresponding to a second speed range.

[0050] Figure 10 A schematic flowchart illustrating an embodiment of this disclosure is shown for updating parameters of a third control strategy network corresponding to a third speed range.

[0051] Figure 11 A schematic diagram of a robot simulation environment according to an embodiment of this disclosure is shown.

[0052] Figure 12 A schematic diagram illustrating the state changes of a robot during a control strategy network switching process is shown in an embodiment of this disclosure.

[0053] Figure 13 A block diagram of a motion control device for a robot in a multi-terrain environment is shown in an embodiment of the present disclosure.

[0054] Figure 14 A block diagram of a motion control model training device for a robot in a multi-terrain environment is shown in an embodiment of the present disclosure.

[0055] Figure 15 A schematic diagram of a type of robot is shown, to which the motion control method of the robot in the embodiments of this disclosure can be applied in a multi-terrain environment.

[0056] Figure 16 A schematic diagram of another type of robot is shown, which can be used to apply the motion control method of the robot in multi-terrain environments according to the embodiments of this disclosure.

[0057] Figure 17 A schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure is shown.

[0058] Figure 18 A schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0059] In this disclosure, the terms "first" and "second" are used for description only and do not indicate relative importance or imply the number of technical features. Therefore, the features referred to as "first" or "second" may explicitly or implicitly include at least one of those features. "A plurality of" means at least two, unless otherwise expressly defined.

[0060] First, the relevant terms used in the exemplary embodiments of this disclosure will be explained: Deep Reinforcement Learning (DRL) is a machine learning method that combines reinforcement learning decision-making mechanisms with deep neural networks. It involves enabling an agent to interact with the environment during training, iteratively updating the neural network parameters based on reward signals from the environment, thereby learning the mapping relationship from state to action. In this embodiment, deep reinforcement learning is used to train the robot's motion control strategies under different speed ranges and terrain conditions.

[0061] Target moving speed: refers to the control parameter used to characterize the magnitude of the robot's expected movement speed. It guides the robot's current motion control process and serves as the basis for selecting the control strategy network and adjusting the speed state.

[0062] Speed ​​range: refers to multiple speed segments divided according to the robot's motion speed range, used to classify the robot's motion state and establish corresponding relationships with different control strategy networks. In this embodiment, the speed range may include a first speed range, a second speed range, and a third speed range. The first speed range is located between the third speed range and the second speed range, and the first speed range is continuously set with the third speed range and the second speed range, respectively. The upper speed limit of the first speed range is less than or equal to the lower speed limit of the second speed range, and the upper speed limit of the third speed range is less than or equal to the lower speed limit of the first speed range. For example, the first speed range is 0.1~1m / s, the second speed range is greater than or equal to 1m / s, and the third speed range is 0~0.1m / s.

[0063] A control strategy network refers to a neural network model trained through deep reinforcement learning, used to output actions or control commands to control the robot's movement based on the robot's current state information. In this embodiment, different control strategy networks correspond to different speed ranges and terrain adaptation requirements, and are used to output control strategies that meet the current motion conditions during the operation phase.

[0064] The first control strategy network refers to the control strategy network used to control the robot's movement in the first terrain. In this embodiment, the first terrain can be complex terrain, which refers to a terrain environment with high ruggedness, such as terrain with large undulations, high unevenness, and frequent changes in the contact state of the robot's feet during movement. Accordingly, the first control strategy network is a complex terrain control strategy network. The complex terrain control strategy network is a control strategy network trained through deep reinforcement learning under low-speed movement conditions for complex terrain environments. It is used to output control strategies with terrain adaptability and motion stability in complex terrain environments.

[0065] The second control strategy network refers to the control strategy network used to control the robot's movement in the second terrain. In this embodiment, the second terrain refers to a terrain environment with lower ruggedness compared to the first terrain, such as flat terrain, meaning a terrain environment with less undulation, higher flatness, and relatively stable contact state of the robot's feet during movement. Accordingly, the second control strategy network is a flat terrain control strategy network. Under high-speed movement conditions, the flat terrain control strategy network is a control strategy network trained through deep reinforcement learning for a flat terrain environment, used to achieve stable posture control, speed tracking, and foot contact control at higher movement speeds.

[0066] The third control strategy network refers to the control strategy network used to control the robot's movement in the third terrain. In this embodiment, the third terrain corresponds to a motion state where the robot's target movement speed is close to zero. Accordingly, the third control strategy network is a zero-speed control strategy network. The zero-speed control strategy network is a control strategy network trained through deep reinforcement learning for the robot's target movement speed being close to zero. It is used to control the robot to maintain a stable attitude, position, and velocity state under static or quasi-static conditions.

[0067] Control strategy: refers to the control results output by the control strategy network for controlling the robot's motion, including but not limited to joint control commands, force or torque control commands, gait parameters or posture adjustment commands, which are used to drive the robot to complete motion control according to the target moving speed and the current speed state.

[0068] Several velocity states refer to the multiple intermediate motion states experienced by the robot during velocity adjustment, based on the direction and magnitude of the target acceleration, as it adjusts from its current speed to the target moving speed. Velocity states are used to describe the robot's motion stages within different speed ranges and to trigger the selection or switching of the control strategy network.

[0069] The first velocity state group refers to the set of velocity states of the robot under positive acceleration conditions, which is used to describe the velocity change stages during the acceleration process.

[0070] The first stationary state refers to the robot's velocity state corresponding to the third velocity range during acceleration. It characterizes the robot's motion state at the beginning of acceleration. In the first stationary state, the robot's target moving speed is low, and it has not yet entered the continuous walking or high-speed movement stage. This velocity state is used as the initial state for the subsequent speed increase process.

[0071] The first low-speed state refers to the robot's speed state corresponding to the first speed range during acceleration. It characterizes the robot's motion phase during the transition from a stationary state to a higher speed state. In the first low-speed state, the robot's speed is higher than the first stationary state but lower than the preparatory high-speed state. This speed state describes the stage where the robot is walking or moving at low speed during acceleration.

[0072] Preparatory high-speed state: This refers to the transitional speed state set between the first low-speed state and the first high-speed state, corresponding to the first speed range. It is used to characterize the intermediate motion state of the robot during the acceleration process, as it transitions from the low-speed stage to the high-speed stage. In the preparatory high-speed state, the robot's speed is higher than that of the first low-speed state but has not yet reached the first high-speed state. This speed state serves as a transitional phase before entering the first high-speed state, ensuring the continuity of motion during the speed increase.

[0073] The first high-speed state refers to the robot's speed state corresponding to the second speed range during acceleration. It characterizes the robot's state after completing the acceleration process and reaching a higher motion speed. In the first high-speed state, the robot's motion speed is within the high-speed range, and this speed state is used to describe the robot's motion state when entering a stable high-speed motion phase.

[0074] The second velocity state group refers to the set of velocity states of the robot under negative acceleration conditions, which is used to describe the velocity change stages during deceleration.

[0075] The second high-speed state refers to the robot's speed state corresponding to the second speed range during deceleration, and is used to characterize the robot's motion state at the beginning of the deceleration phase. In the second high-speed state, the robot's motion speed is at a relatively high level, and this speed state is used as the starting state for the adjustment process from high speed to low speed.

[0076] Preparatory low-speed state: This refers to the transitional speed state set between the second high-speed state and the second low-speed state, corresponding to the second speed range. It is used to characterize the intermediate motion state of the robot during deceleration, as it transitions from the high-speed phase to the low-speed phase. In the preparatory low-speed state, the robot's speed is lower than that of the second high-speed state and has not yet entered the second low-speed state. This speed state serves as a transitional phase before entering the low-speed state, allowing the robot to maintain motion continuity during speed reduction.

[0077] The second low-speed state refers to the robot's speed state corresponding to the first speed range during deceleration. It characterizes the robot's low-speed motion phase after partial deceleration. In the second low-speed state, the robot's speed is lower than the preparatory low-speed state but higher than the second stationary state. This speed state describes the robot's low-speed walking or low-speed movement phase during deceleration.

[0078] The second stationary state refers to the velocity state of the robot during deceleration, corresponding to the third velocity range. It characterizes the stationary or quasi-stationary motion state that the robot enters after completing the deceleration process. The second stationary state serves as the termination state of the deceleration process.

[0079] Speed ​​adjustment step size: refers to the amount of speed change determined by the target acceleration when the robot is at a certain speed state, and is used to control the speed update range of the robot in each speed state.

[0080] Reward function / reward term: refers to the function used to calculate the reward value based on the robot's motion state, action output, and environmental feedback during deep reinforcement learning training; reward term refers to the various sub-terms that make up the reward function, used to constrain different motion characteristics or control objectives respectively.

[0081] Figure 1 A system architecture diagram is shown that can be applied to the motion control method for robots in multi-terrain environments according to embodiments of this disclosure. For example... Figure 1 As shown, the system architecture 100 may include a terminal device 101, a robot 102, a network 103, and a server 104.

[0082] The terminal device 101 includes, but is not limited to, desktop computers, laptops, smartphones, and tablets. The terminal device 101 is equipped with a graphical user interface for configuring and managing the training and deployment process of the control policy network, and for displaying training and deployment-related information.

[0083] Robot 102 is equipped with inertial measurement units, joint angle encoders, and / or speed sensors to collect motion state data and determine the robot's movement speed. The speed sensors can directly acquire the movement speed. Robot 102 includes at least a main controller and joint actuators. The main controller deploys and runs multiple control strategy networks, acquires the robot's target movement speed during operation, determines the speed range to which the target speed belongs, selects the target control strategy network corresponding to the speed range from the multiple control strategy networks, and outputs a control strategy for controlling the robot's movement based on the target control strategy network. The joint actuators receive the control strategy and drive the corresponding joints to perform movements, thereby achieving robot motion control.

[0084] Server 104 is used to train multiple control policy networks and then distribute the trained control policy networks to robot 102. These multiple control policy networks are obtained by training the robot using deep reinforcement learning within a specified speed range.

[0085] Network 103 serves as a medium to provide a communication link between terminal device 101, robot 102, and server 104, enabling the distribution of control strategies and the exchange of control information during the training process. Network 103 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0086] It should be understood that Figure 1 The number and type of terminal devices, robots, networks, and servers shown are merely illustrative. Depending on implementation needs, there can be any number and type of terminal devices, robots, networks, and servers; for example, the servers could be cloud servers.

[0087] Through the coordinated operation of the components in system architecture 100, the robot can select and execute control strategies based on the target movement speed during operation, thereby supporting stable motion control of the robot in multi-terrain environments.

[0088] This disclosure provides an example implementation of a method for motion control of a robot in multi-terrain environments. (See references) Figure 2 As shown, the method may include the following steps S201 to S204: Step S201: Obtain the target moving speed of the robot; Step S202: Determine the speed range of the target's movement speed, and select the target control strategy network corresponding to the speed range from multiple control strategy networks; Step S203: Based on the target control policy network, output the control policy for controlling the robot's motion; wherein, multiple control policy networks are obtained by training the robot based on deep reinforcement learning within a specified speed range.

[0089] This disclosure uses the robot's target moving speed as the basis for selecting the control strategy. By establishing a correspondence between different control strategy networks and preset speed ranges, the control strategy networks can automatically switch according to changes in the target moving speed. This achieves adaptive adjustment of the control strategy to the robot's continuous motion state without the need for manual switching. This method reduces the dependence of control strategy switching on terrain recognition accuracy and switching timing judgment, enabling the robot to maintain the stability and consistency of motion output during continuous transitions across multiple terrains and under different task conditions. Furthermore, it improves the overall stability of motion control and the user experience without increasing operational complexity.

[0090] The following will provide a detailed description of the motion control method for the robot in a multi-terrain environment in this example embodiment.

[0091] In step S201, the target moving speed of the robot is obtained.

[0092] In the exemplary embodiments of this disclosure, the target moving speed of the robot during the current motion control process can be obtained. The target moving speed is used to characterize the desired motion speed level of the robot and serves as a fundamental parameter for subsequent speed range determination and control strategy selection.

[0093] The target movement speed can be obtained based on external control input or motion commands generated by the upper-level control module. The motion commands contain speed parameters used to indicate the robot's movement speed. Of course, the target movement speed can also be determined based on a preset motion mode, task planning results, or the current motion state. This disclosure does not limit this, as long as it can provide a clear target speed reference for the robot in the motion control process.

[0094] In some example implementations, references Figure 3 As shown, step S201 may further include the following steps S301 and S302: Step S301: In response to the speed adjustment command, obtain the target acceleration corresponding to the speed adjustment command.

[0095] In this example, the speed adjustment command is a control command used to trigger the robot to adjust its current speed, and it represents the speed adjustment requirement that the robot needs to perform during the current motion, indicating the direction or intention of the speed adjustment.

[0096] When the robot receives a speed adjustment command, the main controller identifies the current speed adjustment requirement based on the command and then obtains or updates the robot's target movement speed accordingly. It should be noted that the target movement speed can be directly included in the speed adjustment command. Alternatively, the speed adjustment command can simply indicate the speed adjustment requirement and carry information about the acceleration / deceleration gear or multiplier. The main controller updates the original target movement speed based on this gear or multiplier information to obtain the new target movement speed.

[0097] Subsequently, the main controller combines the robot's current speed when it receives the speed adjustment command, processes the speed difference between the target moving speed and the current speed, and calculates the target acceleration corresponding to the speed adjustment command.

[0098] The target acceleration characterizes the direction and magnitude of the robot's velocity change during the current velocity adjustment process, and serves as the fundamental parameter for subsequent velocity state adjustments, velocity state group selection, and velocity adjustment step size updates. The current velocity can be the linear velocity of the robot body in the ground coordinate system, or it can be the projected velocity along the target motion direction. For example, the current velocity can be obtained by fusing information from inertial measurement units, foot contact information, joint encoder feedback, and odometry data.

[0099] In practical applications, to improve the stability of target acceleration calculation, the current velocity can also be a filtered velocity value within a preset time window, such as the velocity estimate obtained by moving average filtering, exponential smoothing, or low-pass filtering, and used as the current velocity. Alternatively, after calculating the target acceleration, the target acceleration can be smoothed to ensure the stability of velocity state switching.

[0100] By using the above method, the target acceleration is determined by both the speed adjustment requirement and the robot's current motion state, thereby ensuring that the robot has good continuity and controllability during the speed adjustment process.

[0101] For example, the speed difference between the target's moving speed and its current speed can be expressed as: in, Indicates the target's movement speed. Indicates the current speed. Indicates the speed difference. When When, it indicates that the robot needs to accelerate; when When, it indicates that the robot needs to slow down; when When the speed is adjusted, it indicates that no speed adjustment or holding mode is required.

[0102] It is understandable that the speed difference between the target's moving speed and the current speed can be a scalar difference or a speed vector difference. For example, the component of the speed vector in the desired direction of motion can be used for acceleration calculation to avoid the influence of lateral speed disturbances on the determination of the target's acceleration.

[0103] Furthermore, the target acceleration is calculated based on the speed difference between the target's moving speed and the current speed. For example, based on a preset speed adjustment period, this speed difference is mapped to the target acceleration as follows: in, Indicates the target acceleration. This indicates the speed adjustment period, which is the time window within which the target moving speed is expected to be reached.

[0104] Speed ​​adjustment cycle The period can be updated discretely based on a velocity state machine. And combined with the number of speed states expected to pass through Determined, that is Then adjust the cycle according to the speed. Calculate the target acceleration so that the subsequent process of adjusting to the target moving speed through several speed states satisfies the preset smoothness and responsiveness.

[0105] The main controller outputs the calculated target acceleration as a speed adjustment parameter and transmits the target acceleration to the speed state adjustment process to determine the speed state group corresponding to acceleration or deceleration, and to update the intermediate movement speed and speed adjustment step size in each speed state, thereby realizing a gradual transition from the current speed to the target movement speed.

[0106] Step S302: Based on the target acceleration, adjust the robot from the current speed to the target moving speed after passing through several speed states; wherein, the robot's movement is controlled based on multiple control strategies in the several speed states.

[0107] Once the target acceleration is determined, the robot can be guided to adjust its movement speed from the current speed to the target speed based on the target acceleration.

[0108] Specifically, the main controller determines the robot's current speed adjustment direction and trend based on the target acceleration, and controls the robot to sequentially experience several speed states during the speed adjustment process to achieve gradual speed change. In each speed state, the main controller invokes multiple control strategies to control the robot's motion according to the motion characteristics corresponding to that speed state, ensuring motion stability and control continuity at different speed stages.

[0109] By adopting a multi-velocity state adjustment method based on target acceleration, the robot maintains the continuity of velocity changes during velocity adjustment and provides a basis for executing corresponding control strategies under different velocity states.

[0110] For example, refer to Figure 4 As shown, step S302 may further include steps S401 and S402: Step S401: If the target acceleration is positive, adjust the current speed to the target moving speed after passing through several speed states corresponding to the positive acceleration.

[0111] Step S402: If the target acceleration is negative, the current speed is adjusted to the target moving speed after passing through several speed states corresponding to the negative acceleration.

[0112] In this example implementation, it can be combined with Figure 5 Steps S401 and S402 will be explained. (See reference) Figure 5 The diagram illustrates the speed state groups and control strategy switching of a robot in different speed ranges. Figure 5 In this process, the robot is divided into multiple speed states according to the current speed range during its movement, and corresponding control strategies are used to control the robot's movement in different speed states.

[0113] When the target acceleration is determined to be positive in step S301, it indicates that the robot needs to adjust from the current speed to a higher speed. At this time, the main controller selects the first speed state group corresponding to the positive acceleration according to the direction of the target acceleration, and controls the robot to gradually adjust the current speed to the target moving speed according to several speed states in the first speed state group.

[0114] The first speed state group includes a first stationary state, a first low-speed state, a preparatory high-speed state, and a first high-speed state. The motion speed corresponding to each speed state increases sequentially, and the motion speed corresponding to the preparatory high-speed state is between the first low-speed state and the first high-speed state. The main controller determines the speed states to be passed sequentially from the first speed state group based on the current speed range, and controls the robot's motion in each speed state. This allows the robot to adopt a control strategy that matches its motion characteristics at different speed stages, achieving a gradual increase in speed from low to high.

[0115] By introducing a first velocity state group, the robot can smoothly transition between different velocity stages during acceleration, avoiding the impact of sudden velocity changes on motion stability and the continuity of control strategy.

[0116] When the target acceleration is determined to be negative in step S301, it indicates that the robot needs to adjust from the current speed to a lower speed. At this time, the main controller selects the second speed state group corresponding to the negative acceleration according to the direction of the target acceleration, and controls the robot to gradually adjust the current speed to the target moving speed according to several speed states in the second speed state group.

[0117] The second speed state group describes the speed change stages of the robot during deceleration, including a second high-speed state, a preparatory low-speed state, a second low-speed state, and a second stationary state. The motion speed corresponding to each speed state decreases sequentially, and the motion speed corresponding to the preparatory low-speed state is between the second high-speed state and the second low-speed state. Based on the current speed range, the main controller determines the speed states that need to be passed sequentially from the second speed state group and controls the robot's motion in each speed state, enabling the robot to gradually adjust its speed from high to low at different speed stages.

[0118] By introducing a second speed state group, the robot can smoothly transition between high-speed, medium-speed, and low-speed phases during deceleration, thereby reducing the impact of rapid deceleration on the robot's posture stability and foot contact state.

[0119] Still referencing Figure 5 As shown, different speed ranges correspond to different control strategies. In the third speed range of 0~0.1 m / s, i.e., when the robot is in the first stationary state or the second stationary state, a zero-speed control strategy is used to control the motion. In the first speed range of 0.1~1 m / s, i.e., when the robot is in any of the first low-speed state, the preparatory high-speed state, or the second low-speed state, a complex terrain control strategy is used to control the motion to adapt to the response requirements of terrain changes during low-speed walking. In the second speed range of 1.0~3.0 m / s, i.e., when the robot is in any of the first high-speed state, the preparatory low-speed state, or the second high-speed state, a flat terrain control strategy is used to control the motion to meet the requirements of posture stability and speed tracking performance in high-speed motion.

[0120] By setting the speed state groups, speed ranges, and control strategies in the above manner, the robot can switch between speed states and execute corresponding control strategies at different speed stages and in different directions of motion, thereby achieving continuous and stable motion control of the robot in multi-terrain environments.

[0121] After completing the velocity state adjustment based on the target acceleration, in order to further improve the flexibility and response efficiency of the robot's velocity adjustment process, the velocity state adjustment path can also be adaptively controlled based on the magnitude of the target acceleration.

[0122] Specifically, the absolute value of the target acceleration is compared with a preset acceleration threshold to determine the acceleration intensity of the current speed adjustment process. The preset acceleration threshold is used to distinguish speed adjustment needs of different intensities to determine whether the current speed adjustment process requires an accelerated speed state adjustment path. The preset acceleration threshold can be set according to the robot's structural characteristics, driving capabilities, and motion stability requirements, and is used to characterize the range of acceleration changes the robot can withstand while maintaining motion stability. This disclosure does not limit the specific value of the acceleration threshold.

[0123] If the absolute value of the target acceleration is greater than the preset acceleration threshold, it indicates that the current speed adjustment requirement is relatively urgent. The robot can be controlled to skip steps during the speed adjustment process. Figure 5 The indicated high-speed or low-speed state accelerates the switching between speed states, improving speed adjustment response efficiency. When the absolute value of the target acceleration is less than or equal to a preset acceleration threshold, it indicates that the current speed adjustment demand is relatively mild. The robot is then controlled to change speed according to a complete speed state adjustment path, allowing it to sequentially pass through several speed states in the first speed state group or several speed states in the second speed state group to complete the speed adjustment, ensuring the smoothness of the speed adjustment process.

[0124] In this way, the robot can make a trade-off between speed adjustment efficiency and speed change smoothness based on the magnitude of the target acceleration, thereby improving its speed adjustment adaptability under different motion requirements.

[0125] In some example implementations, after obtaining the target acceleration corresponding to the speed adjustment command, in order to avoid excessive acceleration changes during the speed adjustment process from affecting the robot's motion stability, a constraint process on the target acceleration can be introduced in the speed adjustment process.

[0126] Specifically, the main controller limits the target acceleration to a preset range, thus preventing adverse effects on robot motion stability due to excessively large or small target acceleration. The limited target acceleration serves as an effective acceleration parameter for subsequent speed adjustment. The preset acceleration range characterizes the allowable range of acceleration variation for the robot under current structural conditions, driving capabilities, and motion stability requirements. This acceleration range can be preset during system initialization or adjusted according to the robot's operating mode, load state, or current motion condition. This disclosure does not limit the specific value of the preset acceleration range.

[0127] For example, amplitude limiting can employ a saturation limiting method, where the target acceleration is limited to the upper limit of the preset acceleration range when it exceeds the upper limit, and to the lower limit when it is less than the lower limit. Amplitude limiting can also use gradual constraints or segmented limitations to restrict the magnitude of changes in target acceleration, thus preventing excessive changes in target acceleration within adjacent control cycles.

[0128] After the amplitude limiting process is completed, the robot's speed adjustment step size in each speed state can be updated based on the target acceleration of the amplitude limiting process. The speed adjustment step size is used to characterize the magnitude of the robot's speed change in the corresponding speed state and to control the robot to gradually update its motion speed during the speed adjustment process.

[0129] For example, at low speeds, based on the target acceleration after amplitude limiting, a speed adjustment step size corresponding to the low-speed state is set for the robot. This allows the robot to perform only small speed updates in each control cycle, resulting in a smoother speed change process, suitable for motion control requirements in low-speed walking or complex terrain conditions. At high speeds, based on the same target acceleration after amplitude limiting, a speed adjustment step size corresponding to the high-speed state is set for the robot. This allows the robot to perform relatively larger speed updates in each control cycle, thereby accelerating the speed adjustment process while maintaining overall motion stability and meeting the response efficiency requirements of high-speed motion.

[0130] For example, when a robot transitions from a stationary state to a low-speed state, the speed adjustment step size can be set based on the target acceleration after amplitude limiting. When the robot further transitions from a low-speed state to a high-speed state, the speed adjustment step size is increased while keeping the target acceleration direction unchanged, thereby matching the speed adjustment method with the current speed state.

[0131] Finally, the step size is adjusted according to the updated speed, and the robot is controlled to adjust to the target moving speed after passing through several speed states from the current speed. That is, the robot's movement speed is updated step by step in each speed state, so that the robot transitions from the current speed through several speed states and finally reaches the target moving speed.

[0132] By limiting the target acceleration, the subsequent speed adjustment process is carried out within a reasonable acceleration range, thereby providing a stable acceleration input for speed adjustment under different speed conditions.

[0133] In step S202, the speed range of the target's moving speed is determined, and the target control strategy network corresponding to the speed range is selected from multiple control strategy networks.

[0134] Specifically, after acquiring the target's moving speed, a speed range determination is performed. The speed range is used to divide the motion control requirements at different speed levels, enabling the robot to adopt a control strategy network that matches the speed stage.

[0135] In the exemplary embodiments of this disclosure, the speed range may include a first speed range, a second speed range, and a third speed range. The first speed range characterizes a low-speed walking or complex terrain adaptation phase, the second speed range characterizes a high-speed movement or rapid walking on flat terrain phase, and the third speed range characterizes a stationary or near-stationary movement state.

[0136] For example, the target moving speed is compared with a preset interval boundary to determine the speed interval to which the target moving speed belongs. For instance, when the target moving speed is between 0.1 and 1 m / s, the target moving speed is determined to correspond to a first speed interval; when the target moving speed is greater than or equal to 1 m / s, the target moving speed is determined to correspond to a second speed interval; and when the target moving speed is between 0 and 0.1 m / s, the target moving speed is determined to correspond to a third speed interval. It should be noted that this disclosure does not limit the specific boundary values ​​of each speed interval; the interval boundaries can be set according to the robot type, driving capability, and application scenario.

[0137] After determining the speed range to which the target's movement speed belongs, a target control strategy network corresponding to the speed range is selected from multiple control strategy networks. In an example embodiment of this disclosure, the multiple control strategy networks may include a first control strategy network, a second control strategy network, and a third control strategy network. The first speed range corresponding to the first control strategy network is 0.1~1m / s, the second speed range corresponding to the second control strategy network is greater than or equal to 1m / s, and the third speed range corresponding to the third control strategy network is 0~0.1m / s.

[0138] Taking the first control strategy network as a complex terrain control strategy network, the second control strategy network as a flat terrain control strategy network, and the third control strategy network as a zero-speed control strategy network as an example: when the target's movement speed is between 0 and 0.1 m / s, the zero-speed control strategy network is selected; when the target's movement speed is between 0.1 and 1 m / s, the complex terrain control strategy network is selected; and when the target's movement speed is greater than or equal to 1 m / s, the flat terrain control strategy network is selected. The correspondence between speed ranges and control strategy networks can be pre-configured and stored in the main controller to quickly select the control strategy network during operation.

[0139] This step enables the robot to adapt and select from multiple control strategy networks based on the target speed range, allowing the robot to output control strategies that match the target speed at different speed stages, thereby improving the stability and adaptability of the robot's motion control in multi-terrain environments.

[0140] In step S203, a control strategy for controlling the robot's motion is output based on the target control strategy network; wherein, multiple control strategy networks are obtained by training the robot based on deep reinforcement learning within a specified speed range.

[0141] Specifically, after selecting the target control strategy network, the robot's current motion state is input into the target control strategy network. The target control strategy network then generates a control strategy for controlling the robot's motion based on the strategy mapping relationship it has trained within the corresponding speed range. This control strategy instructs the robot's motion control behavior at the current moment and serves as the control output that drives the robot to execute its motion.

[0142] It should be noted that multiple control policy networks were trained on the robot using deep reinforcement learning within their respective designated speed ranges. By training the control policy networks separately for different speed ranges, each network can learn a control policy that matches the motion characteristics and control requirements of its corresponding speed range.

[0143] This step enables the robot to use a control strategy network trained on the corresponding speed range to output a control strategy based on the target's movement speed range during operation. This allows the robot to use a control strategy that is adapted to its motion state at different speed stages, thereby optimizing the user's experience of switching control strategies for different terrains.

[0144] In some example implementations, after the target control policy network outputs a control policy for controlling the robot's motion, the control policy can be further adjusted to match the robot's current speed state.

[0145] For example, the speed adjustment step size corresponding to the current speed state of the robot can be obtained, and the output of the control strategy can be adjusted based on the speed adjustment step size, so that the control strategy is consistent with the current speed change range during actual execution, thereby avoiding the situation where the control strength of the control strategy is mismatched under different speed states.

[0146] Taking a control strategy that includes a target speed command as an example, when the robot is at a low speed, the corresponding speed adjustment step size is 0.02. If the target speed command in the previous control cycle was 0.55 m / s, and the target speed command output by the control strategy network in the current control cycle is 0.60 m / s, then the target speed command in the current control cycle is adjusted to 0.57 m / s to ensure that the change in the target speed command within adjacent control cycles does not exceed the speed adjustment step size. When the robot is at a high speed, the corresponding speed adjustment step size is 0.10. If the target speed command in the previous control cycle was 2.00 m / s, and the target speed command output by the control strategy network in the current control cycle is 2.10 m / s, then the target speed command is allowed to be updated according to the speed adjustment step size to meet the response requirements under high-speed motion conditions.

[0147] By introducing an adjustment process based on speed state and speed adjustment step size after the control strategy output, the control strategy can be adjusted accordingly with changes in speed state, thereby ensuring the continuity and stability of the robot's motion during speed adjustment.

[0148] In addition, during the speed adjustment process, in order to enable the selection of the control strategy network to be dynamically updated as the speed changes, the target moving speed can be updated in stages based on the target acceleration, and the control strategy network can be adjusted accordingly.

[0149] Specifically, at each speed state, the robot's current target moving speed is updated based on the target acceleration to obtain an intermediate moving speed corresponding to the current speed state. The intermediate moving speed is used to characterize the robot's staged target speed during the speed adjustment process and to reflect the speed change of the robot during the transition from the current speed to the final target moving speed.

[0150] After obtaining the intermediate movement speed, a speed range determination is performed to identify its corresponding speed range. If the speed range corresponding to the intermediate movement speed matches the speed range of the previous speed state, the currently selected control strategy network remains unchanged. However, if the speed range corresponding to the intermediate movement speed changes relative to the speed range of the previous speed state, the target control strategy network is updated, ensuring that subsequent control strategy outputs are generated based on the control strategy network that matches the speed range to which the intermediate movement speed belongs.

[0151] By introducing a speed range judgment based on the intermediate moving speed during the speed adjustment process, and updating the control strategy network when the speed range changes, the control strategy network can switch with the speed change process to adapt to the motion control requirements at different speed stages.

[0152] In the exemplary embodiments of this disclosure, to ensure that each control strategy network possesses good control performance within its corresponding speed range, each control strategy network can be pre-trained. Therefore, a method for training a robot's motion control model in multi-terrain environments is also provided. (Reference) Figure 6 As shown, the method may include steps S601 to S603: Step S601: In the deep learning training environment, the robot's speed state during acceleration is divided into a first stationary state, a first low-speed state, a preparatory high-speed state, and a first high-speed state; the robot's speed state during deceleration is divided into a second high-speed state, a preparatory low-speed state, a second low-speed state, and a second stationary state.

[0153] The division of speed states in this step is determined based on the trend and range of motion speed changes of the robot during training. Acceleration represents the phase where the robot's speed increases over time, while deceleration represents the phase where the robot's speed decreases over time. Different speed states correspond to different speed level ranges, used to distinguish the typical motion phases the robot is in during training. This disclosure does not limit the specific method for determining speed states.

[0154] The above division enables the training process to cover the robot's motion states under different speed change directions and different speed levels.

[0155] Step S602: Collect the robot's current state, action output, and environmental feedback information at each speed state to form training data corresponding to each speed state.

[0156] The current state characterizes the robot's motion and posture information at the corresponding speed, and may include state information reflecting the robot's speed, posture, pose change trends, and motion stability. The action output characterizes the control actions generated by the control policy network in the current state, driving the robot to perform corresponding movements in the training environment. Environmental feedback information characterizes the motion results produced by the robot after performing actions and its interaction with the environment.

[0157] During training, for different speed states, corresponding current state, action output, and environmental feedback information are collected, enabling the resulting training data to distinguish the robot's motion behavior characteristics at different speed states. By associating the training data with the corresponding speed states, data support is provided for subsequent updates to the control strategy network parameters based on speed ranges.

[0158] Step S603: Based on the training data corresponding to each speed state and in combination with the speed range to which each speed state belongs, update the parameters of the control policy network corresponding to the speed range.

[0159] During training, the collected training data is categorized according to the speed state division in step S601 and the speed range to which each speed state belongs. Specifically, the training data corresponding to speed states belonging to the same speed range is used as the training input for the control policy network corresponding to that speed range, and is used to update the parameters of the control policy network. It can be understood that the control policy networks corresponding to different speed ranges update their parameters based on the training data within their respective speed ranges.

[0160] For example, training data corresponding to speed states in the third speed range is used to update the parameters of the zero-speed control strategy network; training data corresponding to speed states in the first speed range is used to update the parameters of the complex terrain control strategy network; and training data corresponding to speed states in the second speed range is used to update the parameters of the flat terrain control strategy network.

[0161] By associating speed states with speed ranges and allocating training data accordingly, each control strategy network focuses on learning the motion control characteristics within its corresponding speed range during training. This avoids interference between motion control characteristics in different speed ranges and improves the control performance of each control strategy network within its corresponding speed range.

[0162] The control strategy network is trained in the above manner, enabling each control strategy network to learn a control strategy applicable to its corresponding speed range, thereby providing support for selecting and calling the appropriate control strategy network based on the target movement speed during the operation phase.

[0163] In some example implementations, references Figure 7 As shown, step S603 may further include steps S701 to S703: Step S701: When the robot is in a first low-speed state, a second low-speed state, or a preparatory high-speed state, based on the training data corresponding to each speed state and combined with the first speed interval to which each speed state belongs, update the parameters of the first control strategy network corresponding to the first speed interval.

[0164] When the robot is in any of the following states: first low speed state, second low speed state, and preparatory high speed state, the robot's corresponding movement speed is within the first speed range. At this time, the training process focuses on enabling the robot to effectively adapt to terrain changes, maintain gait stability, and reduce posture fluctuations caused by terrain undulations or changes in foot contact during low-speed walking.

[0165] Taking the first control strategy network as an example, which is a control strategy network for complex terrain. (Reference) Figure 8 As shown, step S701 may further include steps S801 to S803: Step S801: Based on the training data, determine the low-speed motion characteristics of the robot within the first speed range.

[0166] Low-speed motion features are used to characterize the robot's motion adaptability and gait stability under low-speed walking and complex terrain conditions, and serve as feature inputs for subsequently constructing a low-speed interval reward function. For example, low-speed motion features may include terrain undulation response, foot lift height deviation, foot contact change, and low-speed posture change.

[0167] Among them, the terrain undulation response reflects the robot's response to changes in terrain undulation during low-speed walking, characterizing the robot's adaptability to complex terrain. The foot lift height deviation reflects the deviation between the robot's foot lift height during the swing phase and the expected lift height, characterizing the rationality of the gait's foot lift action. The foot contact change reflects the change in the robot's foot contact state during the support phase, characterizing contact stability during low-speed walking; and the low-speed posture change reflects the magnitude of changes in the robot's body posture during low-speed walking, characterizing posture stability during low-speed walking.

[0168] This disclosure does not limit the specific calculation methods for terrain undulation response, foot lift height deviation, foot contact change, and low-speed posture change; these can be determined based on the state information in the training data.

[0169] By extracting low-speed motion features based on training data, the training process can characterize the robot's motion characteristics in the first speed range from multiple dimensions such as terrain adaptation, gait foot lifting, foot contact, and posture stability, providing a basis for subsequent reward function construction and parameter updates of complex terrain control strategy networks.

[0170] Step S802: Based on the low-speed motion characteristics, construct a reward function for the low-speed interval.

[0171] In this step, the low-speed interval reward function is used to give higher rewards to movement behaviors that show good terrain adaptability and gait stability in the first speed interval, and to impose penalties or negative constraints on phenomena such as insufficient terrain adaptability, abnormal gait or posture fluctuations.

[0172] For example, the low-speed interval reward function can be obtained by combining terrain adaptation reward, gait lift reward, contact stability reward and walking posture reward. For example, weighting, normalization, piecewise mapping and other combination methods can be used to integrate the evaluation results of each reward item into an overall reward value that reflects the robot's low-speed walking performance. This disclosure does not limit this.

[0173] Among them, the terrain adaptation reward can be constructed based on the response to terrain undulations. When the robot passes through undulating terrain, if its body movement can remain stable with the changes in terrain and there is no obvious vertical vibration or posture instability, a higher reward value is given; when the robot's response to terrain undulations is large or there is obvious disturbance, the reward value is reduced or a penalty is imposed accordingly.

[0174] For example, the constructed terrain adaptation reward is: in, This indicates terrain adaptation bonus items; Indicates the current altitude of the aircraft. Indicates reference altitude; Indicates the roll angle of the aircraft; Indicates the roll angle of the aircraft; , , This represents the penalty coefficient, used to adjust for the influence of each deviation term. The terrain undulation response is described by height deviation and attitude deviation.

[0175] The gait lift reward can be constructed based on the deviation of the foot lift height. When the lift height of the robot's foot during the swing phase is close to the expected lift height and can meet the requirements of traversing complex terrain, a higher reward value is given. When the foot lift height is significantly insufficient or too high, resulting in an unreasonable gait or affecting walking stability, the reward value is reduced or a penalty is imposed.

[0176] For example, the gait lift reward is: in, This indicates a gait lift reward; Indicates the number of feet; , represents the swing indicator value, which is 1 if it is during the swing phase, and 0 otherwise; This indicates the penalty coefficient for increased height; Indicates the first i The current elevation of the foot tip, Indicates the first i The expected height of each foot tip; This indicates the deviation in the height of the foot lift.

[0177] The contact stability reward can be constructed based on the amount of change in foot contact. When the robot's foot contact state changes little, the contact process is continuous and stable, and there are no frequent slippage, abnormal ground contact, or contact interruption, a higher reward value is given; when the foot contact state changes frequently or is unstable, the reward value is reduced or a penalty is imposed.

[0178] For example, the contact stability reward item is: in, This indicates access to stable reward items; Indicates the change in foot contact; Indicates the number of feet; , indicating the first Whether each foot touches the ground; This indicates the contact state at the previous sampling time; , This represents the corresponding penalty coefficient; This indicates the support level; it is 1 for support phases and 0 otherwise. Indicates the tangential velocity at the foot tip; The square norm represents the tangential velocity at the foot.

[0179] The walking posture reward can be constructed based on the amount of low-speed posture change. When the robot's posture changes little during low-speed walking and it can maintain a stable walking posture, a higher reward value is given; when the posture changes significantly, or there is obvious tilting or fluctuation, the reward value is reduced or a penalty is imposed.

[0180] For example, the reward for walking posture is: in, This indicates a reward for walking posture. , Indicates the penalty coefficient. Indicates the roll angle of the aircraft; Indicates the roll angle of the aircraft; Represents the body's angular velocity vector; This represents the square norm of the body's angular velocity vector. Low-speed attitude changes are described by attitude angles and angular velocities.

[0181] Finally, the constructed low-speed interval reward function can comprehensively reflect the robot's ability to walk on complex terrain and its motion stability in the first speed interval from multiple aspects.

[0182] Step S803: Calculate the reward value corresponding to the training data according to the reward function in the low-speed interval, and update the parameters of the first control policy network based on the reward value.

[0183] Specifically, for each time step or sampling segment in the training data, the corresponding reward value is calculated based on the constructed low-speed interval reward function. Subsequently, the reward value is used as a feedback signal in reinforcement learning training to update the parameters of the complex terrain control policy network. This allows the complex terrain control policy network to gradually adjust its policy output direction during training, thereby making the network's output control actions more inclined to obtain higher low-speed interval rewards.

[0184] For complex terrain control policy networks, policy gradient methods can be used for parameter updates. For example, algorithms such as PPO (Proximal Policy Optimization) and SAC (Soft Actor-Critic) can be selected to update parameters according to training requirements. This disclosure does not limit the specific reinforcement learning algorithm used.

[0185] Therefore, during the parameter update process, the parameter adjustment of the complex terrain control strategy network will focus more on improving the robot's adaptability to terrain undulations, the rationality of gait foot lifting, the stability of foot contact, and the smoothness of walking posture in low-speed walking state, so that the trained control strategy is more suitable for the motion control requirements of complex terrain in the first speed range.

[0186] Step S702: When the robot is in the first high-speed state, the second high-speed state, or the preparatory low-speed state, based on the training data corresponding to each speed state and combined with the second speed interval to which each speed state belongs, update the parameters of the second control strategy network corresponding to the second speed interval.

[0187] When the robot is in the first high-speed state, the second high-speed state, or the preparatory low-speed state, the corresponding movement speed of the robot is in the second speed range. At this time, the training process focuses on enabling the robot to maintain good posture stability and speed tracking performance during high-speed movement, and improving the accuracy of foot contact, so as to meet the requirements of motion coordination and control precision for high-speed walking on flat terrain.

[0188] Taking the second control strategy network as an example, specifically the flat terrain control strategy network. (Reference) Figure 9 As shown, step S702 may further include steps S901 to S903: Step S901: Based on the training data, determine the high-speed motion characteristics of the robot in the second speed range.

[0189] High-speed motion features are used to characterize the robot's posture stability, velocity tracking ability, and foot contact accuracy during high-speed walking, and serve as feature inputs for subsequently constructing a high-speed interval reward function. For example, high-speed motion features may include dynamic posture changes, velocity tracking deviations, and foot contact deviations.

[0190] Among them, the dynamic attitude change quantity reflects the dynamic change range of the robot's body attitude during high-speed movement, thus characterizing the attitude stability during high-speed walking. The speed tracking deviation quantity reflects the deviation between the robot's actual movement speed and the target movement speed, thus characterizing the speed tracking performance during high-speed movement. The foot contact deviation quantity reflects the deviation between the actual contact state of the robot's foot and the expected contact state during high-speed walking, thus characterizing the accuracy and reliability of foot contact.

[0191] This disclosure does not limit the specific calculation methods for dynamic attitude change, velocity tracking deviation, and foot contact deviation; they can be determined based on the state information in the training data.

[0192] By extracting high-speed motion features based on training data, the training process can characterize the robot's motion characteristics in the second velocity range from multiple dimensions such as posture stability, speed tracking, and foot contact, providing a basis for subsequent reward function construction and parameter updates of the flat terrain control strategy network.

[0193] Step S902: Based on the high-speed motion characteristics, construct a high-speed interval reward function.

[0194] In this step, the high-speed interval reward function is used to give higher rewards to movement behaviors that are conducive to high-speed walking on flat terrain within the second speed interval, and to impose penalties or negative constraints on phenomena such as posture fluctuations, speed tracking deviations, and inaccurate foot contact.

[0195] For example, the high-speed interval reward function can be obtained by combining dynamic posture reward, velocity tracking reward and contact accuracy reward, with each reward evaluating the robot's high-speed motion performance from different dimensions.

[0196] The dynamic posture reward is constructed based on the dynamic posture changes and is used to reflect the stability of the robot's posture during high-speed movement. When the robot's posture changes are small and the posture remains stable during high-speed walking, a higher reward value is given; when the posture changes are large and there are obvious posture fluctuations such as pitch, roll, or jitter, the reward value is reduced or a penalty is applied to guide the robot to maintain good posture stability during high-speed walking.

[0197] For example, the constructed dynamic pose reward: in, Indicates the dynamic posture reward item. , It represents the angular velocity components of the aircraft about the roll and pitch axes, and is used to characterize the rate of attitude change during high-speed motion. , It represents the angular acceleration component corresponding to the angular velocity, used to characterize jitter and high-frequency fluctuations in attitude changes; , This represents the weighting coefficients, used to adjust the influence of the angular velocity and angular acceleration terms on the dynamic attitude reward.

[0198] The speed tracking reward can be constructed based on the speed tracking deviation to reflect the robot's ability to follow the target's movement speed with its actual speed. When the robot's actual speed can follow the target's movement speed well and the speed tracking error is small, a higher reward value is given; when the speed tracking deviation is large and there is obvious overshoot or lag, the reward value is reduced or a penalty is imposed to guide the robot to improve its speed tracking accuracy and response performance in high-speed motion.

[0199] For example, the speed tracking reward item is constructed: in, Indicates speed tracking reward items; This represents the robot's one-dimensional forward velocity; This indicates the robot's target moving speed; This represents the speed deviation penalty coefficient.

[0200] The contact accuracy reward can be constructed based on the foot contact deviation to reflect the accuracy of the robot's foot contact and lift-off timing during high-speed walking. A higher reward value is given when the robot's foot contact state is highly consistent with the expected contact state and the contact and lift-off timing is reasonable; when there is premature contact, delayed lift-off, or abnormal contact, the reward value is reduced or a penalty is imposed to guide the robot to maintain the accuracy and regularity of foot contact behavior during high-speed walking.

[0201] For example, constructing a reward item based on accurate contact: in, To access accurate reward items; Indicates the number of feet; , indicating the first Whether each foot touches the ground; , indicating the first The desired contact state at the foot tip; This indicates the amount of contact deviation at the foot.

[0202] Finally, the dynamic posture reward, velocity tracking reward, and contact accuracy reward are combined to construct a high-speed interval reward function. The combination method can be implemented using weighted or normalized methods according to training requirements; this disclosure does not limit this approach.

[0203] In this way, the high-speed interval reward function can comprehensively evaluate the robot's high-speed walking behavior from multiple aspects such as posture stability, speed tracking performance, and foot contact accuracy, providing effective reward guidance for the training of the second control strategy network.

[0204] Step S903: Calculate the reward value corresponding to the training data according to the high-speed interval reward function, and update the parameters of the second control policy network based on the reward value.

[0205] The specific implementation of this step can be found in step S803 of another example embodiment, which will not be repeated here. The difference is that the reward function used in this step is a high-speed range reward function, which is used to evaluate the robot's motion behavior on flat terrain within the second speed range.

[0206] Therefore, during the parameter update process, the parameter adjustment of the flat terrain control strategy network will focus more on improving the robot's posture stability, target speed tracking accuracy, and foot contact accuracy in high-speed motion, so that the trained control strategy is more suitable for the fast walking control requirements of flat terrain in the second speed range.

[0207] Step S703: When the robot is in the first stationary state or the second stationary state, based on the training data corresponding to each speed state and combined with the third speed interval to which each speed state belongs, update the parameters of the third control strategy network corresponding to the third speed interval.

[0208] When the robot is in the first or second stationary state, its corresponding motion speed falls within the third speed range. At this point, the training process focuses on maintaining the robot's static stability, suppressing posture and position drift, and reducing ineffective micro-motions.

[0209] Taking the third control strategy network as an example, which is a zero-speed control strategy network. (Reference) Figure 10 As shown, step S703 may further include steps S1001 to S1003: Step S1001: Based on the training data, determine the stationary state characteristics of the robot in the third velocity range.

[0210] Stationary state features are used to quantify the degree to which a robot is truly stationary and stable, providing input for the subsequent construction of the reward function. For example, stationary state features can include changes in stationary posture, changes in stationary position, and changes in stationary velocity.

[0211] Among them, the static attitude change reflects the magnitude of the robot's posture change within adjacent moments or a preset time window. The static position change reflects the displacement change of the robot's body position or center of mass position within adjacent moments or a preset time window, thus characterizing the robot's positional stability in a static state. The static velocity change reflects the fluctuation of the robot's linear velocity within the third velocity range, thus characterizing whether the robot exhibits any unwanted minute movements.

[0212] Specifically, during training, the robot's state information at consecutive moments can be obtained from the training data corresponding to the third velocity interval, and this state information can be processed to extract static state features. Processing the state information may include calculating the difference between state quantities at adjacent moments, statistical analysis of state quantities within a preset time window, or filtering and analyzing the changes in state quantities, thereby obtaining static posture changes, static position changes, and static velocity changes used to characterize the static state.

[0213] By extracting static state features based on training data, the training process can characterize the robot's static state characteristics in the third velocity range from multiple dimensions such as posture, position, and velocity, providing a basis for subsequent reward function construction and parameter updates of the zero-velocity control policy network.

[0214] Step S1002: Construct a zero-speed interval reward function based on the static state characteristics.

[0215] In this step, the zero-speed interval reward function is used to give higher rewards for more stable stationary behavior and to impose penalties or negative constraints on phenomena such as attitude drift, position drift and velocity fluctuation.

[0216] For example, the zero-velocity interval reward function can be obtained by combining the stationary attitude reward term, the stationary position reward term, and the stationary velocity reward term, specifically: The static posture reward can be constructed based on the amount of static posture change. When the robot's posture change is small, a higher reward value is given; when the robot's posture change is large, a lower reward value or penalty is given, so as to guide the robot to suppress posture jitter and tilt changes in a static state.

[0217] For example, the constructed static pose reward term is: in, Indicates the reward for static posture; This is the attitude reward coefficient, used to adjust the sensitivity of the static attitude reward term to changes in attitude. This represents the change in stationary attitude between adjacent time points; Indicates the robot's current state. The body attitude parameters can be obtained using attitude angles, attitude vectors, or other equivalent attitude representations; Indicates the robot at the previous moment The body attitude parameters; This represents the vector norm, used to calculate the magnitude of changes in position parameters.

[0218] The static position reward can be constructed based on the amount of static position change. When the displacement of the robot's body position or center of mass position is small, a higher reward value is given; when there is obvious position drift, the reward value is reduced or a penalty is imposed to guide the robot to maintain positional stability within the third velocity range.

[0219] For example, the constructed static position reward: in, This indicates a reward for a stationary position. This is the position reward coefficient, used to adjust the sensitivity of the static position reward to changes in position. This represents the change in stationary position between adjacent moments; Indicates the robot's current state. The body position parameters; Indicates the robot at the previous moment The body position parameters.

[0220] The stationary speed reward can be constructed based on the change in stationary speed. When the robot's linear speed fluctuates little and is close to zero speed, a higher reward value is given. When the robot has unwanted small movements or large speed fluctuations, the reward value is reduced or a penalty is imposed to suppress invalid movements in the stationary state.

[0221] For example, the constructed static velocity reward: in, This indicates a stationary speed bonus item; This is the speed bonus coefficient, used to adjust the sensitivity of the stationary speed bonus to changes in speed. This represents the change in velocity at rest; Indicates the robot's current state. The linear velocity parameters; Indicates the robot at the previous moment The linear velocity parameter.

[0222] It should be noted that the specific values ​​of the reward coefficients in each reward item can be set according to the robot's structural characteristics, stability requirements, or training needs, and this disclosure does not impose any restrictions on this.

[0223] Finally, a zero-speed interval reward function is constructed based on the static attitude reward term, the static position reward term, and the static velocity reward term.

[0224] For example, a weighted summation can be used to construct the zero-speed interval reward function: in, The zero-speed interval reward function; This is a weighting coefficient used to adjust the contribution of each reward item to the total reward.

[0225] Step S1003: Calculate the reward value corresponding to the training data according to the zero-speed interval reward function, and update the parameters of the third control policy network based on the reward value.

[0226] The specific implementation of this step can be found in step S803 of another example embodiment, which will not be repeated here. The difference is that in this step, the reward function used is a zero-speed interval reward function, which is used to evaluate the robot's zero-speed or quasi-stationary control behavior in the third speed interval.

[0227] Through the above training process, the zero-speed control strategy network can learn a more stable zero-speed control strategy in the third speed range, thereby reducing attitude and position drift and suppressing speed fluctuations, thus improving the stability and controllability of the robot in a stationary state.

[0228] In the exemplary implementation of this disclosure, by introducing speed range division during the training process and setting corresponding reward functions and control policy networks for different speed ranges, each control policy network can perform targeted learning around its applicable speed range during the training phase, avoiding the performance trade-off caused by a single control policy needing to take into account multiple motion targets simultaneously when covering the full speed range.

[0229] Under this design, the control strategy network focuses on maintaining static stability at zero speed, emphasizes improving adaptability to complex terrain during low-speed walking, and pays more attention to attitude stability and target speed tracking during high-speed movement, making the control behavior clearer and more stable in different speed ranges.

[0230] Therefore, the trained multi-control strategy network can select and switch according to the robot's current speed state during actual operation, and maintain the continuity of control output during speed changes, thereby improving the robot's motion control effect and operational reliability under various terrain and speed conditions.

[0231] refer to Figure 11 The diagram shown is a schematic of a robot simulation environment in this example embodiment. Figure 11 The simulation environment includes a large stone road surface composed of multiple large stones to simulate a complex terrain environment with obvious undulations and discontinuous contact conditions.

[0232] refer to Figure 12 The diagram shown illustrates the state changes of a robot during the switching process of a control strategy network in this example embodiment. Figure 12 The horizontal axis represents time, and the vertical axis, from top to bottom, represents the robot's pitch angle, acceleration, and knee joint position. The acceleration change refers to the change in acceleration of the robot along the vertical direction.

[0233] exist Figure 12 In the diagram, 1201 represents the motion phase when the robot is controlled using a flat terrain control strategy network, during which the robot operates on relatively flat terrain. 1202 represents the motion phase when the robot switches to a complex terrain control strategy network, during which the robot operates on complex terrain such as... Figure 11 The large stone road shown continues to move.

[0234] Curve 1203 represents the robot's pitch angle variation curve, reflecting the overall trend of attitude change before and after the motion control strategy switch. When the robot operates on flat terrain, the overall pitch angle fluctuation is small, and the attitude change is relatively smooth. After the terrain changes, the fluctuation of the pitch angle increases due to the undulation of the road surface with large rocks, but there are no obvious abrupt changes during the strategy switch, and the overall change remains continuous but still within a controllable range.

[0235] Curve 1204 represents the acceleration variation curve of the robot's body, reflecting the dynamic response of the robot when in contact with the ground under different terrain conditions. When the robot operates on flat terrain, the acceleration variation is relatively stable. After the terrain changes, the acceleration exhibits more obvious fluctuations with the terrain undulations, but the acceleration variation process remains continuous before and after the switch of motion control strategy, without any abnormal jumps.

[0236] The two curves, 1205, represent the positional changes of the robot's knee joints on both sides, reflecting the overall adjustment trend of the robot's lower limb joints under different terrain conditions and motion control strategy switching. When the robot operates on flat terrain, the knee joint position changes relatively smoothly. After the terrain changes, the knee joint position adjusts accordingly, and the overall change process is continuous, with no obvious discontinuities at strategy switching points.

[0237] It can be seen that the robot's body posture, acceleration, and knee joint position all changed continuously before and after the motion control strategy was switched, indicating that the motion control strategy switching process was stable and reliable, and the robot was able to maintain stable movement on the rocky road surface after the switch was completed.

[0238] In an exemplary embodiment of this disclosure, a motion control device for a robot in multi-terrain environments is also provided. (See reference...) Figure 13 As shown, the motion control device 1300 for robots in multi-terrain environments includes a speed acquisition module 1301, a network selection module 1302, and a strategy output module 1303, wherein: The speed acquisition module 1301 is used to acquire the target moving speed of the robot; The network selection module 1302 is used to determine the speed range of the target's moving speed and select the target control strategy network corresponding to the speed range from multiple control strategy networks; The strategy output module 1303 is used to output a control strategy for controlling the robot's motion based on the target control strategy network; The multiple control strategy networks are obtained by training the robot using deep reinforcement learning within a specified speed range. The multiple control strategy networks include at least a first control strategy network and a second control strategy network. The first control strategy network is used to control the robot's movement in a first terrain, and the second control strategy network is used to control the robot's movement in a second terrain. The ruggedness of the first terrain is higher than that of the second terrain. The first speed range corresponding to the first control strategy network and the second speed range corresponding to the second control strategy network are set continuously, and the upper speed limit of the first speed range is less than or equal to the lower speed limit of the second speed range.

[0239] The specific details of each module in the motion control device of the robot in multi-terrain environments have been described in detail in the corresponding motion control method of the robot in multi-terrain environments, so they will not be repeated here.

[0240] In an exemplary embodiment of this disclosure, a motion control model training device for a robot in multi-terrain environments is also provided. (See reference...) Figure 14 As shown, the robot motion control model training device 1400 in multi-terrain environments includes a velocity state division module 1401, a training data generation module 1402, and a policy network training module 1403, wherein: The velocity state division module 1401 is used to divide the velocity state of the robot during acceleration into a first stationary state, a first low-speed state, a preparatory high-speed state, and a first high-speed state in a deep learning training environment, and to divide the velocity state of the robot during deceleration into a second high-speed state, a preparatory low-speed state, a second low-speed state, and a second stationary state. The training data generation module 1402 is used to collect the robot's current state, action output and environmental feedback information at each speed state to form training data corresponding to each speed state. The policy network training module 1403 is used to update the parameters of the control policy network corresponding to each speed range based on the training data corresponding to each speed state and in combination with the speed range to which each speed state belongs.

[0241] The specific details of each module in the above-mentioned robot motion control model training device in multi-terrain environments have been described in detail in the corresponding robot motion control model training method in multi-terrain environments, so they will not be repeated here.

[0242] In an exemplary embodiment of this disclosure, a robot is also provided. The robot includes a processor and a memory, the memory storing computer-readable instructions. When executed by the processor, the computer-readable instructions implement the aforementioned method. The robot includes any one of legged robots, wheeled robots, and humanoid robots. Legged robots can be quadrupedal robots, bipedal robots, etc., and wheeled robots can be wheel-legged robots, such as quadrupedal robots. Of course, other types of mobile robots can also be used, and this disclosure does not limit them.

[0243] refer to Figure 15 As shown, a schematic diagram of a type of robot, as illustrated in 1501, is shown that can be applied to the motion control method of the robot in a multi-terrain environment according to the embodiments of this disclosure.

[0244] refer to Figure 16 As shown, a schematic diagram of another type of robot for which the motion control method of the robot in multi-terrain environments, as described in the embodiments of this disclosure, can be applied is illustrated. Figure 16 It can be seen that this type of robot is a humanoid robot.

[0245] refer to Figure 17 As shown, an electronic device capable of implementing the above method is also provided. The electronic device 1700 includes a processor 1701 and a memory 1702. The memory 1702 stores computer-readable instructions, which, when executed by the processor 1701, implement the method of this disclosure.

[0246] In an exemplary embodiment of this disclosure, a computer-readable storage medium is also provided, having stored thereon computer program code instructions that, when invoked by a processor, cause the processor to execute the method described in the embodiments.

[0247] refer to Figure 18As shown, a program product 1800 for implementing the above-described method according to an embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0248] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0249] Finally, the above preferred embodiments are only used to illustrate the technical solutions of this application and are not restrictive. Although this application has been described in detail, those skilled in the art should understand that changes in form and detail can be made without departing from the scope defined by the claims of this application. The dimensions in the drawings are not related to the specific physical object, and the physical object dimensions can be arbitrarily changed.

Claims

1. A method for motion control of a robot in multi-terrain environments, characterized in that, The method comprises: obtaining a target moving speed of a robot; judging a speed interval of the target moving speed, and selecting a target control strategy network corresponding to the speed interval from a plurality of control strategy networks; outputting a control strategy for controlling movement of the robot based on the target control strategy network; wherein the plurality of control strategy networks are obtained by training the robot based on deep reinforcement learning within a specified speed interval; the plurality of control strategy networks comprise at least a first control strategy network and a second control strategy network, the first control strategy network is used to control movement of the robot in a first terrain, the second control strategy network is used to control movement of the robot in a second terrain, the ruggedness of the first terrain is higher than that of the second terrain; a first speed interval corresponding to the first control strategy network and a second speed interval corresponding to the second control strategy network are arranged continuously, and the upper limit of the speed of the first speed interval is less than or equal to the lower limit of the speed of the second speed interval.

2. The method of Claim 1, wherein, The first control strategy network is a complex terrain control strategy network, and the second control strategy network is a flat terrain control strategy network.

3. The method of Claim 1, wherein, The first speed interval is 0.1-1 m / s, and the second speed interval is greater than or equal to 1 m / s.

4. The method of Claim 1, wherein, The plurality of control strategy networks further comprise a third control strategy network, a third speed interval corresponding to the third control strategy network is arranged continuously with the first speed interval corresponding to the first control strategy network, and the upper limit of the speed of the third speed interval is less than or equal to the lower limit of the speed of the first speed interval.

5. The method of motion control of a robot in a multi-terrain environment according to claim 4, wherein, The third control strategy network is a zero-speed control strategy network, and the third speed interval is 0-0.1 m / s.

6. The method of motion control of a robot in a multi-terrain environment of claim 1, wherein, The method further comprises: in response to a speed adjustment instruction, obtaining a target acceleration corresponding to the speed adjustment instruction; adjusting the robot from a current speed to the target moving speed through a plurality of speed states according to the target acceleration; wherein the robot is controlled to move based on the plurality of control strategies in the plurality of speed states.

7. The method of motion control of a robot in a multi-terrain environment according to claim 6, wherein, The adjusting the robot from the current speed to the target moving speed through a plurality of speed states according to the target acceleration comprises: if the target acceleration is positive, adjusting the current speed to the target moving speed through a plurality of speed states corresponding to positive acceleration; if the target acceleration is negative, adjusting the current speed to the target moving speed through a plurality of speed states corresponding to negative acceleration.

8. The method of motion control of a robot in a multi-terrain environment according to claim 7, wherein, The plurality of speed states corresponding to positive acceleration are a plurality of speed states in a first speed state group, and the first speed state group comprises a first stationary state, a first low-speed state, a preparatory high-speed state, and a first high-speed state. The plurality of speed states corresponding to negative acceleration are a plurality of speed states in a second speed state group, and the second speed state group comprises a second high-speed state, a preparatory low-speed state, a second low-speed state, and a second stationary state.

9. The method of motion control of a robot in a multi-terrain environment according to claim 8, wherein, The method further comprises: comparing the absolute value of the target acceleration with a preset acceleration threshold; In the case where the absolute value is greater than the preset acceleration threshold, the robot is controlled to skip the preparatory high-speed state or the preparatory low-speed state to speed up the speed adjustment; In the case where the absolute value is less than or equal to the preset acceleration threshold, the robot is controlled to sequentially pass through several speed states in the first speed state group or several speed states in the second speed state group to complete the speed adjustment.

10. The method of Claim 6, wherein, The target acceleration corresponding to the speed adjustment instruction is obtained, including: The current speed of the robot when the speed adjustment instruction is received is obtained; The target acceleration is calculated according to the speed difference between the target moving speed and the current speed.

11. The method of motion control of a robot in a multi-terrain environment of claim 6, wherein, After obtaining the target acceleration corresponding to the speed adjustment instruction, the method further includes: The target acceleration is limited in amplitude; Based on the target acceleration after the amplitude limiting processing, the speed adjustment step length of the robot in each speed state is updated, and the target moving speed is adjusted from the current speed through several speed states according to the speed adjustment step length.

12. The method of motion control of a robot in a multi-terrain environment of claim 11, wherein, After outputting the control strategy for controlling the motion of the robot, the method includes: The control strategy is adjusted based on the speed adjustment step length corresponding to the current speed state.

13. The method of motion control of a robot in a multi-terrain environment of claim 6, wherein, The method further includes: In each speed state, the target moving speed of the robot is updated based on the target acceleration to obtain an intermediate moving speed; The speed interval of the intermediate moving speed is judged, and in the case where the speed interval of the intermediate moving speed changes relative to the speed interval corresponding to the previous speed state, the target control strategy network is updated. 14.A method for training a motion control model of a robot in a multi-terrain environment, the method comprising: Including: In a deep learning training environment, the speed states of the robot in the acceleration process are divided into a first stationary state, a first low-speed state, a preparatory high-speed state, and a first high-speed state, and the speed states of the robot in the deceleration process are divided into a second high-speed state, a preparatory low-speed state, a second low-speed state, and a second stationary state; The current state, action output and environment feedback information of the robot in each speed state are collected to form training data corresponding to each speed state; Based on the training data corresponding to each speed state and in combination with the speed interval to which each speed state belongs, the parameters of the control strategy network corresponding to the speed interval are updated.

15. The method of claim 14, wherein the robot motion control model is trained in a multi-terrain environment. The speed interval includes a first speed interval, a second speed interval and a third speed interval; Based on the training data corresponding to each speed state and in combination with the speed interval to which each speed state belongs, the parameters of the control strategy network corresponding to the speed interval are updated, including: When the robot is in the first low-speed state or the second low-speed state or the preparatory high-speed state, the parameters of the first control strategy network corresponding to the first speed interval are updated based on the training data corresponding to each speed state and in combination with the first speed interval to which each speed state belongs. When the robot is in the first high-speed state or the second high-speed state or the preparatory low-speed state, parameters of a second control strategy network corresponding to a second speed interval to which each speed state belongs are updated based on training data corresponding to each speed state; When the robot is in the first static state or the second static state, parameters of a third control strategy network corresponding to a third speed interval to which each speed state belongs are updated based on training data corresponding to each speed state. 16.The method of claim 15, wherein, The updating of the parameters of the first control strategy network corresponding to the first speed interval based on the training data corresponding to each speed state and in combination with the first speed interval to which each speed state belongs includes: determining low-speed motion characteristics of the robot in the first speed interval based on the training data; constructing a low-speed interval reward function based on the low-speed motion characteristics; calculating reward values corresponding to the training data according to the low-speed interval reward function, and updating the parameters of the first control strategy network based on the reward values.

17. The method of claim 16, wherein the robot motion control model is trained in a multi-terrain environment. The low-speed motion characteristics include a terrain fluctuation response amount, a foot end lifting height deviation amount, a foot end contact change amount, and a low-speed posture change amount. The constructing of the low-speed interval reward function based on the low-speed motion characteristics includes: constructing a terrain adaptation reward item according to the terrain fluctuation response amount; constructing a gait lifting reward item according to the foot end lifting height deviation amount; constructing a contact stability reward item according to the foot end contact change amount; constructing a walking posture reward item according to the low-speed posture change amount; constructing the low-speed interval reward function based on the terrain adaptation reward item, the gait lifting reward item, the contact stability reward item, and the walking posture reward item.

18. The method of claim 15, wherein, The updating of the parameters of the second control strategy network corresponding to the second speed interval based on the training data corresponding to each speed state and in combination with the second speed interval to which each speed state belongs includes: determining high-speed motion characteristics of the robot in the second speed interval based on the training data; constructing a high-speed interval reward function based on the high-speed motion characteristics; calculating reward values corresponding to the training data according to the high-speed interval reward function, and updating the parameters of the second control strategy network based on the reward values.

19. The method of claim 18, wherein, The high-speed motion characteristics include a dynamic posture change amount, a speed tracking deviation amount, and a foot end contact deviation amount. The constructing of the high-speed interval reward function based on the high-speed motion characteristics includes: constructing a dynamic posture reward item according to the dynamic posture change amount; constructing a speed tracking reward item according to the speed tracking deviation amount; constructing a contact accuracy reward item according to the foot end contact deviation amount; constructing the high-speed interval reward function based on the dynamic posture reward item, the speed tracking reward item, and the contact accuracy reward item.

20. The method of claim 15, wherein, The updating of the parameters of the third control strategy network corresponding to the third speed interval based on the training data corresponding to each speed state and in combination with the third speed interval to which each speed state belongs includes: determine a still state feature of the robot in the third speed interval based on the training data; construct a zero speed interval reward function based on the still state feature; calculate a reward value corresponding to the training data according to the zero speed interval reward function, and update parameters of the third control policy network based on the reward value. 21.The method of claim 20, wherein, The still state feature includes a still posture change amount, a still position change amount, and a still speed change amount. The method further includes: construct a still posture reward item according to the still posture change amount; construct a still position reward item according to the still position change amount; construct a still speed reward item according to the still speed change amount; construct the zero speed interval reward function based on the still posture reward item, the still position reward item, and the still speed reward item.

22. A motion control device of a robot in a multi-terrain environment, characterized by, The method further includes: a speed acquisition module configured to acquire a target moving speed of the robot; a network selection module configured to determine a speed interval of the target moving speed, and select a target control policy network corresponding to the speed interval from a plurality of control policy networks; a policy output module configured to output a control policy for controlling movement of the robot based on the target control policy network. The plurality of control policy networks are obtained by training the robot in a specified speed interval based on deep reinforcement learning. The plurality of control policy networks include at least a first control policy network and a second control policy network, the first control policy network is configured to control movement of the robot in a first terrain, the second control policy network is configured to control movement of the robot in a second terrain, ruggedness of the first terrain is higher than ruggedness of the second terrain, a first speed interval corresponding to the first control policy network and a second speed interval corresponding to the second control policy network are arranged continuously, and an upper limit of the speed in the first speed interval is less than or equal to a lower limit of the speed in the second speed interval. 23.A device for training a motion control model of a robot in a multi-terrain environment, characterized in that, The method further includes: a speed state division module configured to divide, in a deep learning training environment, a speed state of the robot in an acceleration process into a first still state, a first low speed state, a preparatory high speed state, and a first high speed state, and divide a speed state of the robot in a deceleration process into a second high speed state, a preparatory low speed state, a second low speed state, and a second still state; a training data generation module configured to acquire current state, action output, and environment feedback information of the robot in each speed state to form training data corresponding to each speed state; a policy network training module configured to update parameters of a control policy network corresponding to each speed state based on training data corresponding to each speed state and a speed interval to which each speed state belongs.

24. An electronic device, comprising: The method further includes: a processor; and a memory having computer readable instructions stored thereon, the computer readable instructions being executed by the processor to implement the method of any one of claims 1-21.

25. A robot characterized by The method further includes: a processor; and A memory having computer readable instructions stored thereon, the computer readable instructions, when executed by the processor, implement the method of any one of claims 1-13.

26. The robot of claim 25, wherein, The robot includes any one of a legged robot, a wheeled robot, and a humanoid robot.

27. A computer-readable storage medium, characterized in that, The computer readable storage medium has computer program code instructions stored thereon, the computer program code instructions, when invoked by a processor, cause the processor to perform the method of any one of claims 1-21.

Images