Method for motion control of a robot and robot

Abstract

The application provides a motion control method of a robot and the robot, and relates to the technical field of computers. The method comprises the following steps: receiving a movement instruction and a posture instruction for a target robot, and obtaining body perception observation data of the target robot; inputting the movement instruction and the posture instruction as independent parallel instruction vectors on a time axis into a pre-trained motion control network, and inputting the body perception observation data into the motion control network, so as to obtain a coordinated action vector between movement execution and posture adjustment by the motion control network; determining a target motion instruction of the target robot according to the coordinated action vector; and controlling the target robot to move and adjust the posture according to the target motion instruction. By introducing two independent parallel instruction input channels into an input layer of the motion control network, the movement instruction and the posture instruction are inputted into the motion control network as independent parallel instruction vectors on the time axis, so that the coordinated control based on the movement execution and the posture adjustment is realized.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a motion control method for a robot and a robot. Background Technology

[0002] In recent years, with the continuous development of robot technology, wheel-legged hybrid robots have become a research hotspot in the field of legged robots because they combine the high mobility of legged robots with the high-speed mobility of wheeled robots. High-precision pose control of wheel-legged hybrid robots has also become a research trend.

[0003] In existing technologies, independent networks are typically used to estimate movement and attitude separately, and then movement and attitude are controlled sequentially according to priority. At the same time, existing technologies usually rely on static foot position constraints to prevent support point drift when aerial robots change attitude.

[0004] The aforementioned separate control of movement and posture makes it difficult for the robot to adjust its posture in real time during continuous movement, limiting the robot's flexibility and the smoothness of task execution; moreover, static foot position constraints can cause gait instability in the robot, making it difficult to simultaneously achieve support stability and movement compliance. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a motion control method for a robot and a robot in order to improve the control accuracy of wheeled robots.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a motion control method for a robot, the method comprising: Receive movement and posture commands for the target robot and acquire the target robot's body perception observation data; The movement command and the posture command are input as independent parallel command vectors on the time axis into a pre-trained motion control network, and the proprioceptive observation data is input into the motion control network. The motion control network then predicts the coordinated action vector between movement execution and posture adjustment. Based on the coordinated motion vector, the target motion command of the target robot is determined; The target robot is controlled to move and adjust its posture according to the target motion command.

[0007] Optionally, the training method for the motion control network includes the following steps: A simulation environment is constructed, and a first network and a second network are initialized. The first network is used to output action vectors based on the ontology perception observation data, movement commands, and posture commands, and the second network is used to output state values ​​based on the ontology perception observation data, privileged observation data, movement commands, and posture commands. Step A: Obtain the sample ontology perception observation data, sample privileged observation data, and the target movement command and target attitude command corresponding to the current time step; Step B: Input the sample ontology perception observation data, target movement command, and target posture command into the current first network to obtain the predicted action vector; Step C: Control the simulated robot to execute the predicted action vector, and obtain the instantaneous reward at the current time step and the sample ontological perception observation data for the next time step; Step D: Input the sample ontology perception observation data, target movement command, target attitude command, and sample privileged observation data into the current second network to obtain the first prediction value at the current time step; Step E: Input the sample ontology perception observation data of the next time step, the sample privileged observation data of the next time step, and the target movement command and target attitude command corresponding to the next time step into the current second network to obtain the second prediction value of the next time step; Step F: Determine the actual state value of the current time step based on the instant reward and the second predicted value, and update the network parameters of the second network based on the actual state value and the first predicted value; Step G: Determine the advantage value based on the actual state value and the first predicted value, and update the network parameters of the first network using the policy gradient algorithm based on the advantage value; the advantage value is used to characterize the superiority of the action performed at the current time step based on the predicted action vector compared to the average level of the current policy. Step H: Repeat steps A to G until the iteration stop condition is met, and use the first network at the time of iteration stop as the motion control network.

[0008] Optionally, the controlled simulation robot executes the predicted action vector and obtains the immediate reward at the current time step, including: Based on the predicted motion vector, determine the joint movement command; Based on the joint movement commands, the simulated robot is controlled to move and adjust its posture, and the actual motion parameters after the action is executed are obtained; The instantaneous reward for the current time step is determined based on the actual motion parameters, the composite reward function, and the reward weight. The composite reward function includes at least the following: linear velocity tracking reward function, yaw rate tracking reward function, attitude tracking reward function, target attitude projection tracking reward function, terrain-adaptive gravity projection constraint reward function, world coordinate system slip velocity constraint reward function, attitude stability reward function, and asymmetric joint force response reward function.

[0009] Optionally, the immediate reward for the current time step is determined based on the actual motion parameters, the composite reward function, and the reward weight, including: Based on the first parameter corresponding to the linear velocity tracking reward function in the actual motion parameters, the first reward value is determined using the linear velocity tracking reward function; Based on the second parameter corresponding to the yaw rate tracking reward function in the actual motion parameters, the second reward value is determined using the yaw rate tracking reward function; Based on the third parameter corresponding to the attitude tracking reward function in the actual motion parameters, the attitude tracking reward function is used to determine the third reward value; Based on the fourth parameter corresponding to the target attitude projection tracking reward function in the actual motion parameters, the fourth reward value is determined using the target attitude projection tracking reward function; Based on the fifth parameter corresponding to the terrain-adaptive gravity projection constraint reward function in the actual motion parameters, the fifth reward value is determined using the terrain-adaptive gravity projection constraint reward function. Based on the sixth parameter corresponding to the sliding velocity constraint reward function of the world coordinate system in the actual motion parameters, the sixth reward value is determined using the sliding velocity constraint reward function of the world coordinate system. Based on the seventh parameter corresponding to the attitude stability reward function in the actual motion parameters, the seventh reward value is determined using the attitude stability reward function; Based on the eighth parameter corresponding to the asymmetric joint force response reward function in the actual motion parameters, the eighth reward value is determined using the asymmetric joint force response reward function. The immediate reward for the current time step is determined based on the first reward value, the second reward value, the third reward value, the fourth reward value, the fifth reward value, the sixth reward value, the seventh reward value, the eighth reward value, and the reward weight corresponding to each reward function.

[0010] Optionally, the first parameter includes: the linear velocity vector of the base in the world coordinate system; the step of determining the first reward value using the linear velocity tracking reward function based on the first parameter corresponding to the linear velocity tracking reward function in the actual motion parameters includes: The actual forward linear velocity of the fuselage on the horizontal plane is determined based on the unit projection direction of the fuselage forward on the horizontal plane and the linear velocity vector of the base in the world coordinate system. The actual lateral linear velocity of the fuselage on the horizontal plane is determined based on the unit projection direction of the fuselage on the horizontal plane and the linear velocity vector of the base in the world coordinate system. The linear velocity error is determined based on the actual forward linear velocity of the fuselage on the horizontal plane, the desired forward linear velocity, the actual lateral linear velocity of the fuselage on the horizontal plane, and the desired lateral linear velocity. The linear velocity error and the tracking accuracy scaling factor are input into the linear velocity tracking reward function to calculate the first reward value.

[0011] Optionally, the second parameter includes: the three-axis angular velocity vector of the fuselage base in the world coordinate system; the step of determining the second reward value using the yaw rate tracking reward function based on the second parameter corresponding to the yaw rate tracking reward function in the actual motion parameters includes: Based on the three-axis angular velocity vectors of the fuselage base in the world coordinate system and the unit projection of the gravity vector in the fuselage base coordinate system, determine the yaw angular velocity components of the fuselage base around the world gravity axis. The yaw rate error is determined based on the yaw rate component of the fuselage base around the world gravity axis and the desired yaw rate. The yaw rate error and the tracking accuracy scaling factor are input into the yaw rate tracking reward function to calculate the second reward value.

[0012] Optionally, the third parameter includes: the actual roll angle of the fuselage and the actual pitch angle of the fuselage; the attitude tracking reward function includes: a roll angle reward function and a pitch angle reward function; the step of determining the third reward value by using the attitude tracking reward function based on the third parameter corresponding to the attitude tracking reward function in the actual motion parameters includes: Input the actual roll angle and the desired roll angle of the fuselage into the roll angle reward function to determine the roll angle reward value; Input the actual pitch angle and the desired pitch angle of the fuselage into the pitch angle reward function to determine the pitch angle reward value; The third reward value is determined based on the roll angle reward value and the pitch angle reward value.

[0013] Optionally, the fourth parameter includes: the actual gravity vector projection in the fuselage base coordinate system; the step of determining the fourth reward value using the target attitude projection tracking reward function based on the fourth parameter corresponding to the target attitude projection tracking reward function in the actual motion parameters includes: Determine the desired gravity direction vector in the fuselage base coordinate system based on the desired roll angle and desired pitch angle; The actual gravity vector projection in the fuselage base coordinate system and the desired gravity direction vector in the fuselage base coordinate system are input into the target attitude projection tracking reward function to calculate the fourth reward value.

[0014] Optionally, the fifth parameter includes: the actual gravity vector projection in the fuselage base coordinate system; the step of determining the fifth reward value based on the fifth parameter corresponding to the terrain-adaptive gravity projection constraint reward function in the actual motion parameters, using the terrain-adaptive gravity projection constraint reward function, includes: Based on the actual gravity vector projection in the fuselage base coordinate system, determine the components of the actual gravity vector projection on the horizontal plane of the fuselage base coordinate system; Determine the terrain adaptation coefficient based on the terrain information; The component of the actual gravity vector projected onto the horizontal plane of the fuselage base coordinate system and the terrain adaptation coefficient are input into the terrain-adaptive gravity projection constraint reward function to calculate the fifth reward value.

[0015] Optionally, the sixth parameter includes: the instantaneous linear velocity of each foot of the simulated robot in the world coordinate system; the step of determining the sixth reward value based on the sixth parameter corresponding to the slip velocity constraint reward function in the world coordinate system in the actual motion parameters, using the slip velocity constraint reward function in the world coordinate system, includes: Based on the ground contact state of each foot of the simulated robot, determine the corresponding slip penalty coefficient for each foot. The instantaneous linear velocity of each foot in the world coordinate system and the corresponding slip penalty coefficient of each foot are input into the slip velocity constraint reward function of the world coordinate system to calculate the sixth reward value.

[0016] Optionally, the seventh parameter includes: the actual pitch rate of the fuselage, the actual roll rate of the fuselage, the actual horizontal velocity of the fuselage, and the actual landing point position of each swing phase foot; the step of determining the seventh reward value based on the seventh parameter corresponding to the attitude stability reward function in the actual motion parameters, using the attitude stability reward function, includes: The body attitude offset is determined based on the vertical height of the robot's center of mass relative to the ground, the actual pitch rate of the robot body, the actual roll rate of the robot body, and the duration of a single gait cycle. The linear velocity error is determined based on the actual horizontal velocity and the desired horizontal velocity of the fuselage. Determine the feedforward of the actual speed based on the actual horizontal speed of the fuselage; Based on the linear velocity error, the feedforward of the actual velocity, and the duration of a single gait cycle, determine the actual foot landing point position without compensation. Based on the actual foot landing point position without compensation, the fuselage attitude offset, the gait phase activation factor, and the expected foot landing point position without compensation, the expected foot landing point position during the swing phase is determined. The total landing point error is determined based on the actual landing point position of each swing phase foot end and the expected landing point position of the swing phase foot end. The total error of the landing point and the terrain adaptation coefficient are input into the attitude stability reward function to calculate the seventh reward value.

[0017] Optionally, the eighth parameter includes: the supporting force of the left leg of the fuselage and the supporting force of the right leg of the fuselage; the step of determining the eighth reward value based on the eighth parameter corresponding to the asymmetric joint force response reward function in the actual motion parameters, using the asymmetric joint force response reward function, includes: The actual asymmetry of the contact force between the left and right fuselage legs is determined based on the supporting force of the left and right fuselage legs. The actual asymmetry of the left and right leg contact forces and the expected asymmetry of the left and right leg contact forces are input into the asymmetric joint force response reward function to calculate the eighth reward value.

[0018] Optionally, the reward weights corresponding to each reward function can be determined in the following ways: Determine the speed priority factor based on the desired horizontal speed vector; Determine the attitude priority factor based on the speed priority factor; The reward weight corresponding to each reward function is determined based on the reward type of each reward function, the speed priority factor, and the attitude priority factor.

[0019] Secondly, this application also provides a robot, which is a wheeled robot, and a motion control network is deployed on the robot; The robot is used to perform the motion control method for the robot described in the first aspect above.

[0020] Thirdly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to implement the motion control method for the robot provided in the first aspect.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the motion control method for a robot as provided in the first aspect.

[0022] The beneficial effects of this application are: This application provides a motion control method for a robot and a robot, comprising: receiving movement commands and posture commands for a target robot, and acquiring proprioceptive observation data of the target robot; inputting the movement commands and posture commands as independent parallel command vectors on the time axis into a pre-trained motion control network, and inputting the proprioceptive observation data into the motion control network, and predicting a coordinated motion vector between movement execution and posture adjustment by the motion control network; determining a target motion command for the target robot based on the coordinated motion vector; and controlling the target robot to move and adjust its posture according to the target motion command. This embodiment introduces two parallel and independent command input channels into the input layer of the motion control network, allowing movement commands and posture commands to be input as independent parallel command vectors on the time axis into the motion control network. Combined with the input proprioceptive observation data, a coordinated motion vector between movement execution and posture adjustment is predicted through an internal attention mechanism. Therefore, based on the coordinated motion vector, coordinated execution of movement execution and posture adjustment can be achieved, improving the robot's motion flexibility and the smoothness of task execution.

[0023] In particular, training the motion control network by introducing a composite reward function can effectively improve the control accuracy of the motion control network.

[0024] By designing a linear velocity tracking reward function, the robot's actual horizontal movement speed (including forward and lateral linear velocity) can be encouraged to reach the expected forward and lateral linear velocities given in the movement command as much as possible. This ensures that the robot's movement speed in the horizontal plane is as close as possible to the expected movement command, thereby achieving accurate horizontal velocity tracking.

[0025] By designing a reward function that tracks yaw rate, the robot's actual yaw rate can be encouraged to accurately track the desired yaw rate given in the movement command, enabling the robot to accurately execute the desired yaw rotation.

[0026] By designing a posture tracking reward function, the robot's actual tilt posture can be encouraged to accurately track the desired posture command, enabling the robot to achieve active posture tracking.

[0027] By designing a reward function for target posture projection tracking, the conflict between traditional forced horizontal constraints and active posture adjustment can be resolved. This encourages the robot to actively tilt to the desired posture command, rather than passively maintaining horizontality.

[0028] By designing a terrain-adaptive gravity projection constraint reward function, the constraint strength on the robot's horizontal attitude can be adaptively adjusted according to the terrain complexity. This maintains high-precision horizontal control on flat terrain and actively relaxes the horizontal requirements on rugged terrain, allowing the robot to adapt to terrain undulations. This improves the robot's compliance and stability on uneven surfaces. It overcomes the limitations of traditional control methods, which often fail to adapt to terrain changes, leading to stiff movement or frequent falls on rugged terrain.

[0029] By designing a reward function that constrains the sliding velocity in the world coordinate system, the robot's foot slippage during the ground contact phase can be effectively suppressed, while allowing the foot to naturally translate along with the body's tilt. This avoids jamming caused by excessive constraints while ensuring support stability. Specifically, only the foot in the ground contact phase is penalized; the foot in the swing phase should move naturally and is not penalized, thus avoiding interference with normal leg-stepping movements.

[0030] By designing a posture stability reward function, the robot can be guided to actively correct the desired foot placement point during dynamic posture adjustment and variable speed movement to guide precise foot tracking. At the same time, the tracking weight can be adaptively adjusted according to the ruggedness of the terrain, thus achieving a unified high-precision foot placement and terrain-adaptive compliant control when the wheeled robot moves and adjusts its posture.

[0031] By designing an asymmetric joint force response reward function, the motion control network can be guided to actively transfer the load to the lower leg when the robot tilts, thereby enhancing support stability. For example, when the robot body tilts actively or the terrain causes it to tilt, the motion control network can be prompted to actively transfer the load to the downhill side or the tilted leg, preventing sideslip or tipping due to improper load distribution.

[0032] Based on the structural design of wheel-legged robots, this solution correspondingly designs various reward functions, which can control wheel-legged robots to adjust their posture flexibly like legged robots and move efficiently like wheeled robots, while exhibiting strong robustness on complex terrain. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart illustrating a motion control method for a robot provided in an embodiment of this application; Figure 2 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 1 ; Figure 3 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 2 ; Figure 4 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 3 ; Figure 5 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 4 ; Figure 6 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 5 ; Figure 7 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 6 ; Figure 8 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 7 ; Figure 9 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 8 ; Figure 10 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 9 ; Figure 11 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 10 ; Figure 12 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 10 one; Figure 13 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 10 two; Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0036] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0037] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0038] In existing technologies, robot motion control and body posture adjustment are typically handled separately by control loops or models, or require time-series switching between different motion modes. In tasks requiring dynamic adjustment of the robot's perspective, such as inspection and monitoring, the operator or upper-level planner must manually program the "movement-attitude adjustment" sequence. The system cannot execute and respond to movement speed commands and independent posture adjustment commands in parallel within the same time window. This fragmentation of control modes prevents the robot from performing parallel operations of moving and adjusting its posture simultaneously, severely restricting the smoothness and timeliness of task execution.

[0039] Furthermore, existing reinforcement learning control methods typically rely on static foot position constraints to prevent support point drift when guiding robots to change their body height or pitch angle. In wheel-legged robot structures, while such static constraints can reduce the "splits" phenomenon during quasi-static posture adjustment, their rigidity prevents the robot from achieving a balance between support stability and motion compliance. When the robot needs to actively adjust its posture to adapt to terrain or task requirements, static constraints force the feet to become stuck in a fixed projection position, causing joint torque impacts and gait instability.

[0040] Based on this, this application introduces two parallel and independent instruction input channels into the input layer of the motion control network, which can input movement instructions and posture instructions into the motion control network in parallel. Combined with the input ontological perception observation data, the coordinated action vector between movement execution and posture adjustment is predicted through an internal attention mechanism. Based on the coordinated action vector, the coordinated execution of movement execution and posture adjustment can be realized, thereby improving the robot's movement flexibility and the smoothness of task execution.

[0041] To address the challenge of simultaneously achieving both support stability and motion compliance, this paper designs a world coordinate system-based sliding velocity constraint reward function and a posture stability reward function. The world coordinate system-based sliding velocity constraint reward function directly constrains the actual sliding of the foot on the ground, suppressing slippage to ensure support stability. The posture feedforward compensation allows the foot to naturally translate with the body tilt, and through smooth superposition compensation by gait phase activation factors, dynamic adaptation of velocity feedforward terms to command changes, and active reduction of foot tracking weight on rugged terrain by terrain adaptation coefficients, the robot avoids jamming or impact caused by forced constraints. As a result, the robot can maintain stable support in the ground contact phase and smoothly follow body dynamics and terrain undulations in the swing phase, achieving a balance between support stability and motion compliance.

[0042] Figure 1 This is a flowchart illustrating a robot motion control method provided in an embodiment of this application. It should be noted that the executing entity of the robot motion control method can be a target robot equipped with a motion control network, specifically a controller within the target robot. Figure 1 As shown, the method includes: S101. Receive movement and posture commands for the target robot and acquire the target robot's body perception observation data.

[0043] Body perception observation data is used to characterize the motion state of the target robot itself.

[0044] The target robot can be a robot that requires posture control in real-world application scenarios. The target robot can be a wheeled robot, and this method is applicable to robot control scenarios involving both wheeled and legged robots.

[0045] It can receive user input for movement and posture commands. The user can instruct the operator. Movement commands are used to indicate the target robot's current desired movement speed, and posture commands are used to indicate the target robot's current desired posture angle.

[0046] For example, a move instruction can be represented as: , These are represented as desired forward velocity, desired lateral velocity, and desired yaw rate, respectively; the attitude command can be expressed as: ,in, These represent the desired roll angle and the desired pitch angle, respectively.

[0047] The target robot's body perception observation data can be automatically collected and uploaded through the sensors on the target robot. The body perception observation data includes, but is not limited to: the position error of each joint of the robot, the joint velocity, the three-axis angular velocity of the base, the projection of the gravity vector in the base coordinate system, the current Euler angle of the body, etc. The body perception observation data represents the motion state of the target robot itself.

[0048] S102. Input the movement command and attitude command as independent parallel command vectors on the time axis into the pre-trained motion control network, and input the proprioception observation data into the motion control network. The motion control network predicts the coordinated action vector between movement execution and attitude adjustment.

[0049] In this embodiment, movement commands and attitude commands can be input into the motion control network as two parallel and independent command vectors. These two command vectors are completely decoupled on the time axis, so that the processing of movement commands and attitude commands within the motion control network remains parallel and independent. Simultaneously, proprioceptive observation data is also input into the motion control network, allowing the network to predict coordinated action vectors based on the proprioceptive observation data, movement commands, and attitude commands through an internal attention mechanism.

[0050] The coordinated motion vector contains both movement-related information and posture-related information, and coordinated control of movement and posture can be achieved based on the coordinated motion vector.

[0051] S103. Determine the target motion command for the target robot based on the coordinated motion vector.

[0052] Optionally, the coordinated motion vector can be converted into a target motion command via a proportional-derivative controller (PD). In this embodiment, the target motion command may include target torque information of each joint of the target robot, thereby controlling the target robot to execute the coordinated motion vector to switch to the next motion state.

[0053] Of course, in some possible implementations, the target motion command can also be the motion information of each foot of the target robot or the motion information of the head, which can be determined by the control target of the target robot.

[0054] Specifically, the coordinated motion vector can be converted into a target motion command based on the coordinated motion vector and joint impedance parameters. Joint impedance parameters include stiffness and damping parameters. This embodiment, targeting a wheeled robot, sets differentiated stiffness and damping parameters for the wheel joints and leg joints. The wheels employ lower stiffness and higher damping to simulate the torque response characteristics of a real hub motor. This differentiated setting allows for better support of joint forces.

[0055] S104. Control the target robot to move and adjust its posture according to the target motion command.

[0056] By using target motion commands, the joints of the target robot can be controlled to perform actions, thereby achieving adjustments in movement and posture, enabling the target robot to more accurately achieve the desired motion state.

[0057] Because movement commands and attitude commands are completely decoupled on the timeline, the operator or the upper-level autonomous planner can simultaneously issue commands such as "move forward at 1.5 m / s" and "tilt the fuselage 20° to the left," without needing to perform a timing sequence of stopping, adjusting attitude, and restarting, thus achieving coordinated control of movement execution and attitude adjustment.

[0058] In summary, the robot motion control method provided in this embodiment includes: receiving movement commands and posture commands for a target robot, and acquiring the target robot's proprioceptive observation data; inputting the movement commands and posture commands as independent parallel command vectors on the time axis into a pre-trained motion control network, and inputting the proprioceptive observation data into the motion control network, with the motion control network predicting a coordinated motion vector between movement execution and posture adjustment; determining the target motion command for the target robot based on the coordinated motion vector; and controlling the target robot to move and adjust its posture according to the target motion command. This embodiment introduces two parallel and independent command input channels into the input layer of the motion control network, allowing movement commands and posture commands to be input as independent parallel command vectors on the time axis. Combined with the input proprioceptive observation data, an internal attention mechanism predicts the coordinated motion vector between movement execution and posture adjustment, thereby enabling collaborative execution of movement execution and posture adjustment based on the coordinated motion vector, improving the robot's motion flexibility and task execution smoothness.

[0059] Figure 2 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 1It should be noted that the execution entity for the motion control network training method can be an electronic device or processor independent of the simulated robot. The electronic device or processor controls the execution of the following training method to train the motion control network. The motion control network training method includes the following steps: S201. Build the simulation environment and initialize the first and second networks.

[0060] The first network is used to output action vectors based on the proprioceptive observation data, movement commands, and posture commands. The second network is used to output state values ​​based on the proprioceptive observation data, privileged observation data, movement commands, and posture commands.

[0061] The construction of the simulation environment specifically includes: constructing a wheeled robot model as the simulation robot, constructing terrain environments with different terrain features, and defining local perception observation data and privileged observation data, etc.

[0062] The first network is used to predict and output action vectors, and the second network is used to score the state after the action vectors predicted by the first network are executed.

[0063] The input data for the first network can be proprioceptive observation data, movement commands, and posture commands; the input data for the second network can be proprioceptive observation data, privileged observation data, movement commands, and posture commands, outputting state values. Privileged observation data cannot be automatically perceived by the robot's sensors and is collected through third-party devices. Privileged observation data includes, but is not limited to: terrain height scan points, foot 3D contact force, ground friction coefficient, actuator delay parameters, and independent physical properties of robot joints. Privileged observation information, as input data for the second network, can guide the second network to predict more accurate value data.

[0064] S202. Obtain the sample ontology perception observation data, sample privileged observation data, and the target movement command and target attitude command corresponding to the current time step.

[0065] In some embodiments, before formal simulation, the simulated robot can be controlled to perform movement in a preset terrain environment according to a pre-planned movement mode to obtain sample ontological perception observation data, sample privileged observation data, target movement instructions and target posture instructions at each time step, thereby obtaining historical reliable training sample data at each time step.

[0066] At each training time step, the ontological perception observation data, privileged observation data, target movement command, and target posture command corresponding to the current time step can be obtained from the training sample data.

[0067] S203. Input the sample ontology perception observation data, target movement command and target posture command into the current first network to obtain the predicted action vector.

[0068] The input layer of the first network also has two parallel and independent instruction input channels, which can simultaneously input independent and parallel target movement instructions and target attitude instructions. Based on the target movement instructions, target attitude instructions and sample ontology perception observation data, the predicted action vector of the current time step can be predicted.

[0069] S204. Control the simulated robot to execute the predicted action vector, and obtain the immediate reward of the current time step and the sample ontological perception observation data of the next time step.

[0070] After the simulated robot executes the predicted action vector, it reaches a new motion state, and based on this new motion state, the immediate reward for the current time step can be calculated. This immediate reward is used to evaluate the quality of the motion state after executing the predicted action vector, thereby driving the updating and optimization of the first and second networks.

[0071] S205. Input the sample ontology perception observation data, target movement command, target attitude command, and sample privileged observation data into the current second network to obtain the first prediction value at the current time step.

[0072] The input data of the second network is based on the input data of the first network, with the addition of sample privileged observation data. By adding sample privileged observation data, the second network can predict long-term value more accurately and estimate the predicted value under different terrain conditions more accurately based on terrain changes. This allows the first network, even if it does not have access to sample privileged observation data, to still predict more accurate action vectors, thus making it more robust to terrain changes and sensor noise.

[0073] For example, even if the first network cannot know the ground friction coefficient, it will still take a more compliant action because the second network knows that large steps under high friction are prone to slipping. Ultimately, the first network learns to infer similar robust behavior from proprioceptive observation data.

[0074] The second network can predict the first predicted value at the current time step based on the input data. The first predicted value is an estimate of the expected cumulative reward that the simulated robot can obtain after performing an action according to the current predicted action vector at the current time step. That is, the first predicted value is not only the immediate reward at the current time step, but also includes the rewards for all future time steps, where the rewards for future time steps decay continuously according to a discount factor.

[0075] If the first predictive value is high, it means that starting from the current time step, the simulated robot can obtain a lot of positive rewards in the future. If the first predictive value is low, it means that the simulated robot may be in danger at the current time step (such as about to fall or slipping severely), and will accumulate a lot of penalties in the future.

[0076] S206. Input the sample ontology perception observation data of the next time step, the sample privileged observation data of the next time step, and the target movement command and target attitude command corresponding to the next time step into the current second network to obtain the second prediction value of the next time step.

[0077] Similarly, the second network can predict the second predicted value of the next time step based on the input data of the next time step corresponding to the current time step. The second predicted value represents the expected cumulative discount reward that can be obtained by continuing to execute the current strategy from the state of the next time step.

[0078] S207. Determine the actual state value of the current time step based on the immediate reward and the second prediction value, and update the network parameters of the second network based on the actual state value and the first prediction value.

[0079] It is worth noting that the actual state value here is not a real observable quantity, but a constructed target value TD. Therefore, the actual state value at the current time step can be determined first based on the immediate reward and the second predicted value. The actual state value can refer to the noisy estimate of the actual value at the current time step.

[0080] For the current time step, the output of the second network is the first predicted value, and we expect the first predicted value to be as close as possible to the true value. The actual state value constructed above can be used as an estimate of the true value to supervise the learning of the second network. The learning objective of the second network is to minimize the error between the actual state value and the first predicted value. Therefore, the network parameters of the second network can be updated according to the actual state value and the first predicted value.

[0081] The reason why the immediate reward at the current time step is not used directly as the actual state value is that the immediate reward ignores all future rewards, does not have the ability to predict in the long term, and will affect the network's prediction performance.

[0082] S208. Determine the advantage value based on the actual state value and the first predicted value, and update the network parameters of the first network using the policy gradient algorithm based on the advantage value.

[0083] The advantage value is used to characterize how superior the action performed at the current time step based on the predicted action vector is compared to the average level of the current policy.

[0084] In some embodiments, the advantage value can be obtained by subtracting the actual state value from the first predicted value.

[0085] The optimization objective of the first network is to maximize the expected cumulative reward. The advantage value can be used as a weight to guide the first network update through the policy gradient algorithm. This will guide the first network to predict action vectors with higher value, and lower value, with a lower probability of occurrence, thereby improving the accuracy of the predicted action vectors.

[0086] S209. Determine whether the iteration stopping condition is met. If it is met, proceed to step S210. If it is not met, repeat steps S202 to S208.

[0087] S210, Use the first network at the time of iteration stop as the motion control network.

[0088] Repeat the above iterative steps to complete the training task for multiple training time steps. When the iteration stopping condition is met, the first network at the time of iteration stopping can be used as the motion control network and deployed in the target robot in the above-mentioned practical application scenario. The second network is only used during training and does not need to be deployed in the target robot.

[0089] Figure 3 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 2 Optionally, in step S204, the simulated robot is controlled to execute the predicted action vector and obtain the immediate reward at the current time step, including: S301. Determine the joint movement command based on the predicted motion vector.

[0090] Similar to the application method, during the training process, at each current time step, joint motion commands can be obtained based on the predicted motion vector.

[0091] S302. Based on the joint movement commands, control the simulated robot to move and adjust its posture, and obtain the actual motion parameters after the action is executed.

[0092] Based on joint movement commands, the simulated robot can be controlled to move and adjust its posture to step into a new motion state and return to the actual motion parameters in the new motion state. The actual motion parameters can be understood as the instantaneous motion parameters in the new motion state after executing the joint movement commands. The actual motion parameters can be sensed and acquired by sensors on the simulated robot.

[0093] S303. Determine the immediate reward for the current time step based on the actual motion parameters, the composite reward function, and the reward weight.

[0094] The composite reward function includes at least the following: linear velocity tracking reward function, yaw rate tracking reward function, target attitude projection tracking reward function, terrain-adaptive gravity projection constraint reward function, world coordinate system slip velocity constraint reward function, attitude stability reward function, and asymmetric joint force response reward function.

[0095] In this embodiment, a composite reward function is used to construct the reward mechanism. The reward weight can refer to the weighted weight of each reward function in the composite reward function. When a reward function is used as a reward item, it is multiplied by a positive coefficient, so its reward weight is positive. In this case, the larger the reward value, the better the robot's behavior at the current time step conforms to the task objective and the better its performance. When a reward function is used as a penalty item, it is multiplied by a negative coefficient, so its reward weight is negative. Penalty items are usually used to punish unwanted behaviors. When used as a penalty item, the larger the reward value, the greater the degree of punishment the robot receives.

[0096] The immediate reward for the current time step is obtained by calculating the reward value of each reward function in the composite reward function based on the actual motion parameters, and finally by weighting and summing the reward values ​​based on the reward weights corresponding to each reward function to obtain the total reward.

[0097] Figure 4 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 3 Optionally, in step S303, the immediate reward for the current time step is determined based on the actual motion parameters, the composite reward function, and the reward weight, including: S401. Based on the first parameter corresponding to the linear velocity tracking reward function in the actual motion parameters, the linear velocity tracking reward function is used to determine the first reward value.

[0098] S402. Based on the second parameter corresponding to the yaw rate tracking reward function in the actual motion parameters, the yaw rate tracking reward function is used to determine the second reward value.

[0099] S403. Based on the third parameter corresponding to the attitude tracking reward function in the actual motion parameters, use the attitude tracking reward function to determine the third reward value.

[0100] S404. Based on the fourth parameter corresponding to the target attitude projection tracking reward function in the actual motion parameters, the target attitude projection tracking reward function is used to determine the fourth reward value.

[0101] S405. Based on the fifth parameter corresponding to the terrain-adaptive gravity projection constraint reward function in the actual motion parameters, the terrain-adaptive gravity projection constraint reward function is used to determine the fifth reward value.

[0102] S406. Based on the sixth parameter corresponding to the sliding velocity constraint reward function in the world coordinate system in the actual motion parameters, the sixth reward value is determined using the sliding velocity constraint reward function in the world coordinate system.

[0103] S407. Based on the seventh parameter corresponding to the attitude stability reward function in the actual motion parameters, use the attitude stability reward function to determine the seventh reward value.

[0104] S408. Based on the eighth parameter corresponding to the asymmetric joint force response reward function in the actual motion parameters, the asymmetric joint force response reward function is used to determine the eighth reward value.

[0105] S409. Determine the immediate reward for the current time step based on the first reward value, the second reward value, the third reward value, the fourth reward value, the fifth reward value, the sixth reward value, the seventh reward value, the eighth reward value, and the reward weight corresponding to each reward function.

[0106] It is worth noting that the calculation parameters of each reward function involve some parameters from the actual motion parameters. Therefore, by substituting the parameters corresponding to each reward function in the actual motion parameters into each reward function, the reward value of each reward function can be calculated. By weighting and summing the reward values ​​of each reward function using the reward weights corresponding to each reward function, the immediate reward of the current time step can be obtained.

[0107] Optionally, this embodiment exemplifies several core reward functions that constitute the composite reward function of this solution. In practical applications, the reward functions included in the composite reward function are not limited to those exemplified above, and more reward functions can be designed according to task requirements.

[0108] Figure 5 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 4 The first parameter includes: the linear velocity vector of the base in the world coordinate system; in step S401, based on the first parameter corresponding to the linear velocity tracking reward function in the actual motion parameters, the linear velocity tracking reward function is used to determine the first reward value, including: S501. Determine the actual forward linear velocity of the fuselage on the horizontal plane based on the unit projection direction of the fuselage forward on the horizontal plane and the linear velocity vector of the base in the world coordinate system.

[0109] The linear velocity tracking reward function is used to encourage the robot's actual horizontal movement speed (including forward linear velocity and lateral linear velocity) to reach as close as possible to the expected forward linear velocity and expected lateral linear velocity given in the movement command. This ensures that the robot's movement speed in the horizontal plane is as close as possible to the expected movement command, thereby achieving accurate horizontal velocity tracking.

[0110] The actual forward linear velocity of the machine body on the horizontal plane can be calculated using the following formula:

[0111] in, This indicates the actual forward linear velocity of the fuselage on the horizontal plane. This represents the linear velocity vector of the base in the world coordinate system, which can be obtained from the actual motion parameters. This indicates the direction of the fuselage's forward projection onto the horizontal plane.

[0112] S502. Determine the actual lateral linear velocity of the fuselage on the horizontal plane based on the unit projection direction of the fuselage on the horizontal plane and the linear velocity vector of the base in the world coordinate system.

[0113] The actual lateral linear velocity of the machine body on the horizontal plane can be calculated using the following formula:

[0114] in, This indicates the actual lateral linear velocity of the fuselage on the horizontal plane. This indicates the direction of the fuselage's side projection onto the horizontal plane.

[0115] in, and The following method can be used for calculation: Projection direction construction:

[0116]

[0117]

[0118] in, , This represents the lateral unit reference vector in the base coordinate system. Represents the unit gravity vector perpendicularly downwards in the world coordinate system. This represents the quaternion of the base attitude, and can also refer to the rotation matrix from the world coordinate system to the base coordinate system. This represents the projection of the world gravity vector onto the base coordinate system.

[0119] The above calculation formula can be used to calculate... and and will and Substituting these values ​​into the formulas for steps S501 and S502 respectively, we obtain... and .

[0120] S503. Determine the linear velocity error based on the actual forward linear velocity of the fuselage on the horizontal plane, the desired forward linear velocity, the actual lateral linear velocity of the fuselage on the horizontal plane, and the desired lateral linear velocity.

[0121] The linear velocity error can be calculated using the following formula:

[0122] in, Indicates linear velocity error. Indicates the desired forward linear velocity. Indicates the desired lateral linear velocity. and It can be obtained from the movement command.

[0123] S504. Input the linear velocity error and the tracking accuracy scaling factor into the linear velocity tracking reward function to calculate the first reward value.

[0124] The expression for the linear velocity tracking reward function is as follows:

[0125] in, This represents the first reward value, which is also the linear velocity tracking reward value. This represents the tracking accuracy scaling factor. This represents a tiny constant to prevent the reward from becoming zero.

[0126] Figure 6 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 5 The second parameter includes: the three-axis angular velocity vector of the fuselage base in the world coordinate system; optionally, in step S402, based on the second parameter corresponding to the yaw rate tracking reward function in the actual motion parameters, the yaw rate tracking reward function is used to determine the second reward value, including: S601. Based on the three-axis angular velocity vectors of the fuselage base in the world coordinate system and the unit projection of the gravity vector in the fuselage base coordinate system, determine the yaw angular velocity components of the fuselage base around the world gravity axis.

[0127] The following formula is used to calculate the yaw rate component of the computer's base about the world gravity axis:

[0128] in, This represents the yaw rate component of the fuselage base about the world's gravity axis. This represents the three-axis angular velocity vector of the base in the world coordinate system, which can be obtained from the actual motion parameters. This represents the unit projection of the gravity vector in the fuselage base coordinate system.

[0129] S602. Determine the yaw rate error based on the yaw rate component of the fuselage base around the world gravity axis and the desired yaw rate.

[0130] Yaw rate error is defined as:

[0131] in, Indicates the yaw rate error. This represents the desired yaw rate, which can be obtained directly from the movement command.

[0132] S603. Input the yaw rate error and the tracking accuracy scaling factor into the yaw rate tracking reward function to calculate the second reward value.

[0133] The expression for the yaw rate tracking reward function is:

[0134] in, This represents the second reward value, which is the yaw rate tracking reward value. This has been explained in step S504.

[0135] By designing a reward function that tracks yaw rate, the robot's actual yaw rate can be encouraged to accurately track the desired yaw rate given in the movement command, enabling the robot to accurately execute the desired yaw rotation.

[0136] Figure 7 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 6 The third parameter includes: the actual roll angle and the actual pitch angle of the fuselage; the attitude tracking reward function includes: the roll angle reward function and the pitch angle reward function; in step S403, based on the third parameter corresponding to the attitude tracking reward function in the actual motion parameters, the attitude tracking reward function is used to determine the third reward value, including: S701. Input the actual roll angle and the desired roll angle of the fuselage into the roll angle bonus function to determine the roll angle bonus value.

[0137] The roll angle reward function expression is:

[0138] in, This represents the roll angle bonus value. The desired roll angle can be obtained directly from the attitude command. The actual roll angle of the fuselage can be obtained from the actual motion parameters.

[0139] S702. Input the actual pitch angle and the desired pitch angle of the fuselage into the pitch angle reward function to determine the pitch angle reward value.

[0140] The expression for the pitch angle reward function is as follows:

[0141] in, This represents the pitch angle bonus value. This represents the desired pitch angle, which is obtained directly from the attitude command. This indicates the actual pitch angle of the fuselage, obtained from actual motion parameters.

[0142] S703. Determine the third bonus value based on the roll angle bonus value and the pitch angle bonus value.

[0143] A third bonus value can be obtained by adding the roll angle bonus value and the pitch angle bonus value.

[0144] By designing a posture tracking reward function, the robot's actual tilt posture can be encouraged to accurately track the desired posture command, enabling the robot to achieve active posture tracking.

[0145] In some embodiments, the calculation of the linear velocity tracking reward, yaw rate tracking reward, and attitude tracking reward is performed in complete parallel execution without temporal mutual exclusion, thereby enabling the robot to respond to movement commands and attitude commands simultaneously, and to adjust its attitude while moving.

[0146] Figure 8 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 7 The fourth parameter includes: the actual gravity vector projection in the fuselage base coordinate system; in step S404, based on the fourth parameter corresponding to the target attitude projection tracking reward function in the actual motion parameters, the target attitude projection tracking reward function is used to determine the fourth reward value, including: S801. Determine the desired gravity direction vector in the fuselage base coordinate system based on the desired roll angle and desired pitch angle.

[0147] The desired gravity direction vector in the fuselage base coordinate system can be calculated based on the desired roll angle and desired pitch angle contained in the attitude command. .

[0148] S802. Input the actual gravity vector projection in the fuselage base coordinate system and the desired gravity direction vector in the fuselage base coordinate system into the target attitude projection tracking reward function to calculate the fourth reward value.

[0149] The expression for the target pose projection tracking reward function is as follows:

[0150] in, The actual gravity vector projection in the fuselage base coordinate system can be obtained from the actual motion parameters. This represents the fourth reward value, which is the target pose projection tracking reward value.

[0151] By designing a reward function for target posture projection tracking, the conflict between traditional forced horizontal constraints and active posture adjustment can be resolved. This encourages the robot to actively tilt to the desired posture command, rather than passively maintaining horizontality.

[0152] When the desired attitude (roll and pitch angles) is non-zero, the robot can be guided to tilt actively to the desired angle; when the desired attitude is zero, it is equivalent to a traditional horizontal constraint, which can keep the robot horizontal, so that the robot can actively perform attitude adjustment and automatically maintain horizontal when no attitude adjustment is needed.

[0153] Figure 9 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 8 The fifth parameter includes: the actual gravity vector projection in the fuselage base coordinate system; in step S405, based on the fifth parameter corresponding to the terrain-adaptive gravity projection constraint reward function in the actual motion parameters, the terrain-adaptive gravity projection constraint reward function is used to determine the fifth reward value, including: S901. Based on the actual gravity vector projection in the fuselage base coordinate system, determine the component of the actual gravity vector projection on the horizontal plane of the fuselage base coordinate system.

[0154] It can be based on the actual gravity vector projection in the fuselage base coordinate system. Determine the components of the actual gravity vector projection onto the horizontal plane of the fuselage base coordinate system. .

[0155] in, = , This represents the component of the actual gravity vector along the x-axis (forward axis) of the base. This represents the component of the actual gravity vector along the y-axis (lateral axis) of the base.

[0156] S902. Determine the terrain adaptive coefficient based on the terrain information.

[0157] Terrain adaptive coefficient is expressed as On flat terrain A value of 1.0 can be chosen to maintain strong horizontal constraints on rugged terrain such as steps and gravel. A value of 0.2 can be used to actively relax constraints, allowing the robot body to adapt to terrain undulations, thereby improving the robot's posture compliance on complex terrains.

[0158] S903. The components of the actual gravity vector projected onto the horizontal plane of the fuselage base coordinate system and the terrain adaptation coefficient are input into the terrain-adaptive gravity projection constraint reward function to calculate the fifth reward value.

[0159] The expression for the terrain-adaptive gravity projection constraint reward function is:

[0160] in, This represents the fifth reward value, which is the gravity projection constraint reward value for terrain adaptation.

[0161] The terrain-adaptive gravity projection constraint reward function serves as a penalty term, and when calculating the immediate reward later, a negative coefficient will be multiplied by the fifth reward value.

[0162] By designing a terrain-adaptive gravity projection constraint reward function, the constraint strength on the robot's horizontal attitude can be adaptively adjusted according to the terrain complexity. This maintains high-precision horizontal control on flat terrain and actively relaxes the horizontal requirements on rugged terrain, allowing the robot to adapt to terrain undulations. This improves the robot's compliance and stability on uneven surfaces. It overcomes the limitations of traditional control methods, which often fail to adapt to terrain changes, leading to stiff movement or frequent falls on rugged terrain.

[0163] Figure 10 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 9 The sixth parameter includes: the instantaneous linear velocity of each foot of the simulated robot in the world coordinate system; in step S406, based on the sixth parameter corresponding to the sliding velocity constraint reward function in the world coordinate system in the actual motion parameters, the sixth reward value is determined using the sliding velocity constraint reward function in the world coordinate system, including: S1001. Determine the slip penalty coefficient for each foot based on the ground contact state of the simulated robot.

[0164] The slip velocity constraint reward function in the world coordinate system only constrains the actual slip velocity of the foot touching the ground in the world coordinate system.

[0165] Define the ground contact state of the i-th foot:

[0166] Based on the above definition, it can be determined that when the foot is in the ground contact phase, the corresponding slip penalty coefficient is 1, and when the foot is in the swing phase, the corresponding slip penalty coefficient is 0.

[0167] S1002. Input the instantaneous linear velocity of each foot end in the world coordinate system and the corresponding slip penalty coefficient of each foot end into the slip velocity constraint reward function of the world coordinate system to calculate the sixth reward value.

[0168] The slip velocity constraint reward function in the world coordinate system is defined as follows:

[0169] in, This represents the sixth reward value, which is the slip speed constraint reward value. The instantaneous linear velocity of foot i in the world coordinate system can be obtained from the actual motion parameters. This represents the slip penalty coefficient corresponding to foot i. It is the Euclidean norm.

[0170] The slip velocity constraint reward function in the world coordinate system is also used as a penalty term. When calculating the immediate reward later, the sixth reward value also needs to be multiplied by a negative coefficient.

[0171] In the calculation formula of the sliding velocity constraint reward function in the world coordinate system, the sum of the squares of the linear velocities is taken, which physically represents the instantaneous dissipation of sliding kinetic energy. This occurs when the foot touching the ground is in motion in the world coordinate system. A negative value can guide the motion control network to suppress robot slippage. Since it does not involve the movement of the base coordinate system, this penalty term allows the feet to naturally translate as the body tilts, avoiding jamming.

[0172] By designing a reward function that constrains the sliding velocity in the world coordinate system, the robot's foot slippage during the ground contact phase can be effectively suppressed, while allowing the foot to naturally translate along with the body's tilt. This avoids jamming caused by excessive constraints while ensuring support stability. Specifically, only the foot in the ground contact phase is penalized; the foot in the swing phase should move naturally and is not penalized, thus avoiding interference with normal leg-stepping movements.

[0173] Traditional methods constrain foot velocity in the base coordinate system, requiring the ground-contacting foot to remain stationary relative to the body. This leads to the foot being forcibly locked during active tilting of the robot (such as turning or walking on a slope), resulting in abnormal slippage or jamming and disrupting motion compliance. In contrast, the slip velocity constraint reward function designed in this scheme in the world coordinate system only penalizes actual slippage (foot sliding relative to the ground) without penalizing natural translations caused by changes in body posture, allowing the robot to adapt to posture adjustments more naturally.

[0174] Figure 11 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 10The seventh parameter includes: the actual pitch rate of the fuselage, the actual roll rate of the fuselage, the actual horizontal velocity of the fuselage, and the actual landing point position of each swing phase foot; in step S407, based on the seventh parameter corresponding to the attitude stability reward function in the actual motion parameters, the attitude stability reward function is used to determine the seventh reward value, including: S1101. Determine the body attitude offset based on the vertical height of the simulated robot's center of mass relative to the ground, the actual pitch angular velocity of the robot body, the actual roll angular velocity of the robot body, and the duration of a single gait cycle.

[0175] The fuselage attitude offset is expressed as:

[0176] in,

[0177] in, This represents the compensation amount for the foot's landing point in the x-axis (forward) direction of the base coordinate system. This represents the compensation amount for the foot's landing point in the y-axis (lateral) direction of the base coordinate system. This represents the vertical height of the simulated robot's center of mass relative to the ground it contacts. This indicates the actual pitch rate of the fuselage. This indicates the actual roll rate of the fuselage. This indicates the duration of a single gait cycle. The negative sign indicates that the compensation direction is opposite to the angular velocity direction; when the robot pitches forward (…), this sign indicates that the angular velocity direction is opposite to the compensation direction. When >0), the center of mass tilts forward, and the landing point should be compensated for backward ( ). <0, to prevent forward leaning. (Riding) The significance is that by integrating the displacement increment caused by the angular velocity over one gait period, the offset of the landing position can be approximated.

[0178] S1102. Determine the linear velocity error based on the actual horizontal velocity and the desired horizontal velocity of the fuselage.

[0179] In the absence of compensation, to respond simultaneously to both movement and attitude commands, a velocity feedforward term can be introduced, and the linear velocity error can be expressed as:

[0180] in, Indicates linear velocity error. Indicates the desired horizontal speed. This indicates the actual horizontal speed of the aircraft.

[0181] S1103. Determine the feedforward of the actual speed based on the actual horizontal speed of the fuselage.

[0182] Feedforward of actual speed:

[0183] That is, the actual horizontal speed of the fuselage is used as a feedforward of the actual speed.

[0184] S1104. Determine the actual foot landing point position without compensation based on the linear velocity error, the feedforward of the actual velocity, and the duration of a single gait cycle.

[0185] The expression for the actual landing point of the foot without compensation is:

[0186] in, This indicates the actual landing point of the foot without compensation.

[0187] S1105. Based on the actual foot landing point position without compensation, the fuselage attitude offset, the gait phase activation factor, and the expected foot landing point position without compensation, determine the expected foot landing point position during the swing phase.

[0188] The formula for calculating the desired landing point position of the swing phase foot is as follows:

[0189] in, This indicates the desired landing point position of the swing phase foot end. This represents the expected landing point of the foot without compensation, that is, the expected landing point of each foot relative to the xy plane of the base coordinate system. Indicates the gait phase activation factor. The gait phase is calculated independently for each foot.

[0190] The gait phase activation factor is introduced to prevent leg impact caused by sudden changes in foot landing point. In this embodiment, the gait phase activation factor gradually adds the compensation amount to the target position only in the middle and late stages of the swing phase.

[0191] Define gait phase ,in Corresponding to the start of the oscillation phase, Corresponding to the instant of ground contact. First, the phase is scaled:

[0192] Define the S-Curve function:

[0193]

[0194] The gait phase activation factor is then characterized as:

[0195] in, As a swing phase indicator, this design allows the foot landing compensation to increase slowly in the early stage of the swing phase to avoid starting impact, reach its maximum in the middle stage to accurately guide the foot, and gradually return to zero in the late stage to prepare for ground contact.

[0196] S1106. Determine the total landing point error based on the actual landing point position of each swing phase foot end and the expected landing point position of each swing phase foot end.

[0197] The expression for the error of the landing point of the swing phase foot is defined as follows:

[0198] in, This indicates the error in the landing point of the swing phase foot i. This indicates the actual landing point of the swing phase foot i, specifically taken as the x and y coordinates of the swing phase foot i. This represents the desired landing point position of the swing phase foot end i, specifically the desired x and y coordinates of the swing phase foot end i, which is also the desired horizontal landing point position of the swing phase foot end i.

[0199] The total error of the landing point of the swing phase is defined as:

[0200] in, This represents the total error of the landing point, also known as the total error of the landing point of the oscillating phase.

[0201] S1107. Input the total error of the landing point and the terrain adaptive coefficient into the attitude stability reward function to calculate the seventh reward value.

[0202] To enhance terrain adaptability, the tracking weight is reduced on uneven terrain. The expression for the attitude stability reward function can be defined as follows:

[0203] in, This represents the seventh reward value, also known as the attitude stability reward value. Indicates the terrain adaptation coefficient. , This is used to ensure numerical stability. Using the square root to compress large errors in the gradient avoids sacrificing posture stability by overemphasizing foot accuracy in early training.

[0204] By designing a posture stability reward function, the robot can be guided to actively correct the desired foot placement point during dynamic posture adjustment and variable speed movement to guide precise foot tracking. At the same time, the tracking weight can be adaptively adjusted according to the ruggedness of the terrain, thus achieving a unified high-precision foot placement and terrain-adaptive compliant control when the wheeled robot moves and adjusts its posture.

[0205] When a robot tilts or changes speed during movement, its center of mass will shift horizontally due to angular velocity and acceleration. Without compensation, the foot landing point will lag behind the desired support point of the body, leading to unstable support, easy tipping over or forward leaning, and the foot failing to reach the ideal position. By using attitude feedforward combined with velocity feedforward for foot landing point compensation design, the nominal landing point, that is, the actual landing point of the foot without compensation, can be corrected to obtain the desired landing point position of the foot during the dynamic swing phase.

[0206] Figure 12 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 10 The eighth parameter includes: the supporting force of the left fuselage leg and the supporting force of the right fuselage leg; in step S408, based on the eighth parameter corresponding to the asymmetric joint force response reward function in the actual motion parameters, the asymmetric joint force response reward function is used to determine the eighth reward value, including: S1201. Determine the actual asymmetry of the contact force between the left and right legs based on the supporting force of the left and right legs of the fuselage.

[0207] The actual asymmetry of contact force between the left and right legs can be calculated using the following formula:

[0208] in, This indicates the supporting force of the left leg of the fuselage. This indicates the supporting force of the right leg of the fuselage. This indicates the degree of asymmetry in the actual contact force between the left and right legs.

[0209] S1202. Input the actual asymmetry of the left and right leg contact forces and the expected asymmetry of the left and right leg contact forces into the asymmetric joint force response reward function to calculate the eighth reward value.

[0210] The desired asymmetry of contact force between the left and right legs is calculated using the following formula:

[0211] in, Indicates the expected roll angle. This indicates the desired asymmetry in the contact force between the left and right legs.

[0212] The expression for the asymmetric joint force response reward function is as follows:

[0213] in, The scaling factor represents the sensitivity of the reward function to force asymmetry errors.

[0214] By designing an asymmetric joint force response reward function, the motion control network can be guided to actively transfer the load to the lower leg when the robot tilts, thereby enhancing support stability. For example, when the robot body tilts actively or the terrain causes it to tilt, the motion control network can be prompted to actively transfer the load to the downhill side or the tilted leg, preventing sideslip or tipping due to improper load distribution.

[0215] Figure 13 Schematic flowchart of the training method for the motion control network provided in the embodiments of this application Figure 10 Second; optionally, in step S409, the method for determining the reward weight corresponding to each reward function includes: S1301. Determine the speed priority factor based on the desired horizontal speed vector.

[0216] This embodiment also provides a task-aware reward weight dynamic arbitration mechanism. In order to avoid conflicts between movement commands and attitude commands in extreme cases, the reward weight of each reward function can be dynamically adjusted based on the priority arbitration mechanism of the current command speed.

[0217] Optionally, a speed priority factor can be calculated based on the command speed norm:

[0218] in, Indicates speed priority factor, This represents the desired horizontal velocity vector.

[0219] S1302. Determine the attitude priority factor based on the speed priority factor.

[0220] Attitude priority factor .

[0221] S1303. Determine the reward weight corresponding to each reward function based on the reward type, speed priority factor, and attitude priority factor of each reward function.

[0222] Among the above reward functions, the reward weight of the movement-related reward function can be multiplied by... The final reward weight is obtained from the posture-related reward function, and its reward weight can be multiplied by... The final reward weight is obtained.

[0223] Therefore, when calculating the immediate reward for the current step, the above reward values ​​can be weighted and summed. When the reward function is used as a penalty term, it is multiplied by a negative coefficient. Simultaneously, the reward values ​​related to movement are multiplied by... The various reward values ​​related to posture are multiplied by Receive an instant reward.

[0224] When the robot is moving at high speed, it automatically focuses on speed tracking and appropriately relaxes the requirements for posture accuracy; when moving at low speed or stationary, it prioritizes ensuring accurate posture adjustment.

[0225] In addition to the reward functions specifically demonstrated above, in practical applications, reward functions may also include penalties for rate of change of motion, joint torque, collision, soft limit of joint position, foot contact force threshold, and static stability L1 norm constraints, in order to ensure the smoothness of policy output, energy rationality, and hardware safety.

[0226] To narrow the gap between simulated and real-world dynamics, this approach implements deep-domain randomization in the simulation platform: Wheel joints: Apply additional viscous friction coefficient (0.2~0.8 N·m·s / rad), Coulomb friction (0.1~0.5 N·m) and actuator delay of 5~15 ms.

[0227] Leg joints: Apply small viscous friction (0.05~0.2 N·m·s / rad) and a delay of 2~5 ms.

[0228] Enhanced terrain perception: By simulating a depth camera scanning a matrix, a local elevation map centered on the aircraft fuselage is generated as privileged observation information, enabling the strategy to predict features such as steps and slopes.

[0229] Ground physical parameters randomization: In each training round, the ground friction coefficient (0.3~1.2), the coefficient of restitution (0.2~0.8), and the rolling friction coefficient are randomly sampled, and the foot contact stiffness is calculated accordingly for attitude adaptive relaxation.

[0230] In this embodiment, a comprehensive test scenario including flat ground, slopes, steps, and gravel ground is constructed in a simulation environment to quantitatively evaluate the trained motion control network. (1) Verification of independent attitude response capability during movement While executing a 1.5 m / s straight-line walking command, the robot receives a 0.5 Hz, ±25° sinusoidal roll angle command. The roll angle tracking error RMSE is less than 1.2°, achieving parallel and independent speed tracking. Regarding pitch, the robot can actively adjust to any target angle within ±15° during movement, with a steady-state tracking error of less than 1°. The decoupled architecture of movement and attitude commands is fully effective, allowing the robot to continuously and smoothly tilt or pitch its body while moving at high speed, achieving the capability of "walking while simultaneously looking / turning."

[0231] (2) Gait stability and compliance during dynamic posture adjustment When performing large pitching or lateral movements while walking, compared with the traditional static foot restraint scheme, this scheme reduces the peak knee joint torque by about 27%, reduces the foot slip distance by 65%, and makes gait transitions smooth and without interruption. During dynamic switching of roll angle ±20° and pitch angle ±15°, the aircraft attitude transitions smoothly without obvious overshoot or oscillation.

[0232] (3) Terrain-adaptive safety during walking posture adjustment On continuous uphill and downhill slopes and gravel roads, the robot maintains a normal walking speed of over 1.0 m / s, and can adaptively adjust its body conformity angle within a range of ±20° roll and ±15° pitch according to terrain undulations. Compared with traditional solutions, the rollover rate is reduced by 80%, and no imbalance occurs due to conflicts between posture adjustment commands and terrain.

[0233] (4) Transfer of walking posture adjustment ability from simulation to reality The dynamic control network trained in simulation was directly deployed to the physical wheeled robot, and the posture adjustment capability during walking could be reproduced without any fine-tuning. In the real environment, the tracking error of roll and pitch commands increased by only about 15% compared with the simulation, and the tracking of normal walking speed and gait coordination remained good, verifying the effectiveness of the heterogeneous domain randomization design of the wheeled robot.

[0234] This application also provides a robot, which can be a wheeled robot, and the robot is equipped with the motion control network trained as described above; the robot is used to execute the motion control method steps described above.

[0235] In summary, the robot motion control method provided in this application includes: receiving movement commands and posture commands for a target robot, and acquiring proprioceptive observation data of the target robot; inputting the movement commands and posture commands as independent parallel command vectors on the time axis into a pre-trained motion control network, and inputting the proprioceptive observation data into the motion control network, and predicting a coordinated motion vector between movement execution and posture adjustment by the motion control network; determining the target motion command for the target robot based on the coordinated motion vector; and controlling the target robot to move and adjust its posture according to the target motion command. This embodiment introduces two parallel and independent command input channels into the input layer of the motion control network, allowing movement commands and posture commands to be input as independent parallel command vectors on the time axis. Combined with the input proprioceptive observation data, the coordinated motion vector between movement execution and posture adjustment is predicted through an internal attention mechanism. Therefore, based on the coordinated motion vector, collaborative execution of movement execution and posture adjustment can be achieved, improving the robot's motion flexibility and the smoothness of task execution.

[0236] In particular, training the motion control network by introducing a composite reward function can effectively improve the control accuracy of the motion control network.

[0237] By designing a linear velocity tracking reward function, the robot's actual horizontal movement speed (including forward and lateral linear velocity) can be encouraged to reach the expected forward and lateral linear velocities given in the movement command as much as possible. This ensures that the robot's movement speed in the horizontal plane is as close as possible to the expected movement command, thereby achieving accurate horizontal velocity tracking.

[0238] By designing a reward function that tracks yaw rate, the robot's actual yaw rate can be encouraged to accurately track the desired yaw rate given in the movement command, enabling the robot to accurately execute the desired yaw rotation.

[0239] By designing a posture tracking reward function, the robot's actual tilt posture can be encouraged to accurately track the desired posture command, enabling the robot to achieve active posture tracking.

[0240] By designing a reward function for target posture projection tracking, the conflict between traditional forced horizontal constraints and active posture adjustment can be resolved. This encourages the robot to actively tilt to the desired posture command, rather than passively maintaining horizontality.

[0241] By designing a terrain-adaptive gravity projection constraint reward function, the constraint strength on the robot's horizontal attitude can be adaptively adjusted according to the terrain complexity. This maintains high-precision horizontal control on flat terrain and actively relaxes the horizontal requirements on rugged terrain, allowing the robot to adapt to terrain undulations. This improves the robot's compliance and stability on uneven surfaces. It overcomes the limitations of traditional control methods, which often fail to adapt to terrain changes, leading to stiff movement or frequent falls on rugged terrain.

[0242] By designing a reward function that constrains the sliding velocity in the world coordinate system, the robot's foot slippage during the ground contact phase can be effectively suppressed, while allowing the foot to naturally translate along with the body's tilt. This avoids jamming caused by excessive constraints while ensuring support stability. Specifically, only the foot in the ground contact phase is penalized; the foot in the swing phase should move naturally and is not penalized, thus avoiding interference with normal leg-stepping movements.

[0243] By designing a posture stability reward function, the robot can be guided to actively correct the desired foot placement point during dynamic posture adjustment and variable speed movement to guide precise foot tracking. At the same time, the tracking weight can be adaptively adjusted according to the ruggedness of the terrain, thus achieving a unified high-precision foot placement and terrain-adaptive compliant control when the wheeled robot moves and adjusts its posture.

[0244] By designing an asymmetric joint force response reward function, the motion control network can be guided to actively transfer the load to the lower leg when the robot tilts, thereby enhancing support stability. For example, when the robot body tilts actively or the terrain causes it to tilt, the motion control network can be prompted to actively transfer the load to the downhill side or the tilted leg, preventing sideslip or tipping due to improper load distribution.

[0245] Based on the structural design of wheel-legged robots, this solution correspondingly designs various reward functions, which can control wheel-legged robots to adjust their posture flexibly like legged robots and move efficiently like wheeled robots, while exhibiting strong robustness on complex terrain.

[0246] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The device can be a computing device with data processing function. The electronic device can be a computing device deployed in a robot and can be used to execute the above-mentioned robot motion control method. The electronic device can also be independent of the robot and used to execute the methods related to training the motion control network in the above-mentioned robot motion control method.

[0247] The electronic device includes a processor 801, a storage medium 802, and a bus 803. The storage medium 802 stores program instructions that can be executed by the processor 801. When the electronic device is running, the processor 801 communicates with the storage medium 802 through the bus 803. The processor 801 executes the program instructions to implement the motion control method of the robot as described in the embodiment.

[0248] The storage medium 802 stores program code, which, when executed by the processor 801, causes the processor 801 to perform various steps in the motion control method of the robot according to various exemplary embodiments of this application as described in the "Exemplary Methods" section above.

[0249] The processor 801 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0250] Storage medium 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The storage medium can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type storage medium, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage medium, magnetic disk, optical disk, etc. The storage medium is any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, storage medium 802 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0251] Optionally, this application also provides a program product, such as a computer-readable storage medium, including a program that, when executed by a processor, performs the above-described method embodiments.

[0252] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0253] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0254] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0255] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A motion control method for a robot, characterized in that, The method includes: Receive movement and posture commands for the target robot and acquire the target robot's body perception observation data; The movement command and the posture command are input as independent parallel command vectors on the time axis into a pre-trained motion control network, and the proprioceptive observation data is input into the motion control network. The motion control network then predicts the coordinated action vector between movement execution and posture adjustment. Based on the coordinated motion vector, the target motion command of the target robot is determined; The target robot is controlled to move and adjust its posture according to the target motion command.

2. The method according to claim 1, characterized in that, The training method for the motion control network includes the following steps: A simulation environment is constructed, and a first network and a second network are initialized. The first network is used to output action vectors based on the body perception observation data, movement commands, and posture commands, and the second network is used to output state values ​​based on the body perception observation data, privileged observation data, movement commands, and posture commands. Step A: Obtain the sample ontology perception observation data, sample privileged observation data, and the target movement command and target attitude command corresponding to the current time step; Step B: Input the sample ontology perception observation data, target movement command, and target posture command into the current first network to obtain the predicted action vector; Step C: Control the simulated robot to execute the predicted action vector, and obtain the instantaneous reward at the current time step and the sample ontological perception observation data for the next time step; Step D: Input the sample ontology perception observation data, target movement command, target attitude command, and sample privileged observation data into the current second network to obtain the first prediction value at the current time step; Step E: Input the sample ontology perception observation data of the next time step, the sample privileged observation data of the next time step, and the target movement command and target attitude command corresponding to the next time step into the current second network to obtain the second prediction value of the next time step; Step F: Determine the actual state value of the current time step based on the instant reward and the second predicted value, and update the network parameters of the second network based on the actual state value and the first predicted value; Step G: Determine the advantage value based on the actual state value and the first predicted value, and update the network parameters of the first network using the policy gradient algorithm based on the advantage value; the advantage value is used to characterize the superiority of the action performed at the current time step based on the predicted action vector compared to the average level of the current policy. Step H: Repeat steps A to G until the iteration stop condition is met, and use the first network at the time of iteration stop as the motion control network.

3. The method according to claim 2, characterized in that, The controlled simulation robot executes the predicted action vector and obtains the immediate reward at the current time step, including: Based on the predicted motion vector, determine the joint movement command; Based on the joint movement commands, the simulated robot is controlled to move and adjust its posture, and the actual motion parameters after the action is executed are obtained; The instantaneous reward for the current time step is determined based on the actual motion parameters, the composite reward function, and the reward weight. The composite reward function includes at least the following: linear velocity tracking reward function, yaw rate tracking reward function, attitude tracking reward function, target attitude projection tracking reward function, terrain-adaptive gravity projection constraint reward function, world coordinate system slip velocity constraint reward function, attitude stability reward function, and asymmetric joint force response reward function.

4. The method according to claim 3, characterized in that, Based on the actual motion parameters, the composite reward function, and the reward weight, the immediate reward for the current time step is determined, including: Based on the first parameter corresponding to the linear velocity tracking reward function in the actual motion parameters, the first reward value is determined using the linear velocity tracking reward function; Based on the second parameter corresponding to the yaw rate tracking reward function in the actual motion parameters, the second reward value is determined using the yaw rate tracking reward function; Based on the third parameter corresponding to the attitude tracking reward function in the actual motion parameters, the attitude tracking reward function is used to determine the third reward value; Based on the fourth parameter corresponding to the target attitude projection tracking reward function in the actual motion parameters, the fourth reward value is determined using the target attitude projection tracking reward function; Based on the fifth parameter corresponding to the terrain-adaptive gravity projection constraint reward function in the actual motion parameters, the fifth reward value is determined using the terrain-adaptive gravity projection constraint reward function. Based on the sixth parameter corresponding to the sliding velocity constraint reward function of the world coordinate system in the actual motion parameters, the sixth reward value is determined using the sliding velocity constraint reward function of the world coordinate system. Based on the seventh parameter corresponding to the attitude stability reward function in the actual motion parameters, the seventh reward value is determined using the attitude stability reward function; Based on the eighth parameter corresponding to the asymmetric joint force response reward function in the actual motion parameters, the eighth reward value is determined using the asymmetric joint force response reward function. The immediate reward for the current time step is determined based on the first reward value, the second reward value, the third reward value, the fourth reward value, the fifth reward value, the sixth reward value, the seventh reward value, the eighth reward value, and the reward weight corresponding to each reward function.

5. The method according to claim 4, characterized in that, The first parameter includes: the linear velocity vector of the base in the world coordinate system; the step of determining the first reward value using the linear velocity tracking reward function based on the first parameter corresponding to the linear velocity tracking reward function in the actual motion parameters includes: The actual forward linear velocity of the fuselage on the horizontal plane is determined based on the unit projection direction of the fuselage forward on the horizontal plane and the linear velocity vector of the base in the world coordinate system. The actual lateral linear velocity of the fuselage on the horizontal plane is determined based on the unit projection direction of the fuselage on the horizontal plane and the linear velocity vector of the base in the world coordinate system. The linear velocity error is determined based on the actual forward linear velocity of the fuselage on the horizontal plane, the desired forward linear velocity, the actual lateral linear velocity of the fuselage on the horizontal plane, and the desired lateral linear velocity. The linear velocity error and the tracking accuracy scaling factor are input into the linear velocity tracking reward function to calculate the first reward value.

6. The method according to claim 4, characterized in that, The second parameter includes: the three-axis angular velocity vector of the fuselage base in the world coordinate system; the step of determining the second reward value using the yaw rate tracking reward function based on the second parameter corresponding to the yaw rate tracking reward function in the actual motion parameters includes: Based on the three-axis angular velocity vectors of the fuselage base in the world coordinate system and the unit projection of the gravity vector in the fuselage base coordinate system, determine the yaw angular velocity components of the fuselage base around the world gravity axis. The yaw rate error is determined based on the yaw rate component of the fuselage base around the world gravity axis and the desired yaw rate. The yaw rate error and the tracking accuracy scaling factor are input into the yaw rate tracking reward function to calculate the second reward value.

7. The method according to claim 4, characterized in that, The third parameter includes: the actual roll angle and the actual pitch angle of the fuselage; the attitude tracking reward function includes: a roll angle reward function and a pitch angle reward function; the step of determining the third reward value based on the third parameter corresponding to the attitude tracking reward function in the actual motion parameters, using the attitude tracking reward function, includes: Input the actual roll angle and the desired roll angle of the fuselage into the roll angle reward function to determine the roll angle reward value; Input the actual pitch angle and the desired pitch angle of the fuselage into the pitch angle reward function to determine the pitch angle reward value; The third reward value is determined based on the roll angle reward value and the pitch angle reward value.

8. The method according to claim 4, characterized in that, The fourth parameter includes: the actual gravity vector projection in the fuselage base coordinate system; the step of determining the fourth reward value using the target attitude projection tracking reward function based on the fourth parameter corresponding to the target attitude projection tracking reward function in the actual motion parameters includes: Determine the desired gravity direction vector in the fuselage base coordinate system based on the desired roll angle and desired pitch angle; The actual gravity vector projection in the fuselage base coordinate system and the desired gravity direction vector in the fuselage base coordinate system are input into the target attitude projection tracking reward function to calculate the fourth reward value.

9. The method according to claim 4, characterized in that, The fifth parameter includes: the actual gravity vector projection in the fuselage base coordinate system; the step of determining the fifth reward value based on the fifth parameter corresponding to the terrain-adaptive gravity projection constraint reward function in the actual motion parameters, using the terrain-adaptive gravity projection constraint reward function, includes: Based on the actual gravity vector projection in the fuselage base coordinate system, determine the components of the actual gravity vector projection on the horizontal plane of the fuselage base coordinate system; Determine the terrain adaptation coefficient based on the terrain information; The component of the actual gravity vector projected onto the horizontal plane of the fuselage base coordinate system and the terrain adaptation coefficient are input into the terrain-adaptive gravity projection constraint reward function to calculate the fifth reward value.

10. The method according to claim 4, characterized in that, The sixth parameter includes: the instantaneous linear velocity of each foot of the simulated robot in the world coordinate system; the determination of the sixth reward value based on the sixth parameter corresponding to the sliding velocity constraint reward function in the world coordinate system in the actual motion parameters, using the sliding velocity constraint reward function in the world coordinate system, includes: Based on the ground contact state of each foot of the simulated robot, determine the corresponding slip penalty coefficient for each foot. The instantaneous linear velocity of each foot in the world coordinate system and the corresponding slip penalty coefficient of each foot are input into the slip velocity constraint reward function of the world coordinate system to calculate the sixth reward value.

11. The method according to claim 4, characterized in that, The seventh parameter includes: the actual pitch rate of the fuselage, the actual roll rate of the fuselage, the actual horizontal velocity of the fuselage, and the actual landing point position of each swing phase foot; the step of determining the seventh reward value based on the seventh parameter corresponding to the attitude stability reward function in the actual motion parameters, using the attitude stability reward function, includes: The body attitude offset is determined based on the vertical height of the robot's center of mass relative to the ground, the actual pitch rate of the robot body, the actual roll rate of the robot body, and the duration of a single gait cycle. The linear velocity error is determined based on the actual horizontal velocity and the desired horizontal velocity of the fuselage. Determine the feedforward of the actual speed based on the actual horizontal speed of the fuselage; Based on the linear velocity error, the feedforward of the actual velocity, and the duration of a single gait cycle, determine the actual foot landing point position without compensation. Based on the actual foot landing point position without compensation, the fuselage attitude offset, the gait phase activation factor, and the expected foot landing point position without compensation, the expected foot landing point position during the swing phase is determined. The total landing point error is determined based on the actual landing point position of each swing phase foot end and the expected landing point position of the swing phase foot end. The total error of the landing point and the terrain adaptation coefficient are input into the attitude stability reward function to calculate the seventh reward value.

12. The method according to claim 4, characterized in that, The eighth parameter includes: the supporting force of the left leg of the fuselage and the supporting force of the right leg of the fuselage; the process of determining the eighth reward value based on the eighth parameter corresponding to the asymmetric joint force response reward function in the actual motion parameters, using the asymmetric joint force response reward function, includes: The actual asymmetry of the contact force between the left and right fuselage legs is determined based on the supporting force of the left and right fuselage legs. The actual asymmetry of the left and right leg contact forces and the expected asymmetry of the left and right leg contact forces are input into the asymmetric joint force response reward function to calculate the eighth reward value.

13. The method according to claim 4, characterized in that, The methods for determining the reward weights corresponding to each reward function include: Determine the speed priority factor based on the desired horizontal speed vector; Determine the attitude priority factor based on the speed priority factor; The reward weight corresponding to each reward function is determined based on the reward type of each reward function, the speed priority factor, and the attitude priority factor.

14. A robot, characterized in that, The robot is a wheel-legged robot, and a motion control network is deployed on the robot. The robot is used to perform the motion control method for the robot as described in claim 1.

Images