Robot high jump platform motion control model training method and device

Abstract

This disclosure provides a method and apparatus for training a robot high jump motion control model, relating to the fields of sensors and robotics. The method involves acquiring current state data of a virtual robot, including observation data from an inertial measurement device, adding noise to the observation data, and training the robot high jump motion control model in a high-platform environment based on deep reinforcement learning, using the noise-infused current state data. By introducing noise, the present disclosure enhances the diversity and robustness of the state data, enabling the trained robot high jump motion control model to adapt to complex terrain and sensor interference.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of sensors and robots, and in particular to a method and device for training a robot high jump platform motion control model. Background Art

[0002] In the field of robotic motion control, inertial measurement units (IMUs), as key state observation sensors, are widely used for robot posture estimation, motion prediction, and control strategy optimization. However, in real-world environments, IMU observation data is often subject to interference from shock, vibration, and external noise, resulting in reduced observation accuracy and impacting the stability of robot motion control. In particular, in scenarios involving robots jumping from a high platform, IMU measurement deviations can prevent the robot from accurately acquiring its own state information during jumps or landings, potentially affecting its decision-making process and even causing the robot to lose balance and fall.

[0003] In addition, when the robot continuously performs jumping tasks, the impact noise of the IMU when landing will gradually accumulate, causing the observation data to be used for the next control decision before it returns to normal, further aggravating the motion deviation and making it difficult for the robot to maintain a stable jumping rhythm.

[0004] In existing technologies, robot motion control strategies are typically trained in a simulation environment and adapted to real-world conditions through transfer learning. However, IMU data in simulation is typically idealized. Therefore, when the trained control strategy is applied in a real-world environment, IMU observation errors can cause the robot to fail to correctly perceive state deviations at the moment of landing, leading to control instability and even a fall. Summary of the Invention

[0005] The present disclosure provides a method and device for training a robot high jump platform motion control model, which are used to reduce control instability caused by IMU measurement deviation in robot high jump platform scenarios.

[0006] Additional aspects and advantages of the disclosure will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the disclosure.

[0007] According to a first aspect of the present disclosure, a method for training a robot high jump motion control model is provided, comprising:

[0008] receiving current state data of the virtual robot, the current state data including observation data of an inertial measurement device;

[0009] adding a first noise to observation data of the inertial measurement device;

[0010] Combined with the current state data after adding the first noise, the robot jumping high platform motion control model is trained based on the high platform terrain through deep reinforcement learning.

[0011] In an exemplary embodiment of the present disclosure, the first noise includes impact noise; adding the first noise to the observation data of the inertial measurement device includes:

[0012] When it is determined that the virtual robot has landed according to the current state data of the virtual robot, impact noise is added to the observation data of the inertial measurement device.

[0013] In an exemplary embodiment of the present disclosure, adding first noise to observation data of an inertial measurement device further includes:

[0014] When it is determined according to the current state data of the virtual robot that the virtual robot is in continuous jumping motion, impact noise is added after each jumping motion.

[0015] In an exemplary embodiment of the present disclosure, the first noise further includes vibration noise; and adding the first noise to the observation data of the inertial measurement device further includes:

[0016] When it is determined based on current state data of the virtual robot that the virtual robot is in continuous jumping motion, vibration noise is added to observation data of the inertial measurement device.

[0017] In an exemplary embodiment of the present disclosure, the first noise includes random Gaussian noise, and the mean and variance of the random Gaussian noise are determined according to data characteristics of a real inertial measurement device.

[0018] In an exemplary embodiment of the present disclosure, the method further includes:

[0019] The intensity of the first noise is dynamically adjusted according to the jumping height of the virtual robot.

[0020] In an exemplary embodiment of the present disclosure, the first noise includes impact noise; and dynamically adjusting the intensity of the first noise according to the jumping height of the virtual robot includes:

[0021] according to:

[0022]

[0023] Adjust the intensity of the first noise;

[0024] in, is the impact noise at time t, is the shock indicator function, The mean is 0. is a Gaussian distribution with a variance of is the basic noise variance, is the linear amplification factor of noise intensity to jump height, and h is the jump height.

[0025] In an exemplary embodiment of the present disclosure, the first noise includes vibration noise; and dynamically adjusting the intensity of the first noise according to the jumping height of the virtual robot comprises:

[0026] according to:

[0027]

[0028]

[0029] Adjust the intensity of the first noise;

[0030] in, is the vibration noise at time t, is the amplitude of vibration noise, is the basic vibration amplitude, is the linear coefficient of the vibration amplitude increasing with the jump height, h is the jump height, is the vibration frequency, is the phase offset.

[0031] In an exemplary embodiment of the present disclosure, the first noise includes random Gaussian noise; and dynamically adjusting the intensity of the first noise according to the jumping height of the virtual robot includes:

[0032] according to:

[0033]

[0034] Adjust the intensity of the first noise;

[0035] in, is the current noise standard deviation calculated based on the jump height h, which is used to construct the Gaussian distribution later. is the basic noise standard deviation, is the amplification coefficient of the noise as the jump height changes.

[0036] In an exemplary embodiment of the present disclosure, the intensity range of the first noise is determined based on a statistical distribution of measurement errors of a real inertial measurement device at the moment the robot lands.

[0037] In an exemplary embodiment of the present disclosure, adding first noise to observation data of an inertial measurement device includes:

[0038] A first noise is added to one or more of the acceleration observation data and the angular velocity observation data of the inertial measurement device.

[0039] In an exemplary embodiment of the present disclosure, a robot high-platform jumping motion control model is trained based on high-platform terrain by deep reinforcement learning, including:

[0040] In a high platform terrain simulation environment, the teacher-student model framework is used to train the policy network used to output the action strategy for controlling the robot's motion, and a robot high platform jumping motion control model is obtained.

[0041] In an exemplary embodiment of the present disclosure, a teacher-student model framework is used to train a policy network for outputting an action policy for controlling robot motion, including:

[0042] The robot's historical motion state information is encoded by the student encoder and a first latent vector is generated. The robot's privileged state information is encoded by the teacher encoder and a second latent vector is generated.

[0043] Selecting a first latent vector or a second latent vector according to a preset strategy and inputting the vector into the strategy network;

[0044] Outputting, through the policy network, an action policy for controlling the motion of the robot based on the received first latent vector or the second latent vector and current motion state information, wherein the current motion state information includes observation data of the inertial measurement device;

[0045] Utilize privileged state information through the value network to output a value estimate of the current state;

[0046] optimizing parameters of the student encoder based on a difference between the first latent vector and the second latent vector;

[0047] Based on the action strategy and value estimation, the parameters of the policy network are updated using the deep reinforcement learning algorithm.

[0048] In an exemplary embodiment of the present disclosure, utilizing privileged state information through a value network to output a value estimate of the current state further includes:

[0049] The value network utilizes privileged state information and combines it with the first latent vector or the second latent vector selected by the preset strategy to output a value estimate of the current state.

[0050] According to a second aspect of the present disclosure, a robot high jump platform motion control method is provided, comprising:

[0051] Get the current motion state information of the robot;

[0052] Output the motion strategy for controlling the robot's motion based on the current motion state information and the pre-trained robot high jump platform motion control model;

[0053] The robot high jump platform motion control model is obtained according to the robot high jump platform motion control model training method in the above embodiment.

[0054] In one exemplary embodiment of the present disclosure, a pre-trained robot high jump motion control model includes a student encoder and a policy network;

[0055] Based on the current motion state information and the pre-trained robot high jump platform motion control model, the motion strategy for controlling the robot's motion is output, including:

[0056] The robot's historical motion state information is encoded through the student encoder to generate a first latent vector, which is then input into the policy network.

[0057] The policy network outputs an action policy for controlling the robot's motion based on the received first potential vector and the current motion state information.

[0058] According to a third aspect of the present disclosure, a robot high jump platform motion control model training device is provided, comprising:

[0059] A data acquisition module is used to receive the current state data of the virtual robot, the current state data including the observation data of the inertial measurement device;

[0060] A noise adding module, configured to add a first noise to the observation data of the inertial measurement device;

[0061] The model training module is used to combine the current state data after adding the first noise, and train the robot jumping high platform motion control model based on the high platform terrain through deep reinforcement learning.

[0062] According to a fourth aspect of the present disclosure, there is provided a robot high jump platform motion control device, comprising:

[0063] Information acquisition module, used to obtain the current motion state information of the robot;

[0064] The strategy output module is used to output the action strategy for controlling the robot's motion based on the current motion state information and the pre-trained robot high jump platform motion control model;

[0065] The robot high jump platform motion control model is obtained according to the robot high jump platform motion control model training method in the above embodiment.

[0066] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0067] processor; and

[0068] The memory stores computer-readable instructions, which implement the method in the above embodiment when executed by the processor.

[0069] According to a sixth aspect of the present disclosure, there is provided a robot, comprising:

[0070] processor; and

[0071] The memory stores computer-readable instructions, which implement the method in the above embodiment when executed by the processor.

[0072] In an exemplary embodiment of the present disclosure, the robot includes any one of a legged robot, a quadruped robot, a bipedal robot, a wheeled robot, a wheel-legged robot, a quadrupedal robot, a humanoid robot, a cleaning robot, a transport robot, a mobile robot and a robotic arm.

[0073] According to a seventh aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program code instructions are stored. When the computer program code instructions are called by a processor of a robot, the robot executes the method as described in the above embodiment.

[0074] It can be seen from the above technical solutions that the present disclosure has at least one of the following advantages and positive effects:

[0075] The method for training a robot high jump motion control model in an exemplary embodiment of the present disclosure adds noise to the observed data when receiving the robot's current state data to simulate the measurement deviations of the IMU in a real environment. This allows the control model to adapt to observation errors caused by impact or vibration during the training phase. Existing techniques typically use idealized observation data for training, which makes it difficult for the control strategy to make correct judgments when encountering IMU deviations in a real environment. This is especially true at the moment of landing or during consecutive jumps, where errors can gradually accumulate and affect motion stability. The present disclosure actively introduces noise into the observed data, allowing the trained control strategy to better adapt to the deviations in actual IMU data, thereby reducing the risk of decision-making instability caused by measurement errors. Furthermore, based on the noisy observation data, the present disclosure performs deep reinforcement learning training on high platform terrain, enabling the control model to simultaneously adapt to different altitude terrain conditions and IMU observation errors. When a robot lands at different altitudes, the IMU is affected differently by the impact. If the control strategy fails to fully account for this effect, it may still lead to posture estimation deviations, which in turn affect the next jump. During the training process, the present disclosure enables the control strategy to maintain more stable decision-making capabilities when facing landing impact vibrations at different heights, thereby improving the reliability of motion control and further avoiding control instability. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0077] Figure 1 A system architecture diagram is shown to which the robot high jump platform motion control model training method and the robot high jump platform motion control method in the embodiments of the present disclosure can be applied.

[0078] Figure 2 A flow chart of a robot high jump platform motion control model training method in an embodiment of the present disclosure is shown.

[0079] Figure 3 A schematic diagram of a two-stage training-inference framework of a policy network in an embodiment of the present disclosure is shown.

[0080] Figure 4 A schematic diagram of a process of pre-training a strategy network in an embodiment of the present disclosure is shown.

[0081] Figure 5 A schematic diagram of a two-stage training-inference framework of another policy network in an embodiment of the present disclosure is shown.

[0082] Figure 6 A schematic diagram showing the principles of a robot high jump platform motion control model training method in an embodiment of the present disclosure is shown.

[0083] Figure 7 A schematic diagram of a scene of a robot jumping from a high platform in an embodiment of the present disclosure is shown.

[0084] Figure 8 A schematic flow chart of a robot high jump platform motion control method in an embodiment of the present disclosure is shown.

[0085] Figure 9 A block diagram of a robot high jump platform motion control model training device in an embodiment of the present disclosure is shown.

[0086] Figure 10 A block diagram of a robot high jump platform motion control device in an embodiment of the present disclosure is shown.

[0087] Figure 11 A schematic diagram of a robot in an embodiment of the present disclosure is shown.

[0088] Figure 12 A schematic diagram of another robot in an embodiment of the present disclosure is shown.

[0089] Figure 13A schematic diagram of another robot in an embodiment of the present disclosure is shown.

[0090] Figure 14 A schematic diagram of another robot in an embodiment of the present disclosure is shown.

[0091] Figure 15 A schematic structural diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0092] Figure 16 A schematic structural diagram of a program product in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0093] In this disclosure, the terms "first" and "second" are used for descriptive purposes only and do not indicate relative importance or imply the number of technical features. Therefore, "first" and "second" may explicitly or implicitly include at least one of these features. "Multiple" means at least two, unless expressly limited otherwise.

[0094] Figure 1 A system architecture diagram is shown to which the robot high jump platform motion control model training method and the robot high jump platform motion control method in the embodiments of the present disclosure can be applied.

[0095] like Figure 1 As shown, system architecture 100 may include terminal device 101, robot 102, network 103, and server 104. Terminal device 101 includes, but is not limited to, desktop computers, portable computers, smartphones, and tablet computers, and is used to provide a human-computer interaction interface, allowing users to set training parameters, monitor the training process, or view training results. It can also issue task control instructions and receive status feedback from server 104 or robot 102.

[0096] Robot 102 is a virtual robot deployed in a simulation platform, equipped with comprehensive state observation and action execution capabilities. It is used to collect training data and validate strategies for jumping tasks in a high-altitude terrain simulation environment. Robot 102's current state data includes observations from its inertial measurement device (IMD). To improve the strategy's robustness and adaptability to actual deployments, server 104 injects noise into the observation data to simulate measurement errors and environmental disturbances present in real sensors, thereby training a more fault-tolerant control strategy.

[0097] The network 103 is used to provide a medium for the communication link between the terminal device 101, the robot 102 and the server 104, and can include various connection types, such as wired, wireless communication links or fiber optic cables, etc., to ensure stable and efficient data interaction between the terminal device 101, the robot 102 and the server 104, including status data uploading, training result synchronization and model parameter distribution.

[0098] Server 104 is used to build a high-platform terrain simulation environment, schedule a virtual robot to perform high-platform jumps under different initial states, collect training samples, and execute a deep reinforcement learning training process based on the noise-added state data to continuously optimize the jumping strategy. After training is completed, server 104 deploys the high-platform control model to robot 102 for verification and testing, allowing terminal device 101 to remotely view the strategy execution results.

[0099] Architecture 100 implements an integrated closed-loop process from simulation environment construction, noisy state modeling to robust control strategy training, effectively improving the robot's jumping ability in height-difference scenarios and its adaptability to sensor errors.

[0100] It should be understood that Figure 1 The number and types of terminal devices, robots, networks, and servers in the embodiment are merely illustrative. Depending on the implementation requirements, any number and type of terminal devices, robots, networks, and servers may be used.

[0101] The present disclosure provides a method for training a robot high jump platform motion control model. Figure 2 As shown, the method may include the following steps S210 to S230:

[0102] Step S210, receiving current state data of the virtual robot, the current state data including observation data of an inertial measurement device;

[0103] Step S220, adding first noise to the observation data of the inertial measurement device;

[0104] Step S230 , combining the current state data after adding the first noise, and training the robot high platform jumping motion control model based on the high platform terrain through deep reinforcement learning.

[0105] The robot high jump platform motion control model training method provided by the example embodiment of the present disclosure is implemented. By actively introducing noise into the observation data, the trained control strategy can better adapt to the deviation of the actual IMU data, thereby reducing the risk of decision-making instability caused by measurement errors. In addition, based on the observation data combined with the added noise, deep reinforcement learning training is carried out in the high platform terrain, so that the control model can simultaneously adapt to different altitude terrain conditions and IMU observation errors. Moreover, during the training process, the control strategy can maintain a more stable decision-making ability when facing landing impact vibrations at different heights, thereby improving the reliability of motion control and further avoiding control instability.

[0106] The following describes in detail the robot high jump platform motion control model training method in this example embodiment.

[0107] In step S210 , current state data of the virtual robot is received, where the current state data includes observation data of an inertial measurement device.

[0108] Among them, a virtual robot refers to a robot model built in a simulation environment that can simulate the movement, perception, and control processes of a real robot. For example, a virtual robot can output virtual sensor data such as acceleration and angular velocity for training control strategies or performing system verification without the use of real hardware. It is safe, efficient, and low-cost. Inertial measurement devices refer to sensors used in the inertial navigation system that simulate real robots, including accelerometers, gyroscopes, etc., which are used to observe the acceleration, angular velocity, magnetic field data, etc. of the virtual robot in the virtual space. The observation data of the inertial measurement device can be output in a multi-axis form, such as three-axis acceleration and three-axis angular velocity, which can reflect the robot's posture changes, motion trends, and force characteristics in the virtual environment.

[0109] By collecting observation data from the virtual robot, we can accurately estimate its motion state, including information such as its velocity, posture, and displacement. This provides precise and stable input for subsequent deep reinforcement learning, helping to accelerate model convergence and improve training efficiency. Especially in high-altitude terrain tasks, accurate state data can more clearly reflect the virtual robot's real-world interaction with the environment, facilitating the learning of key control features for jumping movements and ultimately establishing a more effective initial motion control strategy.

[0110] Moreover, without relying on external positioning systems such as GPS or visual sensors, inertial measurement devices alone can achieve high-frequency state estimation in a short period of time, offering the advantages of strong real-time performance and fast response. Furthermore, during simulation training, algorithm verification, and virtual debugging, virtual inertial data can effectively simulate the robot's real-world response behavior in complex dynamic environments, reducing reliance on actual hardware and improving system development efficiency.

[0111] In step S220 , first noise is added to the observation data of the inertial measurement device.

[0112] In the example implementation of the present disclosure, adding the first noise to the observation data of the inertial measurement device refers to artificially introducing a certain random disturbance when processing the sensor data of the virtual robot to simulate the uncertainty caused by factors such as environmental interference and hardware errors in the operation of the real sensor.

[0113] For example, the observation data of an inertial measurement device includes acceleration observation data and angular velocity observation data. The first noise can be Gaussian noise, random noise, or the like. Accordingly, the first noise can be added to the acceleration observation data and / or the angular velocity observation data, respectively. The processed observation data is more closely aligned with the sensor output in real-world application scenarios, helping to improve the deep reinforcement learning model's tolerance for uncertain information, enhance the generalization and robustness of the training strategy, and make it more stable and reliable in real-world systems.

[0114] In some example embodiments, the first noise includes impact noise, which refers to a sudden noise pattern that simulates a sudden impact on the sensor. Impact noise may appear as a short, high-amplitude, random pulse signal that is temporally localized and has uncertain amplitude.

[0115] Optionally, impact noise can be added to the inertial measurement device's observation data when the virtual robot's current state data indicates that the virtual robot has landed. For example, at the moment the virtual robot falls from a platform and contacts the ground, the inertial measurement device's observation data, such as acceleration and angular velocity, will experience a sharp change. At this point, impact noise can be injected into the observation data to simulate the instantaneous sensor disturbances caused by factors such as structural impact, ground reaction force, or mechanical vibration when the real robot lands. For example, the impact noise can be directly superimposed on the original observation data.

[0116] By determining that the virtual robot is in a landing state and adding impact noise to the observation data of the inertial measurement device, not only can the diversity of training data be enhanced, but also the deep reinforcement learning model can learn how to maintain stable control or quickly recover when subjected to impact interference, which helps to improve the robustness and reliability of motion control strategies in actual complex scenarios.

[0117] Optionally, if the virtual robot's current state data indicates that it is performing continuous jumping movements, impact noise can be added after each jump. Specifically, upon recognizing that the virtual robot is in a continuous jumping task, the system detects changes in its current state, such as a rapid decrease in altitude or a sudden increase in acceleration, at each moment of landing to determine that the robot has completed a jump and made contact with the ground. In this case, impact noise is injected into the inertial measurement device's observation data to simulate the transient interference or vibration response of the sensor caused by the impact of landing during the real robot's frequent jumping.

[0118] By judging that the virtual robot is in continuous jumping motion and completing each jump, and adding impact noise to the observation data of the inertial measurement device, the authenticity of the virtual training data can be effectively enhanced, making the deep reinforcement learning model more adaptable when facing frequent and repeated landing impacts, avoiding overfitting to ideal data, and improving the tolerance to posture fluctuations and sensor disturbances in actual jumping motion, thereby making the robot control strategy more robust and stable.

[0119] In some example embodiments, the first noise also includes vibration noise, which simulates the subtle vibration interference experienced by a sensor during high-frequency repetitive motion. Vibration noise is generally persistent and can manifest as small, continuous, high-frequency fluctuations. It simulates the background disturbances experienced by an inertial measurement device during a real robot's jumping, takeoff, or hovering due to structural resonance, motor vibration, or ground vibration.

[0120] For example, when the virtual robot's current state data indicates that the virtual robot is in a continuous jumping motion, vibration noise can be added to the inertial measurement device's observation data. Specifically, during the process of identifying the virtual robot performing a continuous jumping task, the robot's state data, such as periodic displacement changes, posture adjustment frequency, and contact period, are analyzed to determine that the virtual robot is in a continuous jumping state. During this period, vibration noise is superimposed on the observed acceleration and angular velocity data. For example, a high-frequency sine wave, bandpass filtered random noise, or an oscillation model simulating mechanical resonance can be superimposed on the observed values.

[0121] By adding vibration noise, the training data's ability to simulate vibration interference in actual scenarios can be enhanced, making the deep reinforcement learning model more robust when processing real sensor data, and improving its control stability and reliability in dynamic and complex actions such as continuous jumping.

[0122] In some example embodiments, the first noise includes random Gaussian noise. Random Gaussian noise refers to the introduction of a random disturbance signal based on a normal distribution into the observed data when simulating the output data of an inertial measurement device in order to more realistically reflect the measurement errors and background interference existing in its actual working state.

[0123] It's important to note that the mean and variance of random Gaussian noise are determined based on the data characteristics of the actual inertial measurement device. For example, given the accelerometer's bias error and noise density, as well as the gyroscope's angular velocity bias drift and noise density, a corresponding Gaussian distribution model can be constructed based on these characteristic parameters and added as a noise term to the acceleration and angular velocity observations output by the virtual robot.

[0124] By adopting a Gaussian noise model determined by the characteristics of a real inertial measurement device, the distribution characteristics of the virtual robot's observation data are closer to the sensor output in actual use, thereby improving the adaptability of the trained model when deployed on a real robot system.

[0125] It should be noted that in the exemplary embodiments of the present disclosure, the intensity of the first noise can also be dynamically adjusted according to the jumping height of the virtual robot. That is, when the first noise such as impact noise, vibration noise, and random Gaussian noise is added to the observation data of the inertial measurement device, a fixed noise intensity parameter is no longer used. Instead, the amplitude of the first noise is adaptively adjusted according to the maximum height or height change amplitude in the current jumping action. For example, the higher the jumping height, the greater the impact of the robot when it lands, and the more severe the structural jitter and inertial disturbance. Therefore, the observation error or noise generated by the sensor in these cases is also more obvious. Accordingly, stronger noise should be set for higher jumps during simulation to more realistically reflect the highly correlated dynamic interference process.

[0126] For example, for impact noise, assuming that the amplitude of the impact noise is proportional to the jump height, we have:

[0127] (1)

[0128] in, is the impact noise at time t, is the impact indicator function, which takes the value of 1 when the virtual robot lands at time t and 0 at other times. It is used to control the noise to only take effect at the moment of landing; The mean is 0. is a Gaussian distribution with a variance of is the basic noise variance, is the linear amplification factor of noise intensity to jump height, and h is the jump height.

[0129] Formula (1) indicates that at the moment of landing, a Gaussian noise with a variance that increases with the increase of height is superimposed according to the current jump height h.

[0130] For another example, for vibration noise, assuming that the amplitude of vibration noise can be adjusted with the jump height, we have:

[0131] (2)

[0132] (3)

[0133] in, is the vibration noise at time t, is the amplitude of vibration noise, is the basic vibration amplitude, is the linear coefficient of the vibration amplitude increasing with the jump height, h is the jump height, is the vibration frequency, is the phase offset.

[0134] Formulas (2) and (3) indicate that the vibration amplitude increases linearly with the jump height, thus simulating more severe high-frequency structural responses.

[0135] For another example, for random Gaussian noise, assuming that the variance of random Gaussian noise can be dynamically scaled according to the jump height, we have:

[0136] (4)

[0137] in, is the current noise standard deviation calculated based on the jump height h, which is used to construct the Gaussian distribution later. is the basic noise standard deviation, indicating the minimum noise level when there is no jump height, is the amplification coefficient of the noise as the jump height changes.

[0138] By dynamically adjusting the intensity of the first noise, the observation data of the inertial measurement device will show a more reasonable noise change trend when the virtual robot performs jumping movements of different amplitudes. This not only improves the realism of the sensor model, but also enables deep reinforcement learning to learn more robust strategies when facing sensor uncertainties under different motion intensities, thereby improving the generalization ability of the control strategy and the reliability of real-world deployment.

[0139] Furthermore, the intensity range of the first noise can be determined based on the statistical distribution of measurement errors of a real inertial measurement device at the moment the robot lands. Specifically, by collecting observed error data of the inertial measurement device during the robot's jump and landing process, the error distribution characteristics at the moment of landing are statistically analyzed. Based on these error distribution characteristics, a reasonable value range for the first noise intensity parameter used in the simulation system is derived.

[0140] For example, the actual error sequence is first obtained by extracting the difference between the robot's inertial observation value at the moment of landing and its theoretical acceleration or posture change through measured data. This error sequence is then statistically analyzed, including calculating its mean, standard deviation, extreme value, confidence interval, and probability distribution model such as Gaussian distribution, to obtain the typical intensity range of the error under landing impact conditions. Furthermore, the standard deviation of the landing error and the maximum disturbance amplitude can be used as key parameters in the first noise generation function, ensuring that the noise added in the simulation is consistent with the actual sensor performance in terms of intensity and distribution characteristics.

[0141] In this example, the intensity range of the first noise determined has a realistic basis, which can effectively improve the physical rationality of the inertial observation data in the simulation, making the virtual sensor output during training closer to the real environment, thereby improving the generalization ability and stability of the deep reinforcement learning model during actual deployment.

[0142] In step S230, in combination with the current state data after adding the first noise, the robot jumping high platform motion control model is trained based on the high platform terrain through deep reinforcement learning.

[0143] Among them, a task scenario including complex terrains such as high platforms is constructed in a virtual simulation environment. By obtaining the observation data of the inertial measurement device in the virtual robot after adding the first noise and using it as training data input, the adaptability of the deep reinforcement learning algorithm to the uncertainty factors in the real dynamic environment is improved.

[0144] During training, the virtual robot continuously attempts jumping movements to complete the target task of reaching the platform. Each action decision depends on current state data, including position, velocity, posture, and inertial sensor observations. The added first noise effectively simulates the errors and interference of real sensors during jumping and landing, making the training process more realistic. By combining it with the reward function, the deep reinforcement learning model gradually learns how to maintain posture stability, launch appropriately, and land accurately on the platform despite noisy state perception, ultimately developing a jumping control strategy that is robust to dynamic noise such as impact and vibration. This not only improves training efficiency and policy generalization, but also provides a more stable and reliable control foundation for real robots to perform highly dynamic action tasks in complex terrain.

[0145] In some example implementations, a policy network that outputs action strategies for controlling robot motion can be trained using a teacher-student model framework in a simulated environment with a high platform terrain, thereby obtaining a robot high-platform motion control model. This training process can enhance the policy network's learning of successful high-platform jump behaviors. Furthermore, by maintaining a reasonable closeness of the policy network to the teacher's behavior through imitation loss or auxiliary supervision, the policy convergence speed and performance stability are improved. Ultimately, a robot high-platform motion control model with both imitation and enhanced adaptive capabilities is obtained, suitable for continuous jumping control tasks in complex terrain.

[0146] refer to Figure 3Figure 1 shows a schematic diagram of a two-stage training and inference framework for a policy network. The training phase combines a teacher-student encoder structure with a deep reinforcement learning framework. The teacher-student encoder structure includes a teacher encoder 301 and a student encoder 302. The deep reinforcement learning framework consists of a policy network 303, a value network 304, and a PPO algorithm. During the training phase, privileged state information is introduced to guide policy learning, allowing the student encoder 302 to stably output high-quality control actions during the inference phase, relying solely on observable data. This observable data includes the robot's current motion state information.

[0147] It is important to note that the teacher encoder 301 is only used during the training phase to extract high-dimensional semantic representations from the privileged state information, which are subsequently used as policy input to guide learning. During the training and inference phases, the student encoder 302 encodes the robot's own historical motion states to obtain dynamic features related to action decisions. This information is then fed into the policy network 303 to generate control actions. Furthermore, the student encoder 302 ultimately needs to mimic the representations output by the teacher encoder 301. Therefore, distillation training is performed during training by minimizing the mean squared error (MSE).

[0148] The policy network 303 is the core module for motion generation. During the training phase, the policy network 303 receives the latent vector and current motion state information from the teacher encoder 301 or the student encoder 302. In conjunction with the value network 304, it uses the PPO algorithm to update the policy. During the inference phase, the policy network 303 receives the latent vector and current motion state information from the student encoder 302 and outputs the motion policy for controlling the robot's motion. It should be noted that the current motion state information includes observation data from the inertial measurement device after adding the first noise.

[0149] based on Figure 3 The frame diagram shown is shown in Figure 4 As shown, the process of training the policy network for outputting the action policy for controlling the robot movement may include the following steps S401 to S406:

[0150] In step S401 , the robot's historical motion state information is encoded by the student encoder and a first latent vector is generated. The robot's privileged state information is encoded by the teacher encoder and a second latent vector is generated.

[0151] The robot's historical motion state information can include joint angles, joint velocities, foot contact states, center of gravity trajectory, and other information from the past several frames. For example, the past 10 frames of the historical motion state sequence are input into the student encoder 302 for encoding to generate a first latent vector. This first latent vector can capture the continuity and dynamic characteristics of the robot's motion.

[0152] Privileged state information can include the robot's actual contact force with the ground, an environmental height map, and disturbance information. This privileged state information is input into the teacher encoder 301 for encoding, generating a third latent vector. This third latent vector can highly condense key semantics such as terrain structure, obstacle distribution, and robot-environment interaction, helping to construct a more complete high-dimensional feature representation of the environmental state.

[0153] It should be noted that privileged state information can only be obtained during the training phase but cannot be observed during the testing or deployment phase. Therefore, the teacher encoder 301 can use the complete information to learn the best potential expression, thereby guiding the student encoder 302 to learn. Both the teacher encoder 301 and the student encoder 302 can compress high-dimensional, temporal state information into low-dimensional potential representations, providing behavioral semantic representations from different sources for the policy network 303. For example, the teacher encoder 301 and the student encoder 302 can be multi-layer perceptrons, temporal convolutional networks, or recurrent neural networks, capable of extracting temporal features and compressing them into fixed-length semantic vectors. In addition, the network architecture of the teacher encoder 301 and the network architecture of the student encoder 302 can be the same or different, and this disclosure does not limit this.

[0154] Step S402: Select a first latent vector or a second latent vector according to a preset strategy and input it into a strategy network.

[0155] According to a preset strategy, the first latent vector generated by the student encoder 302 or the second latent vector generated by the teacher encoder 301 is fed into the policy network 303 for decision-making. The preset strategy can be set based on the real-time environment state, the training stage, or specific performance indicators, such as the action execution error threshold, the difference between the simulation and the real environment, etc.

[0156] For example, one preset strategy is to give priority to using the high-quality second latent vector generated by the teacher encoder 301 at the beginning of training to guide the policy network 303 to quickly converge to an approximately optimal solution. After the student encoder 302 is optimized through knowledge distillation, it gradually transitions to using only the first latent vector in the deployment phase to reduce dependence on privileged information. For another example, another preset strategy is to select the second latent vector according to the ratio p, select the first latent vector according to 1-p, and gradually reduce the value of p. Of course, the first latent vector and the second latent vector can also be spliced ​​and sent to the policy network 303, and dynamically weighted through the attention mechanism or the gating module, so that the policy network 303 can flexibly combine historical experience and privileged knowledge in complex scenarios.

[0157] The selective input mechanism can not only accelerate the training process by utilizing the ideal state prior knowledge provided by the teacher encoder 301, but also cope with sensor limitations or environmental disturbances in actual operation through the generalization ability of the student encoder 302. At the same time, the policy network 303 conditions the latent vector to achieve smooth switching and robust decision-making of motion control. For example, when the robot encounters unknown terrain, it preferentially adjusts the gait based on the second latent vector, while relying on the first latent vector to maintain efficiency during the stable walking stage. Finally, the closed-loop feedback optimizes the policy selection rules, so that the robot can balance motion performance and adaptability in different stages and environmental conditions.

[0158] In addition, in addition to the selected potential vector, the current motion state information of the robot can also be input into the strategy network 303 for decision making.

[0159] Step S403: outputting an action strategy for controlling the motion of the robot based on the received first latent vector or the second latent vector and the current motion state information through the strategy network; wherein the current motion state information includes observation data of the inertial measurement device.

[0160] The strategy network 303 can be a multi-layer fully connected perceptron, a Transformer structure, etc. The strategy network 303 performs multimodal feature fusion on the first latent vector or the second latent vector and the current motion state information, and outputs an action strategy, which is recorded as .

[0161] For example, a policy network can be created based on:

[0162] (5)

[0163] Output action strategies to control robot motion;

[0164] in, Indicates the current motion state Take action The probability distribution of MLP represents a multi-layer perceptron. represents the first latent vector or the second latent vector selected according to a preset strategy.

[0165] For another example, the first latent vector or the second latent vector and the current motion state information are spliced ​​or weighted in the embedding space to generate action strategies such as joint angle targets, torque commands or gait phase parameters through nonlinear transformation.

[0166] Step S404: Using the privileged state information through the value network, output the value estimate of the current state.

[0167] During the training of the policy network 303 , the long-term return of the current state is estimated through the value network 304 . That is, starting from this state, if the current policy is continuously executed, how much cumulative reward can be obtained in the future, so as to optimize the policy network 303 .

[0168] The value network 304 may be a multi-layer fully connected perceptron. For example, when the value network 304 uses the privileged state information to estimate the value of the current state, the privileged state information is first encoded into a high-dimensional feature vector, such as by extracting dynamic features through a convolutional or fully connected layer, and then fused with the current motion state information in the latent space. Subsequently, a multi-layer nonlinear transformation is performed to output the value estimate representing the current state, which is recorded as This value estimate is used for policy optimization in reinforcement learning to guide the policy network 303 to learn better behaviors.

[0169] Step S405 : Optimizing parameters of the student encoder based on the difference between the first latent vector and the second latent vector.

[0170] This step quantifies the distribution difference between the two in the latent space and back-propagates the difference gradient to update the network weights of the student encoder 302, thereby guiding the student encoder 302 to learn to generate a latent representation close to that of the teacher encoder 301, thereby achieving teacher knowledge distillation.

[0171] Exemplarily, during the training phase, the parameters of the teacher encoder 301 are fixed, and the historical motion state information and the corresponding motion state information are taken as parallel inputs. After generating latent vectors through the student encoder 302 and the teacher encoder 301 respectively, the distance between the two is minimized using a contrastive learning framework, or adversarial training is used to make the output distribution of the student encoder 302 approach the latent space characteristics of the teacher encoder 301. At the same time, noise injection or data enhancement is introduced to simulate sensor errors in actual deployment, forcing the student encoder 302 to extract feature expressions compatible with privileged information encoding under the condition of limited input information.

[0172] For example, to make the output of the student encoder 302 as close as possible to the teacher encoder 301, a difference loss function, such as the least squares difference or KL divergence, can be constructed. By minimizing this loss, the parameters of the student encoder 302 are optimized, allowing it to learn to extract feature expressions close to the privileged information from the historical motion state, thereby enhancing the generalization ability of the policy network 303 and improving the decision quality of the policy network 303 in real environments, so that it can approach the teacher's level when deployed without relying on privileged information.

[0173] Step S406: Based on the action strategy and value estimation, the parameters of the policy network are updated using a deep reinforcement learning algorithm.

[0174] Take the PPO algorithm as an example. During the training process of the policy network 303, the current policy network 303 is used to interact with the environment. Select an action strategy , returns the reward after executing the action and the next state , and the interaction trajectory sequence is obtained by sampling ( ), which is used for subsequent strategy optimization. Next, the value network 304 is introduced to evaluate the value of each state .

[0175] Based on this, if we construct the advantage function :

[0176] (6)

[0177] in, is a discount factor used to measure the importance of future rewards.

[0178] Then, an optimization objective is constructed based on the clipping objective function of the PPO algorithm. Policy gradients are calculated through backpropagation to optimize the parameters of policy network 303, allowing it to continuously improve the expected cumulative reward under the current policy. This update process guides policy network 303 to gradually learn the action strategy that maximizes long-term cumulative reward while maintaining stable output, thereby improving the robustness and execution efficiency of the control strategy in real-world environments.

[0179] refer to Figure 5 As shown in Figure 2, a schematic diagram of a two-stage training-reasoning framework for another strategy network is shown. Figure 5 The framework diagram shown in Figure 2 shows the training process of the policy network. Figure 4 The training process is similar to that shown in FIG. 4 , except that the step of outputting the value estimate of the current state by using the privileged state information through the value network 304 is different from the implementation of step S404. Figure 6 The training phase shown is to output a value estimate of the current state by utilizing the privileged state information through the value network 304 in combination with the first latent vector or the second latent vector selected by the preset strategy.

[0180] In this example, the latent features are fused with privileged information and fed into the value network 304. This not only preserves the high credibility of the privileged information in understanding the environment, but also enhances the adaptability of the value estimate to the strategy's behavior trajectory and motion trends, better reflecting the expected long-term reward for the robot after taking actions in its current state. Furthermore, the fusion of latent vector inputs enhances the representational richness and generalization capabilities of the value network, making its output more closely aligned with the value distribution under real-world strategy behavior, thereby improving the stability and performance of overall strategy training.

[0181] refer to Figure 6 As shown, a schematic diagram of the principle of a method for training a robot high jump platform motion control model is shown. First, the current state data 601 of the robot is collected, including the observation data of the inertial measurement device. Then, noise is added to the observation data of the inertial measurement device to enhance the generalization ability of the model, so that it can maintain stable decision-making under simulated sensor errors or real-world disturbances. Then, based on the current state data 602 after adding noise, the robot high jump platform motion control model 603 is trained in the high platform terrain, so that the robot high jump platform motion control model 603 learns how to autonomously complete the high jump platform task according to the current state. Reference Figure 7 As shown, a schematic diagram of a scene of a robot jumping from a high platform is shown.

[0182] The present disclosure also provides a method for controlling robot motion. Figure 8 As shown, the method may include the following steps S801 to S802:

[0183] Step S801: Acquire the current motion state information of the robot.

[0184] The robot can be a real robot or a virtual robot. While the robot is performing a motion task, it collects its current motion state data in real time, including but not limited to the robot's own position, posture, and velocity. Importantly, the current motion state information includes observation data from an inertial measurement device, such as three-axis acceleration and three-axis angular velocity. Furthermore, the current motion state information may also include sensor readings of the robot body, such as joint angles, joint velocities, and end-point posture, though this disclosure does not specifically limit this.

[0185] Step S802 : outputting a motion strategy for controlling the robot's motion based on the current motion state information and the pre-trained robot high jump motion control model.

[0186] When the robot performs the high jump task, it uses the current motion state information input obtained in real time and combines it with the trained robot high jump motion control model to make action decisions.

[0187] For example, a robot high jump motion control model consists of a student encoder and a policy network. The student encoder encodes the robot's historical motion state information to generate a first latent vector, which is then fed into the policy network for further reference. The policy network then outputs an action strategy for controlling the robot's motion, such as takeoff timing, posture adjustment, or power distribution, based on the received first latent vector and current motion state information, thereby guiding the robot to complete the target jump onto the platform.

[0188] In this example, by combining real-time current motion state information with historical motion state encoding, the robot high jump platform motion control model can integrate historical motion trends based on the perception of the current state and make more accurate action decisions. Among them, the first latent vector extracted by the student encoder provides a dynamic understanding of past motion behavior, and the policy network uses this latent vector and the current state to jointly reason and output a reasonable jumping action strategy. Therefore, the robot high jump platform motion control model can not only identify whether the robot is in the take-off preparation, take-off or landing stage, but also adjust the take-off timing and posture according to the historical rhythm, thereby effectively improving the coordination, coherence and success rate of the jumping behavior, and enhancing the control accuracy and robustness in complex task scenarios such as high platform terrain.

[0189] It can be understood that the robot high jump platform motion control model is trained according to the robot high jump platform motion control model training method described in detail in other embodiments of the present disclosure, and will not be repeated here.

[0190] In an exemplary embodiment of the present disclosure, a robot high jump platform motion control model training device is also provided. Figure 9 As shown, the robot high jump platform motion control model training device 900 includes a data acquisition module 901, a noise addition module 902 and a model training module 903, wherein:

[0191] The data acquisition module 901 is used to receive the current state data of the virtual robot, and the current state data includes the observation data of the inertial measurement device;

[0192] A noise adding module 902 is configured to add a first noise to the observation data of the inertial measurement device;

[0193] The model training module 903 is used to combine the current state data after adding the first noise, and train the robot jumping high platform motion control model based on the high platform terrain through deep reinforcement learning.

[0194] The specific details of each module in the above-mentioned robot high jump platform motion control model training device have been described in detail in the corresponding robot high jump platform motion control model training method, so they will not be repeated here.

[0195] In an exemplary embodiment of the present disclosure, a robot high jump platform motion control device is also provided. Figure 10 As shown, the robot high jump platform motion control device 1000 includes an information acquisition module 1001 and a strategy output module 1002, wherein:

[0196] The information acquisition module 1001 is used to obtain the current motion state information of the robot;

[0197] A strategy output module 1002 is configured to output a motion strategy for controlling the robot's motion based on current motion state information and a pre-trained robot high jump motion control model;

[0198] Among them, the robot high jump platform motion control model is obtained according to the robot high jump platform motion control model training method in the embodiment of the present disclosure.

[0199] The specific details of each module in the above-mentioned robot high jump platform motion control device have been described in detail in the corresponding robot high jump platform motion control method, so they will not be repeated here.

[0200] In an exemplary embodiment of the present disclosure, a robot is further provided, the robot comprising a processor and a memory, the memory storing computer-readable instructions, and the computer-readable instructions implementing the above method when executed by the processor. The robot comprises any one of a legged robot, a quadruped robot, a bipedal robot, a wheeled robot, a wheeled-legged robot, a quadrupedal robot, a humanoid robot, a cleaning robot, a transport robot, a mobile robot, and a robotic arm. Figures 11 to 14 As shown, four different robot schematics are shown respectively.

[0201] refer to Figure 15 As shown, an electronic device capable of implementing the above method is also provided. The electronic device 1500 includes a processor 1501 and a memory 1502. The memory 1502 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 1501, the method in the embodiment of the present disclosure is implemented.

[0202] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is further provided, on which computer program code instructions are stored. When the computer program code instructions are called by a processor of a robot, the robot executes the method in the embodiment.

[0203] refer to Figure 16 As shown, a program product 1600 for implementing the above-described method according to an embodiment of the present disclosure is described. This program product 1600 may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be executed on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0204] Through the description of the above embodiments, it will be readily understood by those skilled in the art that the example embodiments described herein can be implemented via software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or mobile hard drive) or on a network and includes several instructions to enable a computing device (such as a personal computer, server, touch terminal, or network device) to execute the methods according to the embodiments of the present disclosure.

[0205] Finally, the above preferred embodiments are intended only to illustrate the technical solutions of this application and are not intended to be limiting. Although this application has been described in detail, those skilled in the art will appreciate that variations in form and detail may be made without departing from the scope of the claims. The dimensions of the drawings are not related to the actual objects and may be modified arbitrarily.

Claims

1. A robot high jump motion control model training method, characterized in that: include: receiving current state data of the virtual robot, wherein the current state data includes observation data of an inertial measurement device; Adding a first noise to the observation data of the inertial measurement device; wherein the intensity of the first noise is dynamically adjusted according to the jumping height of the virtual robot; Combined with the current state data after adding the first noise, the robot jumping high platform motion control model is trained based on the high platform terrain through deep reinforcement learning.

2. The high jump platform motion control model training method according to claim 1, characterized in that: The first noise includes impact noise; and adding the first noise to the observation data of the inertial measurement device includes: When it is determined according to current state data of the virtual robot that the virtual robot has landed, the impact noise is added to observation data of the inertial measurement device.

3. The high jump platform motion control model training method according to claim 2, characterized in that: The adding a first noise to the observation data of the inertial measurement device further includes: When it is determined according to the current state data of the virtual robot that the virtual robot is in continuous jumping motion, the impact noise is added after each jumping motion.

4. The high jump platform motion control model training method according to claim 2, characterized in that: The first noise further includes vibration noise; and the adding the first noise to the observation data of the inertial measurement device further includes: When it is determined according to current state data of the virtual robot that the virtual robot is in continuous jumping motion, the vibration noise is added to observation data of the inertial measurement device.

5. The robot high jump platform motion control model training method according to claim 1, characterized in that: The first noise includes random Gaussian noise, and a mean value and a variance of the random Gaussian noise are determined according to data characteristics of a real inertial measurement device.

6. The high jump platform motion control model training method according to claim 1, characterized in that: The intensity range of the first noise is determined based on a statistical distribution of measurement errors of a real inertial measurement device at the moment the robot lands.

7. The high jump platform motion control model training method according to claim 1, characterized in that: The first noise includes impact noise; and the dynamically adjusting the intensity of the first noise according to the jumping height of the virtual robot includes: according to: adjusting the intensity of the first noise; in, is the impact noise at time t, is the shock indicator function, The mean is 0. is a Gaussian distribution with a variance of is the basic noise variance, is the linear amplification factor of noise intensity to jump height, and h is the jump height.

8. The high jump platform motion control model training method according to claim 1, characterized in that: The first noise includes vibration noise; and the dynamically adjusting the intensity of the first noise according to the jumping height of the virtual robot includes: according to: adjusting the intensity of the first noise; in, is the vibration noise at time t, is the amplitude of vibration noise, is the basic vibration amplitude, is the linear coefficient of the vibration amplitude increasing with the jump height, h is the jump height, is the vibration frequency, is the phase offset.

9. The high jump platform motion control model training method according to claim 1, characterized in that: The first noise includes random Gaussian noise; and the dynamically adjusting the intensity of the first noise according to the jumping height of the virtual robot includes: according to: adjusting the intensity of the first noise; in, is the current noise standard deviation calculated based on the jump height h, which is used to construct the Gaussian distribution later. is the basic noise standard deviation, is the amplification coefficient of the noise as the jump height changes.

10. The high jump platform motion control model training method according to claim 1, characterized in that: The adding first noise to the observation data of the inertial measurement device comprises: The first noise is added to one or more of acceleration observation data and angular velocity observation data of the inertial measurement device.

11. The high jump platform motion control model training method according to any one of claims 1 to 10, characterized in that: The method of training a robot high-platform jumping motion control model based on high-platform terrain by deep reinforcement learning includes: In a high platform terrain simulation environment, a teacher-student model framework is used to train a strategy network for outputting action strategies for controlling robot motion, thereby obtaining the robot high platform jumping motion control model.

12. The high jump platform motion control model training method according to claim 11, characterized in that: The teacher-student model framework is used to train a policy network for outputting an action policy for controlling robot motion, including: Encoding the robot's own historical motion state information through a student encoder and generating a first latent vector, and encoding the robot's privileged state information through a teacher encoder and generating a second latent vector; Selecting a first latent vector or a second latent vector according to a preset strategy and inputting the vector into the strategy network; Outputting, through the policy network, an action policy for controlling the motion of the robot based on the received first latent vector or second latent vector and current motion state information; wherein the current motion state information includes observation data of an inertial measurement device; Utilizing the privileged state information through a value network, outputting a value estimate of the current state; optimizing parameters of the student encoder based on a difference between the first latent vector and the second latent vector; Based on the action strategy and the value estimate, the parameters of the policy network are updated using a deep reinforcement learning algorithm.

13. The high jump platform motion control model training method according to claim 12, characterized in that: Outputting a value estimate of the current state by utilizing the privileged state information through the value network further includes: The value network utilizes privileged state information and combines the first latent vector or the second latent vector selected by the preset strategy to output a value estimate of the current state.

14. A robot high jump platform motion control method, characterized in that: include: Get the current motion state information of the robot; Outputting a motion strategy for controlling the robot's motion based on the current motion state information and a pre-trained robot high jump platform motion control model; Wherein, the robot high jump platform motion control model is obtained according to the robot high jump platform motion control model training method according to any one of claims 1 to 13.

15. The robot high jump platform motion control method according to claim 14, characterized in that: The pre-trained robot high jump platform motion control model includes a student encoder and a policy network; The motion strategy for controlling the robot motion based on the current motion state information and the pre-trained robot high jump platform motion control model output includes: Encoding the robot's own historical motion state information through the student encoder to generate a first latent vector, and inputting the first latent vector into the policy network; The strategy network outputs an action strategy for controlling the movement of the robot based on the received first potential vector and current motion state information.

16. A robot high jump platform motion control model training device, characterized in that: include: A data acquisition module, configured to receive current state data of the virtual robot, wherein the current state data includes observation data of an inertial measurement device; a noise adding module, configured to add a first noise to the observation data of the inertial measurement device; wherein the intensity of the first noise is dynamically adjusted according to the jumping height of the virtual robot; The model training module is used to combine the current state data after adding the first noise, and train the robot jumping high platform motion control model based on the high platform terrain through deep reinforcement learning.

17. A robot high jump platform motion control device, characterized in that: include: Information acquisition module, used to obtain the current motion state information of the robot; A strategy output module, configured to output a motion strategy for controlling the robot's motion based on the current motion state information and a pre-trained robot high jump platform motion control model; Wherein, the robot high jump platform motion control model is obtained according to the robot high jump platform motion control model training method according to any one of claims 1 to 13.

18. An electronic device, characterized in that: include: processor; as well as A memory having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by the processor, implement the method according to any one of claims 1 to 15.

19. A robot, characterized in that: include: processor; as well as A memory having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by the processor, implement the method according to any one of claims 1 to 15.

20. The robot according to claim 19, characterized in that The robot includes any one of a legged robot, a quadruped robot, a bipedal robot, a wheeled robot, a wheel-legged robot, a quadrupedal robot, a humanoid robot, a cleaning robot, a transport robot, a mobile robot and a robotic arm.

21. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program code instructions, which, when called by a processor of a robot, enable the robot to execute the method according to any one of claims 1 to 15.

Images