Biped robot control method and device, electronic equipment and storage medium

Abstract

The invention discloses a biped robot control method and device, electronic equipment and a storage medium. The method comprises the steps that actual observation data at the current moment are collected through a target sensor of a biped robot body; wherein the target sensor comprises a joint encoder, an inertia measuring instrument and a plantar force sensor; predicting a state prediction value at the current moment by combining the state prediction value at the previous moment and a motion model of the biped robot; predicting predicted observation data at the current moment by using the state predicted value at the current moment based on the relationship between the observation data and the state data; correcting the state prediction value at the current moment based on the deviation between the actual observation data at the current moment and the prediction observation data to obtain a state correction value at the current moment; selecting a control parameter in a current state for the unified reinforcement learning control strategy through the correction function based on the state correction value at the current moment, and correcting the control parameter to obtain a current control parameter; and controlling the current biped robot according to the current control parameters.

Description

Technical Field

[0001] This application relates to the field of robot control, and in particular to a control method and device, electronic device, and storage medium for a bipedal robot. Background Technology

[0002] Because bipedal robots can walk in many complex environments, the demand for their application in fields such as industry, rehabilitation, public services, and exploration is constantly increasing. This means they need to face more complex environments, making their motion stability and adaptability in complex environments a core issue.

[0003] Currently, in order to enable bipedal robots to move stably and adaptively in complex environments, multiple different types of external sensors, such as cameras and LiDAR, are installed on the bipedal robots. These external sensors identify the current state of the external environment, and then switch or adjust the control measurements based on the identified state information of the current external environment, so that the current movement of the bipedal robot can adapt to the current environment and thus move stably.

[0004] However, since external sensors collect information from the external environment, they are easily affected by factors such as lighting, shading, and inclement weather, resulting in inaccurate data collection and compromising the stable movement of the bipedal robot. Furthermore, environmental recognition is prone to delays, which can also lead to instability. Additionally, the reliance on external sensors limits versatility across different robot platforms and hardware, hindering portability. Summary of the Invention

[0005] In view of the shortcomings of the prior art, this application provides a control method and device for bipedal robots, electronic devices, and storage media to solve the problems that the prior art cannot guarantee that the robot can move stably in various complex environments and is not universal.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] The first aspect of this application provides a control method for a bipedal robot, including:

[0008] The actual observation data at the current moment is obtained by collecting observation data through target sensors on the bipedal robot body; wherein, the target sensors include joint encoders, inertial measurement units, and plantar force sensors;

[0009] By combining the state prediction value of the previous moment with the motion model of the bipedal robot, the state data of the current moment is predicted to obtain the state prediction value of the current moment.

[0010] Based on the relationship between the observation data and the state data, the predicted state value at the current moment is used to predict the observation data, so as to obtain the predicted observation data at the current moment.

[0011] Based on the deviation between the actual observation data and the predicted observation data at the current moment, the predicted state value at the current moment is corrected to obtain the corrected state value at the current moment.

[0012] By using the correction function based on the state correction value at the current moment, the pre-trained unified reinforcement learning control policy is corrected to select the best control parameters under the current state, and the current control parameters are obtained.

[0013] The movement of the bipedal robot is controlled according to the current control parameters.

[0014] Optionally, in the above-described control method for a bipedal robot, the step of predicting the observed data based on the relationship between the observed data and the state data, using the predicted state value at the current moment, to obtain the predicted observed data at the current moment, includes:

[0015] The predicted state value at the current moment is input into the observation function for calculation to obtain the predicted observation data at the current moment. The observation function consists of the relationship between various state data and observation data. The gait phase in the state data is equal to the ratio of the time difference between the current moment and the heel strike event detected by the plantar force sensor to the gait period. The gait phase change rate in the state data is the reciprocal of the gait period. The velocity in the state data is the sum of the integral value of the acceleration collected by the inertial measurement unit and the correction amount of the hip joint angular velocity collected by the joint encoder. The slope in the state data is equal to the difference between the tilt angle between the horizontal and vertical components of the ground reaction force collected by the plantar force sensor and the pitch angle collected by the inertial measurement unit. The step height in the state data is equal to the larger of the difference between the foot trajectory height and the flat ground reference height and zero. The foot trajectory height is obtained by integrating the vertical component of the acceleration collected by the inertial measurement unit.

[0016] Optionally, in the above-described control method for a bipedal robot, the step of correcting the pre-trained unified reinforcement learning control strategy to select the optimal control parameters in the current state based on the current state correction value using a correction function, to obtain the current control parameters, includes:

[0017] The correction function analyzes the relationship between the speed, slope, and step height in the current state correction value and the corresponding threshold to determine the adjustment amount of each target control parameter; wherein, the target control parameters include hip joint extension torque, swing leg lift height, and ankle joint push-off torque.

[0018] Each of the target control parameters is adjusted by adding the adjustment amount, and the pre-trained unified reinforcement learning control strategy selects the corresponding target control parameters from the best control parameters in the current state, and outputs the current control parameters.

[0019] Optionally, in the above-described control method for a bipedal robot, before correcting the predicted state value at the current moment based on the deviation between the actual observed data and the predicted observed data at the current moment to obtain the corrected state value at the current moment, the method further includes:

[0020] The prediction error covariance at the current moment is analyzed by combining the prediction error covariance of the previous moment with the motion model of the bipedal robot.

[0021] The Kalman gain at the current time is calculated using the variance written by the prediction error at the current time.

[0022] The step of correcting the predicted state value at the current moment based on the deviation between the actual observed data and the predicted observed data at the current moment to obtain the corrected state value at the current moment includes:

[0023] The current state prediction value is added to the product of the current Kalman gain and the observation data deviation to obtain the current state correction value; wherein, the observation data deviation is the deviation between the current actual observation data and the predicted observation data.

[0024] Optionally, the above-described control method for bipedal robots further includes:

[0025] The prediction error covariance at the current time is updated based on the Kalman gain at the current time.

[0026] Optionally, in the above-described control method for a bipedal robot, controlling the current movement of the bipedal robot according to the current control parameters includes:

[0027] According to the current control parameters, a corresponding drive command is sent to the actuator interface of the bipedal robot to drive the movement of the bipedal robot; wherein, the drive command includes one of torque command, position command or velocity command.

[0028] A second aspect of this application provides a control device for a bipedal robot, comprising:

[0029] The observation data acquisition unit is used to acquire observation data through the target sensors of the bipedal robot body to obtain the actual observation data at the current moment; wherein, the target sensors include a joint encoder, an inertial measurement unit, and a plantar force sensor;

[0030] The state data prediction unit is used to predict the state data at the current moment by combining the state prediction value at the previous moment with the motion model of the bipedal robot, so as to obtain the state prediction value at the current moment.

[0031] An observation data prediction unit is used to predict the observation data based on the relationship between the observation data and the state data, using the state prediction value at the current moment, to obtain the predicted observation data at the current moment.

[0032] The state data correction unit is used to correct the state prediction value at the current time based on the deviation between the actual observation data and the predicted observation data at the current time, so as to obtain the state correction value at the current time.

[0033] The control parameter determination unit is used to select the best control parameters in the current state of the pre-trained unified reinforcement learning control policy and correct them based on the state correction value at the current time using a correction function, so as to obtain the current control parameters.

[0034] A control unit is used to control the movement of the bipedal robot according to the current control parameters.

[0035] Optionally, in the control device for the bipedal robot described above, the observation data prediction unit includes:

[0036] The observation data prediction subunit is used to input the current state prediction value into the observation function for calculation to obtain the predicted observation data for the current moment. The observation function consists of the relationship between various state data and observation data. The gait phase in the state data is equal to the ratio of the time difference between the current moment and the heel strike event detected by the plantar force sensor to the gait period. The gait phase change rate in the state data is the reciprocal of the gait period. The velocity in the state data is the sum of the integral value of the acceleration collected by the inertial measurement unit and the correction amount of the hip joint angular velocity collected by the joint encoder. The slope in the state data is equal to the difference between the tilt angle between the horizontal and vertical components of the ground reaction force collected by the plantar force sensor and the pitch angle collected by the inertial measurement unit. The step height in the state data is equal to the larger of the difference between the foot trajectory height and the flat ground reference height and zero. The foot trajectory height is obtained by integrating the vertical component of the acceleration collected by the inertial measurement unit.

[0037] Optionally, in the control device for the bipedal robot described above, the control parameter determination unit includes:

[0038] The adjustment analysis unit is used to analyze the adjustment amount of each target control parameter based on the relationship between the speed, slope, and step height in the current state correction value and the corresponding threshold through the correction function; wherein, the target control parameters include hip joint extension torque, swing leg lift height, and ankle joint push torque;

[0039] The parameter correction unit is used to add the adjustment amount of each of the target control parameters respectively, and the pre-trained unified reinforcement learning control strategy selects the target control parameters corresponding to the best control parameters in the current state and outputs the current control parameters.

[0040] Optionally, the control device for the bipedal robot described above further includes:

[0041] The error analysis unit is used to analyze the prediction error covariance at the current moment by combining the prediction error covariance of the previous moment with the motion model of the bipedal robot.

[0042] The gain calculation unit is used to calculate the Kalman gain at the current time using the variance of the prediction error at the current time.

[0043] The state data correction unit includes:

[0044] The state data correction subunit is used to add the current state prediction value to the product of the current Kalman gain and the observation data deviation to obtain the current state correction value; wherein, the observation data deviation is the deviation between the current actual observation data and the predicted observation data.

[0045] Optionally, the control device for the bipedal robot described above further includes:

[0046] An error update unit is used to update the prediction error covariance at the current time based on the Kalman gain at the current time.

[0047] Optionally, in the control device for the bipedal robot described above, the control unit includes:

[0048] The control subunit is used to send corresponding drive commands to the driver interface of the bipedal robot according to the current control parameters to drive the movement of the bipedal robot; wherein the drive command includes one of torque command, position command or speed command.

[0049] A third aspect of this application provides an electronic device, comprising:

[0050] Memory and processor;

[0051] The memory is used to store programs;

[0052] The processor is used to execute the program, which, when executed, is specifically used to implement the control method for the bipedal robot as described in any of the above.

[0053] The fourth aspect of this application provides a computer storage medium for storing a computer program, which, when executed by a processor, is used to implement the control method for a bipedal robot as described in any of the preceding claims.

[0054] This application provides a control method for a bipedal robot. It collects observation data using target sensors on the bipedal robot body to obtain the actual observation data at the current moment. These target sensors include a joint encoder, an inertial measurement unit (IMU), and a plantar force sensor. Data acquisition is achieved using these three minimal sensors, all present in the bipedal robot body, adapting to various bipedal robots and not limited by platform. Furthermore, the collected data is the robot's own data, not external environmental data, and therefore unaffected by external environmental influences. Then, by combining the previous moment's state prediction value with the bipedal robot's motion model, the current moment's state data is predicted to obtain the current moment's state prediction value. Based on the relationship between the observed data and the state data, the current moment's state prediction value is used to predict the observed data, resulting in the current moment's predicted observed data. Next, based on the deviation between the current moment's actual observed data and the predicted observed data, the current moment's state prediction value is corrected to obtain the current moment's corrected state value, thus obtaining the accurate current state of the robot. Finally, based on the current moment's state correction value, a correction function is used to select the optimal control parameters for the pre-trained unified reinforcement learning control strategy under the current state, obtaining the current control parameters. The movement of the bipedal robot is then controlled according to these current control parameters. Therefore, by adopting a unified reinforcement learning control strategy, the optimal control parameters can be accurately selected for the current state in various environments. Furthermore, a correction function is introduced to adapt and adjust the parameters to cope with sudden environmental changes, thus ensuring stable motion in various environments. This achieves a method for motion control based on local sensors of the bipedal robot collecting data from the robot itself. It is not limited by platform hardware conditions and is not affected by external environments, thus ensuring stable robot motion in various complex environments and is applicable to various bipedal robots. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0056] Figure 1 A flowchart illustrating a control method for a bipedal robot provided in this application embodiment;

[0057] Figure 2 A flowchart illustrating a method for obtaining current control parameters provided in this application embodiment;

[0058] Figure 3 A schematic diagram of the architecture of a control device for a bipedal robot provided in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of the architecture of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0062] This application provides a control method for a bipedal robot, such as... Figure 1 As shown, it includes the following steps:

[0063] S101. Collect observation data through the target sensor of the bipedal robot body to obtain the actual observation data at the current moment.

[0064] The target sensors include a joint encoder, an inertial measurement unit (IMU), and a plantar force sensor. Since the joint encoder, IMU, and plantar force sensor are the smallest sensors on the bipedal robot, they are present in bipedal robots across various scenarios. Therefore, the method provided in this application can be directly ported, thus possessing universality. Furthermore, the data collected by the joint encoder, IMU, and plantar force sensor is local data of the bipedal robot, such as speed and force, and therefore does not depend on or be affected by the external environment. Moreover, data can be collected rapidly without any delay in recognizing the external environment.

[0065] Optionally, the observation data collected by the target sensor may specifically include:

[0066] 1. Angles of each joint acquired by the joint encoder and angular velocity These are the main dynamic characteristics reflecting a robot's gait. This data serves as crucial input for inferring robot gait patterns and movement speed.

[0067] 2. Linear acceleration of the torso and legs of the bipedal robot collected by the IMU. and angular velocity This data is used to calculate the robot's overall speed and attitude (such as pitch angle), helping to determine whether it is in an uphill, downhill, or stairwell environment.

[0068] 3. Ground reaction force collected by plantar force sensor It is used to identify the support and swing phases. Specifically, this data helps identify key gait events (such as heel strike) and is combined with the IMU to estimate slope.

[0069] Since this application embodiment provides analysis using the Kalman filter algorithm, the observation data collected by the above-mentioned target sensors are used to construct vectors:

[0070]

[0071] The main functions of the observation vector in this application embodiment are reflected in the following aspects:

[0072] 1. As input for state estimation: That is, as input for the observation equation of the Kalman filter algorithm, specifically as input for the observation equation of the Extended Kalman Filter (EKF):

[0073]

[0074] Where h(x) is the observation equation; x(k) is the state vector; This represents observation noise. Therefore, the observation vector provides directly measurable data related to the state vector, which is used to continuously correct the predicted state.

[0075] 2. Critical event detection: Foot pressure It is used to detect gait events (heel strike, toe lift) to trigger a gait phase reset. Changes in peak acceleration of the IMU can be used to help detect steps or slope changes.

[0076] 3. Multi-source information fusion: A single sensor is often insufficient to stably estimate complex states. For example, velocity estimation relying solely on IMU integration will drift and needs to be corrected by combining encoder data; slope estimation relying solely on IMU attitude will be affected by body swaying and needs to be combined with ground force direction. The introduction of observation vectors allows EKF to optimize estimation through the complementarity of information from different sensors.

[0077] 4. Reduce the impact of delay and noise: Different modal data in the observation vector have different sampling frequencies and noise characteristics. EKF reduces the impact of noise from a single sensor through weighted fusion, thus alleviating the delay problem.

[0078] 5. Correction function The core input is the state estimation result, which is essentially driven by the observation vector. Therefore, the observation vector is the bridge between the "sensor layer" and the "control layer".

[0079] S102. By combining the state prediction value of the previous moment with the motion model of the bipedal robot, the state data of the current moment is predicted to obtain the state prediction value of the current moment.

[0080] In order to control the bipedal robot to adapt to the environment and move, it is necessary to construct a state vector that reflects the key gait of the bipedal robot and the state of the environment, and control the robot's movement through this state vector.

[0081] Alternatively, in another embodiment of this application, the constructed state vector may be as follows:

[0082]

[0083] in, The gait phase (0~1) describes the progress of one gait cycle; This is the gait phase change rate, used to reflect the step frequency; This represents the robot's forward velocity. Ground slope; This refers to the height of a step or staircase.

[0084] Since the motion model of the bipedal robot, specifically the kinematic / dynamic model, standardizes the robot's motion, the state vector at the next moment, i.e. the predicted state vector at the previous moment, can be estimated by combining the kinematic / dynamic model of the bipedal robot with the predicted state vector at the previous moment.

[0085] Optionally, in this embodiment, an Extended Kalman Filter (EKF) is used for state estimation; other Kalman filtering algorithms can also be used. The EKF is chosen primarily because: the relationship between gait and environmental variables is non-linear, and EKF can achieve near-optimal estimation with low computational cost through first-order linearization. The prediction step of EKF can alleviate sensor latency issues; for example, when the IMU update rate is low, the process model can provide phase and velocity predictions in advance. Compared to deep learning methods, EKF is easier to interpret and debug, making it suitable for robot safety control scenarios.

[0086] Specifically, by combining the state prediction value from the previous moment with the bipedal robot's motion model, the predicted state value for the current moment is:

[0087]

[0088] in, The current state prediction value is the predicted state value at the current time (k) based on the state prediction value at the previous time (k-1); F is the motion model of the bipedal robot. This is the predicted state value from the previous moment.

[0089] S103. Based on the relationship between the observation data and the state data, the observation data is predicted using the state prediction value at the current moment to obtain the predicted observation data at the current moment.

[0090] It's important to note that state data is used to control the robot's movement, which generates corresponding observation data. Therefore, there's a relationship between observation data and state data. Based on this relationship, we can estimate the observation data that corresponds to the current state prediction value when moving, i.e., the predicted observation data. We can then compare the predicted observation data with the actual observation data to correct the current state prediction value.

[0091] Optionally, in another embodiment of this application, the state data includes gait phase, gait phase change rate, speed, slope, step height, and foot trajectory height. Correspondingly, a specific real-time mode of step S103 includes:

[0092] The predicted state value at the current moment is input into the observation function for calculation to obtain the predicted observation data at the current moment.

[0093] The observation function consists of the relationship between various state data and observation data. Specifically, it consists of the relationship between gait phase, gait phase change rate, speed, slope, step height, and foot trajectory height and the observation data.

[0094] Specifically, the gait phase in the state data is equal to the ratio of the time difference between the current moment and the heel strike event detected by the plantar force sensor to the gait cycle, expressed as:

[0095]

[0096] Where t is the current time; The phase start point is the detected heel strike event. The gait cycle is determined by the time interval between two consecutive heel strikes. Specifically, the heel strike event is determined based on the ground reaction force collected by the plantar force sensor.

[0097] The gait phase change rate in the state data is the reciprocal of the gait period.

[0098] The velocity in the state data is the sum of the integral value of the acceleration collected by the inertial measurement unit (IMU) and the correction amount of the hip joint angular velocity collected by the joint encoder. In other words, the horizontal acceleration collected by the IMU is integrated and corrected by the encoder to ensure the stability of the robot's velocity estimation and avoid error accumulation during long-term walking. Therefore, the velocity is specifically expressed as:

[0099]

[0100] in, k1 represents the horizontal component of the acceleration acquired by the IMU; k1 is the correction coefficient. The change in hip joint angle, i.e., hip joint angular velocity, hip joint angle Data collected by the key encoder.

[0101] The slope in the condition data is equal to the difference between the tilt angle between the horizontal and vertical components of the ground reaction force collected by the plantar force sensor and the pitch angle collected by the inertial measurement unit. Therefore, the relationship between slope and the observed data is:

[0102]

[0103] in, This refers to the IMU's pitch angle, which can be obtained through analysis of the data it collects. and Ground reaction force collected by plantar force sensor The components are in the horizontal and vertical directions. This allows for accurate estimation of the environmental slope without adding a camera, which can then be used to correct joint torques.

[0104] The step height in the state data is equal to the larger of the difference between the foot trajectory height and the ground reference height and zero. This allows for identification of ascending a step or stairs when the foot height exceeds the reference height during the foot lift process. Therefore, the relationship between the step height in the state data and the observed data is as follows:

[0105]

[0106] in, This is the reference height for flat ground. The height of the foot trajectory is calculated by integrating the data from the IMU, and is therefore obtained by integrating the vertical component of the acceleration collected by the inertial measurement unit, specifically:

[0107]

[0108] in, The vertical component of the acceleration acquired by the IMU; This is the acceleration due to gravity.

[0109] S104. Based on the deviation between the actual observation data and the predicted observation data at the current moment, the predicted state value at the current moment is corrected to obtain the corrected state value at the current moment.

[0110] It should be noted that the predicted observation data at the current moment is based on the state prediction value at the current moment. Therefore, the deviation between the predicted observation data and the actual observation data at the current moment can reflect the deviation of the state prediction value at the current moment. Thus, based on the deviation between the actual observation data and the predicted observation data at the current moment, the state prediction value at the current moment can be corrected, thereby obtaining the corrected state value at the current moment.

[0111] Optionally, in another embodiment of this application, before performing step S104, the following is further performed:

[0112] By combining the prediction error covariance of the previous moment with the motion model of the bipedal robot, the prediction error covariance of the current moment is analyzed, and the Kalman gain of the current moment is calculated using the prediction error covariance of the current moment.

[0113] It should be noted that, in order to determine the error of the current predicted state data and accurately modify the predicted state data based on the error, it is necessary to first analyze the prediction error covariance at the current moment by combining the prediction error covariance of the previous moment and the motion model of the bipedal robot. Specifically, it can be expressed as:

[0114]

[0115] in, This represents the covariance of the prediction error at the previous time step. Let be the prediction error covariance at the current moment; Q is the covariance matrix.

[0116] Accordingly, in order to accurately correct the predicted state value based on the prediction error covariance at the current moment, it is necessary to calculate the Kalman gain at the current moment using the prediction error covariance at that moment, specifically:

[0117]

[0118] Where H is the observation function.

[0119] Accordingly, in another embodiment of this application, the specific implementation of step S104 includes:

[0120] The current state prediction is added to the product of the current Kalman gain and the observation data bias to obtain the current state correction.

[0121] Here, the observation data bias is the deviation between the actual observation data and the predicted observation data at the current moment, and is therefore specifically expressed as:

[0122]

[0123] Accordingly, in another embodiment of this application, after performing step S104, the following further steps are required:

[0124] The prediction error covariance at the current time is updated based on the Kalman gain at the current time.

[0125] Since the predicted state data has been corrected at this point, its prediction error has also been corrected to some extent. Therefore, in order to facilitate the next state prediction based on the current prediction error, it is necessary to update the prediction error covariance at the current moment, specifically as follows:

[0126]

[0127] Where I is the identity matrix, which is used to keep the dimension of the error covariance unchanged, and the matrix identity transformation is achieved by multiplying by the identity matrix.

[0128] S105. Based on the state correction value at the current moment, the pre-trained unified reinforcement learning control policy is modified by selecting the best control parameters under the current state through the correction function to obtain the current control parameters.

[0129] It should be noted that the strategies trained by reinforcement learning have strong generalization capabilities and can adapt well to motion control in complex environments. Therefore, in this embodiment, a reinforcement learning control strategy is used to select the motion at the current moment in the current environment, that is, to select the optimal control parameters to control the robot's motion. Since the robot may move in various different environments, such as flat surfaces, ramps, and stairs, building and training corresponding strategies for different environments separately would require environment identification and task switching or classifier recognition, which could easily lead to recognition errors or delays, resulting in instability. Therefore, in this embodiment, a unified reinforcement learning control strategy is used to select the optimal control parameters in the current state, that is, the state correction value at the current moment, to avoid the aforementioned problems.

[0130] However, while policies trained through reinforcement learning have strong generalization capabilities, they may become unstable in extreme environments (such as a suddenly appearing staircase). Therefore, in this embodiment, a correction function is added outside the policy to enable application in extreme environments and to quickly "fine-tune" actions using estimated environmental information, avoiding switching between multiple independent policies and improving robustness. Specifically, the current state correction value is used as the input to a pre-trained unified reinforcement learning control policy to select the optimal control parameters for the current state. The correction function then determines the abrupt changes in the environment based on the current state correction value, thereby determining the adaptive correction amount and fine-tuning the selected optimal control parameters.

[0131] Specifically, it can be expressed as:

[0132]

[0133] in, The analysis results of the general policy obtained from pre-training are the optimal control parameters selected under the current state. This is a correction function.

[0134] It should also be noted that, due to differences in the driving capabilities and sensor characteristics of different robot platforms, the specific correction magnitude and gain coefficient need to be obtained through experimental calibration or simulation optimization. Therefore, in this embodiment, the correction function is defined as an adjustable parameterized function, and developers can set its gain, proportional coefficient, and threshold according to the actual platform and requirements. The technical effect of this design is to ensure that the control framework has universality and scalability, rather than relying on a single fixed value.

[0135] Optionally, in another embodiment of this application, one specific implementation of step S105 is as follows: Figure 2 As shown, it includes:

[0136] S201. By using the correction function, the adjustment amount of each target control parameter is analyzed based on the relationship between the speed, slope, and step height in the current state correction value and the corresponding threshold.

[0137] The target control parameters include hip extension torque, swing leg lift height, and ankle push-off torque.

[0138] It should be noted that the abrupt change in the environment is reflected in the velocity of the current state correction value. ,slope and step height Therefore, the input to the correction function is the speed, slope, and step height from the current state correction value. To determine the degree of abrupt change, these values ​​are compared with their corresponding thresholds. Based on the comparison results, the adjustment amounts for the corresponding target control parameters are analyzed, allowing the system to adapt to environmental changes.

[0139] Since the hip extension torque, swing leg lift height, and ankle push torque are adapted to the current environment, they can walk stably in the current environment. Therefore, the hip extension torque, swing leg lift height, and ankle push torque are modified.

[0140] Optionally, the specific correction method can be: when When, it increases the hip joint extension torque, enhancing uphill ability; when When v < v_ref, increase the height of the swing leg to avoid tripping; when v < v_ref, increase the ankle joint pushing torque to maintain the target speed.

[0141] It's important to note that the greater the deviation between the predicted state value and the threshold, the larger the parameter adjustment. In other words, the correction scale for slope, step height, and speed is not a fixed constant, but rather parameterized based on the actual application environment and the robot's hardware characteristics. The basic principle is: when the estimated value exceeds the threshold, the correction amount gradually increases with the degree of exceedance. When the estimated value is below or close to the threshold, the correction amount approaches zero, meaning it approaches the point where no correction is needed.

[0142] S202. Add the adjustment amount of each objective control parameter respectively, select the corresponding objective control parameters from the best control parameters in the current state using the pre-trained unified reinforcement learning control strategy, and output the current control parameters.

[0143] S106. Control the movement of the current bipedal robot according to the current control parameters.

[0144] Optionally, one specific implementation of step S106 can be:

[0145] Optionally, in the control device for the bipedal robot described above, a corresponding drive command is sent to the driver interface of the bipedal robot according to the current control parameters to drive the movement of the bipedal robot; wherein, the drive command includes one of torque command, position command or speed command.

[0146] Specifically, the actuator interface of a robot varies across different robot hardware platforms, so the control parameters can be tailored accordingly. Therefore, it is necessary to issue commands in the appropriate form based on the actuator interface to drive the robot's movement. Typically, one of the following commands is issued: torque command, position command, or velocity command, depending on the actuator interface. Of course, other command formats can also be issued; this application does not limit this, meaning that the driving commands include, but are not limited to, torque commands, position commands, or velocity commands.

[0147] This application provides a control method for a bipedal robot. It collects observation data using target sensors on the bipedal robot body to obtain the actual observation data at the current moment. These target sensors include a joint encoder, an inertial measurement unit (IMU), and a plantar force sensor. Data collection is achieved using these three minimal sensors found on the bipedal robot body, adapting to various bipedal robots and not limited by platform. Furthermore, the collected data is the robot's own data, not external environmental data, and therefore is not affected by the external environment or subject to timing issues. Then, by combining the predicted state value from the previous moment with the bipedal robot's motion model, the current state data is predicted to obtain the predicted state value. Based on the relationship between the observed data and the state data, the predicted state value is used to predict the observed data to obtain the predicted observed data for the current moment. Next, based on the deviation between the actual observed data and the predicted observed data, the predicted state value is corrected to obtain the corrected state value, thus obtaining the accurate current state of the robot. Finally, based on the corrected state value, a correction function is used to select the optimal control parameters for the current state using a pre-trained unified reinforcement learning control strategy, obtaining the current control parameters. The movement of the bipedal robot is then controlled according to these current control parameters. Therefore, by adopting a unified reinforcement learning control strategy, the optimal control parameters can be accurately selected for the current state in various environments. Furthermore, a correction function is introduced to adapt and adjust the parameters to cope with sudden environmental changes, thus ensuring stable motion in various environments. This achieves a method for motion control based on local sensors of the bipedal robot collecting data from the robot itself. It is not limited by platform hardware conditions and is not affected by external environments, thus ensuring stable robot motion in various complex environments and is applicable to various bipedal robots.

[0148] This application provides a control device for a bipedal robot, which can specifically be a processing device for the bipedal robot. For example... Figure 3 As shown in the embodiment of this application, the control device for a bipedal robot includes:

[0149] The observation data acquisition unit 301 is used to acquire observation data through the target sensor of the bipedal robot body to obtain the actual observation data at the current moment.

[0150] The target sensors include a joint encoder, an inertial measurement unit, and a plantar force sensor.

[0151] The state data prediction unit 302 is used to predict the state data at the current moment by combining the state prediction value at the previous moment with the motion model of the bipedal robot, so as to obtain the state prediction value at the current moment.

[0152] The observation data prediction unit 303 is used to predict the observation data based on the relationship between the observation data and the state data, using the state prediction value at the current moment, to obtain the predicted observation data at the current moment.

[0153] The state data correction unit 304 is used to correct the state prediction value at the current time based on the deviation between the actual observation data and the predicted observation data at the current time, so as to obtain the state correction value at the current time.

[0154] The control parameter determination unit 305 is used to select the best control parameters in the current state of the pre-trained unified reinforcement learning control strategy and correct them based on the state correction value at the current time using a correction function, so as to obtain the current control parameters.

[0155] Control unit 306 is used to control the movement of the current bipedal robot according to the current control parameters.

[0156] Optionally, in another embodiment of the control device for a bipedal robot provided in this application, the observation data prediction unit includes:

[0157] The observation data prediction subunit is used to input the current state prediction value into the observation function for calculation, obtaining the predicted observation data for the current moment. The observation function consists of the relationships between various state data and observation data. The gait phase in the state data is equal to the ratio of the time difference between the current moment and the heel strike event detected by the plantar force sensor to the gait period. The gait phase change rate in the state data is the reciprocal of the gait period. The velocity in the state data is the sum of the integral value of the acceleration collected by the inertial measurement unit and the correction amount of the hip joint angular velocity collected by the joint encoder. The slope in the state data is equal to the difference between the tilt angle between the horizontal and vertical components of the ground reaction force collected by the plantar force sensor and the pitch angle collected by the inertial measurement unit. The step height in the state data is the larger of the difference between the foot trajectory height and the flat ground reference height and zero. The foot trajectory height is obtained by integrating the vertical component of the acceleration collected by the inertial measurement unit.

[0158] Optionally, in another embodiment of the control device for a bipedal robot provided in this application, the control parameter determination unit includes:

[0159] The adjustment analysis unit is used to analyze the adjustment amount of each target control parameter based on the relationship between the speed, slope, and step height in the current state correction value and the corresponding threshold, using a correction function. These target control parameters include hip extension torque, swing leg lift height, and ankle push-off torque.

[0160] The parameter correction unit is used to add the adjustment amount of each objective control parameter respectively. The pre-trained unified reinforcement learning control strategy selects the corresponding objective control parameters from the best control parameters in the current state and outputs the current control parameters.

[0161] Optionally, in another embodiment of the control device for a bipedal robot provided in this application, the device further includes:

[0162] The error analysis unit is used to analyze the prediction error covariance at the current moment by combining the prediction error covariance of the previous moment with the motion model of the bipedal robot.

[0163] The gain calculation unit is used to calculate the Kalman gain at the current time using the variance of the prediction error at the current time.

[0164] The state data correction unit in this embodiment includes:

[0165] The state data correction subunit is used to add the current state prediction value to the product of the current Kalman gain and the observation data bias to obtain the current state correction value. The observation data bias is the deviation between the actual observed data and the predicted observed data at the current time.

[0166] Optionally, in another embodiment of the control device for a bipedal robot provided in this application, the device further includes:

[0167] The error update unit is used to update the prediction error covariance at the current time based on the Kalman gain at the current time.

[0168] Optionally, in another embodiment of the bipedal robot control device provided in this application, the control unit includes:

[0169] The control subunit is used to send corresponding drive commands to the bipedal robot's actuator interface according to the current control parameters to drive the movement of the bipedal robot; wherein, the drive command includes one of torque command, position command or velocity command.

[0170] It should be noted that the specific working process of each unit provided in the above embodiments of this application can be referred to the implementation process of the corresponding steps in the above method embodiments, and will not be repeated here.

[0171] Another embodiment of this application provides an electronic device, such as... Figure 4 As shown, it includes:

[0172] Memory 401 and processor 402.

[0173] The memory 401 is used to store the program.

[0174] The processor 402 is used to execute the program stored in the memory 401. When the program is executed, it is specifically used to implement the control method of the bipedal robot provided in any of the above embodiments.

[0175] Another embodiment of this application provides a computer storage medium for storing a computer program, which, when executed by a processor, is used to implement the control method for a bipedal robot as provided in any of the above embodiments.

[0176] Computer storage media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0177] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0178] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A control method for a bipedal robot, characterized in that, include: The actual observation data at the current moment is obtained by collecting observation data through target sensors on the bipedal robot body; wherein, the target sensors include joint encoders, inertial measurement units, and plantar force sensors; By combining the state prediction value of the previous moment with the motion model of the bipedal robot, the state data of the current moment is predicted to obtain the state prediction value of the current moment. Based on the relationship between the observation data and the state data, the predicted state value at the current moment is used to predict the observation data, so as to obtain the predicted observation data at the current moment. Based on the deviation between the actual observation data and the predicted observation data at the current moment, the predicted state value at the current moment is corrected to obtain the corrected state value at the current moment. By using the correction function based on the state correction value at the current moment, the pre-trained unified reinforcement learning control policy is corrected to select the best control parameters under the current state, and the current control parameters are obtained. The movement of the bipedal robot is controlled according to the current control parameters.

2. The method according to claim 1, characterized in that, The method of predicting the observed data based on the relationship between the observed data and the state data, using the predicted state value at the current moment, to obtain the predicted observed data at the current moment, includes: The predicted state value at the current moment is input into the observation function for calculation to obtain the predicted observation data at the current moment. The observation function consists of the relationship between various state data and observation data. The gait phase in the state data is equal to the ratio of the time difference between the current moment and the heel strike event detected by the plantar force sensor to the gait period. The gait phase change rate in the state data is the reciprocal of the gait period. The velocity in the state data is the sum of the integral value of the acceleration collected by the inertial measurement unit and the correction amount of the hip joint angular velocity collected by the joint encoder. The slope in the state data is equal to the difference between the tilt angle between the horizontal and vertical components of the ground reaction force collected by the plantar force sensor and the pitch angle collected by the inertial measurement unit. The step height in the state data is equal to the larger of the difference between the foot trajectory height and the flat ground reference height and zero. The foot trajectory height is obtained by integrating the vertical component of the acceleration collected by the inertial measurement unit.

3. The method according to claim 1, characterized in that, The process involves using a correction function to adjust the pre-trained unified reinforcement learning control policy based on the current state correction value, selecting the optimal control parameters for the current state, and obtaining the current control parameters, including: The correction function analyzes the relationship between the speed, slope, and step height in the current state correction value and the corresponding threshold to determine the adjustment amount of each target control parameter; wherein, the target control parameters include hip joint extension torque, swing leg lift height, and ankle joint push-off torque. Each of the target control parameters is adjusted by adding the adjustment amount, and the pre-trained unified reinforcement learning control strategy selects the corresponding target control parameters from the best control parameters in the current state, and outputs the current control parameters.

4. The method according to claim 1, characterized in that, Before correcting the predicted state value at the current moment based on the deviation between the actual observed data and the predicted observed data at the current moment, and obtaining the corrected state value at the current moment, the method further includes: The prediction error covariance at the current moment is analyzed by combining the prediction error covariance of the previous moment with the motion model of the bipedal robot. The Kalman gain at the current time is calculated using the variance written by the prediction error at the current time. The step of correcting the predicted state value at the current moment based on the deviation between the actual observed data and the predicted observed data at the current moment to obtain the corrected state value at the current moment includes: The current state prediction value is added to the product of the current Kalman gain and the observation data deviation to obtain the current state correction value; wherein, the observation data deviation is the deviation between the current actual observation data and the predicted observation data.

5. The method according to claim 4, characterized in that, Also includes: The prediction error covariance at the current time is updated based on the Kalman gain at the current time.

6. The method according to claim 1, characterized in that, The step of controlling the movement of the bipedal robot according to the current control parameters includes: According to the current control parameters, a corresponding drive command is sent to the actuator interface of the bipedal robot to drive the movement of the bipedal robot; wherein, the drive command includes one of torque command, position command or velocity command.

7. A control device for a bipedal robot, characterized in that, include: The observation data acquisition unit is used to acquire observation data through the target sensors of the bipedal robot body to obtain the actual observation data at the current moment; wherein, the target sensors include a joint encoder, an inertial measurement unit, and a plantar force sensor; The state data prediction unit is used to predict the state data at the current moment by combining the state prediction value at the previous moment with the motion model of the bipedal robot, so as to obtain the state prediction value at the current moment. An observation data prediction unit is used to predict the observation data based on the relationship between the observation data and the state data, using the state prediction value at the current moment, to obtain the predicted observation data at the current moment. The state data correction unit is used to correct the state prediction value at the current time based on the deviation between the actual observation data and the predicted observation data at the current time, so as to obtain the state correction value at the current time. The control parameter determination unit is used to select the best control parameters in the current state of the pre-trained unified reinforcement learning control policy and correct them based on the state correction value at the current time using a correction function, so as to obtain the current control parameters. A control unit is used to control the movement of the bipedal robot according to the current control parameters.

8. The apparatus according to claim 7, characterized in that, The observation data prediction unit includes: The observation data prediction subunit is used to input the current state prediction value into the observation function for calculation to obtain the predicted observation data for the current moment. The observation function consists of the relationship between various state data and observation data. The gait phase in the state data is equal to the ratio of the time difference between the current moment and the heel strike event detected by the plantar force sensor to the gait period. The gait phase change rate in the state data is the reciprocal of the gait period. The velocity in the state data is the sum of the integral value of the acceleration collected by the inertial measurement unit and the correction amount of the hip joint angular velocity collected by the joint encoder. The slope in the state data is equal to the difference between the tilt angle between the horizontal and vertical components of the ground reaction force collected by the plantar force sensor and the pitch angle collected by the inertial measurement unit. The step height in the state data is equal to the larger of the difference between the foot trajectory height and the flat ground reference height and zero. The foot trajectory height is obtained by integrating the vertical component of the acceleration collected by the inertial measurement unit.

9. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program, which, when executed, is specifically used to implement the control method for the bipedal robot as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, is used to implement the control method for a bipedal robot as described in any one of claims 1 to 6.

Images