Humanoid robot control system, control method and storage medium

Abstract

The present invention discloses a humanoid robot control system, a control method and a storage medium. The humanoid robot control system includes a joint controller and a learning forward model containing multiple module units. Each module unit includes a state estimator, a responsibility predictor, a value function estimator, an expected controller and a responsibility predictor. The state estimator estimates the potential state information of the robot at the next moment; the responsibility predictor allocates the control contribution rate of each module unit; the value function estimator is used to evaluate the robot's next execution action; the expected controller of each module unit is established through the value function and the currently estimated potential weighted state information; the joint controller combines the final output and the motion trajectory instruction to generate a torque output, and controls each joint of the robot to operate in a corresponding order and amplitude. The present invention improves the control robustness of large observation noise and partial observation systems, and the system control performance is relatively good.

Description

Technical Field

[0001] The present invention relates to the field of robot control technology, and in particular to a humanoid robot control system, a control method and a storage medium. Background Art

[0002] The foundation of the most advanced robot control technology can be traced back to at least the classic control module selection and recognition model. In the modular structure of the control module selection and recognition model, each module usually has its own input and output, and the modules are coordinated and integrated through information transmission. This modular structure helps to improve the flexibility and adaptability of the system because each module can learn and adjust independently without affecting the entire system. For example, in visual perception, the modular structure of the control module selection and recognition model can be applied to tasks such as identifying the shape of an object, determining the position of an object, and detecting motion. Each module focuses on processing one of the tasks, and the entire system completes more complex perception tasks by integrating the output of each module. In short, the modular structure of the control module selection and recognition model is a structural design that decomposes complex perception tasks into multiple modules and realizes them through collaboration and integration between modules.

[0003] The joint degrees of freedom of bipedal robots are generally higher than those of common six-degree-of-freedom industrial robots. The dynamic model is very complex, including continuous variable dynamic systems, discrete event dynamic systems, and a hybrid system of the interaction between the two. Therefore, controlling the posture stability of bipedal robots is a difficult problem in the field of robot control. The modular structure of the control module selection and identification model can be used to solve nonlinear and non-stationary control problems. The control module selection and identification model, as a human motion control model and a control framework for humanoid robots, has great potential for the posture stability control of bipedal robots. Its flexible structure enables bipedal robots to control nonlinear and non-stationary environments. However, the existing control module selection and identification model is easily affected by observation noise, which will lead to problems such as reduced model performance, increased decision uncertainty, and difficulty in model training. When it is applied to partially observable systems, the characteristics of partially observable systems may cause the existing control module selection and identification model to be unable to effectively use input data for appropriate expert allocation and decision making, thereby limiting its application effect and performance in observable systems. Summary of the invention

[0004] The object of the present invention is to provide a humanoid robot control system, a control method and a storage medium to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A humanoid robot control system includes a learning forward model containing multiple module units, wherein the learning goal of the learning forward model is to minimize the weighted prediction error of each linear dynamic model, and is used to receive control feedback and observation state information, provide state information of the robot under various critical postures, and determine the regression parameters and deviation parameters of each module unit;

[0007] Each module unit includes: a state estimator, which estimates the potential state information of the robot at the next moment based on the initial state information of the robot or the currently estimated potential weighted state information combined with the output information of the learning forward model;

[0008] The responsibility predictor allocates the control contribution rate of each module unit according to the Bayesian rule. The control contribution rate is used to configure the weight of the potential state information output by each module unit. The potential weighted state information of the robot at the next moment is obtained by weighting the potential state information of each module unit.

[0009] The value function estimator is used to evaluate the robot's next action and output the estimated value function based on the current estimated potential weighted state information and the output information of the learned forward model;

[0010] The expected controller of each module unit is established by using the value function estimated by the value function estimator and the currently estimated potential weighted state information, and the expected controller is used to derive the control output;

[0011] The control contribution rate of the responsibility predictor is also used to configure the weight of the control output exported to each module unit, and the final output is obtained by the weighted sum of the control outputs of each module unit;

[0012] The joint controller combines the final output and motion trajectory instructions to generate torque output, control each joint of the robot to operate in the corresponding order and amplitude, and perform the specified action.

[0013] As a further solution of the present invention, in each module unit, the expression of the linear dynamic model is:

[0014] x(t+1)=A i x(t)+B i u(t)+c i +n(t);

[0015] The expression of the linear observation model is:

[0016] y(t)=H i x(t)+v(t);

[0017] The expression of the quadratic model is:

[0018]

[0019] Among them, x is the state information, y is the observed state information, n is the system noise, v is the observed noise, u is the final output of the controller, A i ∈R N×N is the state regression parameter of the linear dynamics model of the i-th module, and B i ∈R N×D The i-th module is the output regression parameter of the linear dynamics model, c i ∈R N is the bias parameter of the i-th module, H i ∈R L×N is the observation matrix of the i-th module, Q i ∈R N×N is the state deviation parameter of the quadratic cost function of the i-th module, R i ∈R D×D is the output deviation parameter of the quadratic cost function, r i (x(t), u(t)) is the quadratic cost function of the i-th module, i.e., the control feedback.

[0020] As a further solution of the present invention, the state estimator specifically adopts a linear state estimator, and its expression is:

[0021]

[0022] in, is the potential state information estimated by the i-th module, K i are the parameters of the state estimator in the i-th module.

[0023] As a further solution of the present invention, the control contribution rate assigned by the responsibility predictor to each module unit adopts a probability distribution λ i It means that its expression is:

[0024]

[0025] Where M is the set of module indicators, p(x(t)|y(1:t), i) is the likelihood probability of the i-th module, and P(i) is the prior probability of the i-th module;

[0026] The expression of potential weighted state information is:

[0027]

[0028] in, is the potential state estimated by the i-th module at time t.

[0029] As a further solution of the present invention, the value function estimator adopts a quadratic function to locally estimate the value function, and the expression of the value function is:

[0030]

[0031] Among them, P i Solving the Riccati equation below, the expression of the Riccati equation is:

[0032] 0=P i A i +A i TP i -P i B i R i -1B i TP i +Q i ;

[0033] x i v =-(Q i +P i A i )-1P i c i ;

[0034] Among them, Q i ∈R N×N is the state deviation parameter of the quadratic cost function of the i-th module;

[0035] The control output expression of the expected controller is:

[0036]

[0037] The final output expression is:

[0038] u(t)=∑ i∈M λ i (t)u i (t).

[0039] As a further solution of the present invention, the potential state information estimated by the state estimator specifically includes:

[0040] Position state: The forward learning model obtains the current position information of the robot detected by the sensor and sets it as the origin of the space. The state estimator estimates the potential position information of the robot at the next moment.

[0041] Speed ​​state: The forward model is learned to obtain the current movement rate of the robot at different joints detected by the sensor, and the state estimator estimates the posture state equation of the robot at the next moment;

[0042] Noisy state: Account for sensor errors, modeling errors, and / or noisy state to more accurately estimate the true state of the system.

[0043] As a further solution of the present invention, the step of the value function estimator evaluating the next action to be performed by the robot includes:

[0044] Target analysis: confirm the robot's current location information and clarify the next action to be performed;

[0045] Environmental assessment: Based on the environmental data information in the observation status information, check whether there is enough space ahead to perform the next action and whether the ground is stable;

[0046] Power and structure considerations: determine the actuators that need to move, the order and amplitude of the actuator movements to achieve efficient and stable execution;

[0047] Safety verification: simulate the action plan to determine whether the robot maintains stability during execution and avoids causing damage to the surrounding environment.

[0048] As a further solution of the present invention, the joint controller further includes a standing balance module, and the balance control output expression of the standing balance module is:

[0049]

[0050] in,

[0051] Where x0 is the horizontal displacement of the pressure center, is the horizontal velocity of the pressure center, z is the vertical position of the pressure center, g is the gravitational acceleration, x is cp is the ground action point where the robot can step on and stop completely, FootEdge1 is the distance from the initial position of the pressure center to the front edge of the support area, the support area refers to the set of CP points, FootEdge2 is the distance from the initial position of the pressure center to the rear edge of the support area, τ 躯干min is the minimum torque produced by the trunk relative to the center of mass, τ 躯干max is the maximum torque produced by the trunk relative to the center of mass, is the maximum negative acceleration of the trunk relative to the center of mass in the Z direction, where the Z direction is the opposite direction of gravity, τ 躯干 is the generalized torque of the trunk joint, x CoP is the location of the center of pressure.

[0052] A humanoid robot control method, based on the humanoid robot control system, comprises:

[0053] By iteratively calculating the responsibility signal and updating the parameters, the learning goal of the learning forward model is achieved, minimizing the weighted prediction error of each linear dynamics model;

[0054] The state estimator estimates the potential state information of the robot at the next moment based on the robot's initial state information or the currently estimated potential weighted state information and the output information of the learning forward model;

[0055] The control contribution rate of each module unit is allocated according to the Bayesian rule. The control contribution rate is used to configure the weight of the potential state information output by each module unit. The potential weighted state information of the robot at the next moment is obtained by weighting the potential state information of each module unit.

[0056] Evaluate the robot's next action, output an estimated value function based on the current estimated potential weighted state information and the output information of the learned forward model;

[0057] The expected controller of each module unit is established through the value function and the currently estimated potential weighted state information, and the expected controller is used to derive the control output; the control contribution rate is also used to configure the weight of the control output derived from each module unit, and the final output is obtained by the weighted sum of the control outputs of each module unit;

[0058] Combine the final output and motion trajectory instructions to generate torque output, control the robot's joints to run in the corresponding order and amplitude, and perform the specified action.

[0059] The storage medium stores computer instructions, and when the computer instructions are executed by the processor, the above control method is executed.

[0060] Compared with the prior art, the beneficial effects of the present invention are as follows: the linear dynamics model in the module unit of the present invention takes into account the system noise, the linear observation model takes into account the observation noise, the control robustness to large observation noise and partial observation system is improved, and the system control performance is relatively good;

[0061] When the biped robot performs a specified action, the learning forward model automatically switches the module units based on the weighted responsibility signal, so that the biped robot can maintain a good balance while completing the specified action.

[0062] In the invention, each module unit is represented by a linear dynamic model, and the parameters of the desired controller can be solved. In specific control, the linear dynamic model of the module unit is switched to approximate the nonlinear and non-stationary dynamics, and the nonlinear cost function is approximated by switching the quadratic model. Compared with the method of using complex nonlinear functions to represent each control module in the prior art, the control system of the invention is relatively simpler and more practical, and is suitable for driving rapid motion.

[0063] The invented joint controller also has a standing balance module, which performs coordinated control to prevent the bipedal robot from having its balance stability greatly affected by a large change in its center of gravity, so that the bipedal robot always remains in a stable state and does not fall over. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Schematic diagram of the humanoid robot control system of the present invention.

[0065] Figure 2 It is a control flow chart of the humanoid robot control method of the present invention.

[0066] Figure 3 4 is a diagram showing the relationship between the phase of the periodic pattern generator and the posture of the biped robot in Example 1 of the present invention.

[0067] Figure 4 This is a phase trajectory diagram of the periodic pattern generator in Example 1 of the present invention.

[0068] Figure 5 This is a pitch angle trajectory diagram of the gyro sensor in Example 1 of the present invention.

[0069] Figure 6 This is a pitch angle trajectory diagram of the gyro sensor in Example 2 of the present invention.

[0070] Figure 7 Schematic diagram of the center of mass and CP point of the biped robot of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] See also Figure 1, a humanoid robot control system includes a learning forward model containing multiple module units. The learning goal of the learning forward model is to minimize the weighted prediction error of each linear dynamic model, which is used to give the state information of the robot under various critical postures and determine the regression parameters and deviation parameters of each module unit. Each module unit includes a state estimator, a responsibility predictor and a value function estimator, wherein the state estimator estimates the potential state information of the robot at the next moment according to the potential weighted state information currently estimated by the robot and the output information of the learning forward model; the responsibility predictor allocates the control contribution rate of each module unit according to the Bayesian rule, and the control contribution rate is used to configure the weight of the potential state information output by each module unit. Through the The potential weighted state information is weighted and obtained to obtain the potential weighted state information of the robot at the next moment; the value function estimator is used to evaluate the robot's next execution action, and outputs the estimated value function based on the current estimated potential weighted state information and the output of the learning forward model; the expected controller of each module unit is established through the value function estimated by the value function estimator and the current estimated potential weighted state information, and the expected controller is used to derive the control output; the control contribution rate of the responsibility predictor is also used to configure the weight of the control output derived from each module unit, and the final output is obtained by the weighted sum of the control outputs of each module unit; the joint controller, combining the final output and the motion trajectory instruction, generates a torque output, controls each joint of the robot to operate in the corresponding order and amplitude, and performs the specified action.

[0073] The present invention approximates nonlinear and non-stationary dynamics by switching the linear dynamics model, and approximates the nonlinear cost function by switching the quadratic model, wherein the expression of the linear dynamics model is:

[0074] x(t+1)=A i x(t)+B i u(t)+c i +n(t) (1);

[0075] The expression of the linear observation model is:

[0076] y(t)=H i x(t)+v(t) (2);

[0077] The expression of the quadratic model (i.e., quadratic cost function) is:

[0078]

[0079] Among them, x is the state information, y is the observed state information, n is the system noise, n~N(0,∑ x ), N(0,∑ x ) indicates that the mean is zero and the system noise covariance is ∑ xGaussian distribution, v is the observation noise, v~N(0,∑ y ), N(0,∑ y ) indicates that the mean is zero and the observation noise covariance is ∑ y Gaussian distribution, u is the final output of the controller, A i ∈R N×N is the state regression parameter of the linear dynamics model of the i-th module, B i ∈R N×D is the output regression parameter of the linear kinetic model of the i-th module, c i ∈R N is the bias parameter of the i-th module, H i ∈R L×N is the observation matrix of the i-th module, Q i ∈R N×N is the state deviation parameter of the quadratic cost function of the i-th module, R i ∈R D×D is the output deviation parameter of the quadratic cost function of the i-th module, r i (x(t), u(t)) is the quadratic cost function of the i-th module.

[0080] In the present invention, as a learning forward model, its learning goal is to minimize the weighted prediction error of each linear dynamics model. When each linear dynamics model is in the linear form x i (t+1)=W i Z(t) represents, then the regression parameter W i The expected value of can be derived as:

[0081]

[0082] Where Z = [x T u T 1] T , T is the time parameter, <g> i (T) represents the h function relative to the responsibility signal λ i The weighted average value of (T) is expressed as:

[0083]

[0084] By iteratively calculating the responsibility signal and updating the parameters, the probability is improved to the suboptimal level.

[0085] In the present invention, the state estimator estimates the potential state information of the linear dynamics model at the next moment based on the potential weighted state information currently estimated by the robot and the output information of the learning forward model. The present invention uses a linear state estimator, and its expression is:

[0086]

[0087] in, is the mean of the potential state information estimated by the i-th module at time t+1 based on the potential state information at time t, is the potential state information estimated by the i-th module at time t+1, K i are the parameters of the state estimator in the ith module (obtained by solving the linear optimal estimation problem).

[0088] In the present invention, the control contribution rate assigned by the responsibility predictor to each module unit is calculated using the probability distribution λ i Represents, also known as "responsibility signal", according to Bayes' rule, the responsibility signal λ i The expression is:

[0089]

[0090] Where M is the set of module indicators, p(x(t)|y(1:t), i) is the likelihood probability of the i-th module, and P(i) is the prior probability of the i-th module;

[0091] The potential weighted state information is obtained by weighting the potential state information of each module unit. The expression of the potential weighted state information is:

[0092]

[0093] in, is the potential state information estimated by the i-th module at time t.

[0094] In the present invention, the value function estimator derives the desired controller by locally solving the linear quadratic optimal control problem. Since the present invention approximates nonlinear and non-stationary dynamics through multiple linear dynamic models in formula (1), and approximates the nonlinear cost function through the quadratic cost function in formula (3), the value function estimator of the present invention uses a quadratic function to locally estimate the value function, and the expression of the value function is:

[0095]

[0096] Among them, P i Solving the Riccati equation below, the expression of the Riccati equation is:

[0097] 0=P i A i +A i T P i -P i B i R i -1B i T P i +Q i (11);

[0098] x i v =-(Q i +P i A i ) -1 P i c i (12);

[0099] Among them, Q i ∈R N×N is the state deviation parameter of the i-th module of the quadratic cost function;

[0100] In the present invention, the value function estimated by formula (10) is used to derive the expected controller of the i-th module, and the expression of its control output is:

[0101]

[0102] The final output is obtained by weighting the control output of each module unit. The expression of the final output is:

[0103] u(t)=∑ i∈M λ i (t)u i (t) (14).

[0104] In the present invention, the potential state information estimated by the state estimator specifically includes:

[0105] Position state: The forward learning model obtains the current position information of the robot detected by the sensor and sets it as the origin of the space. The state estimator estimates the potential position information of the robot at the next moment.

[0106] Speed ​​state: The forward model is learned to obtain the current movement rate of the robot at different joints detected by the sensor, and the state estimator estimates the posture state equation of the robot at the next moment;

[0107] Noise state: Account for sensor errors, modeling errors, and noise state to more accurately estimate the true state of the system.

[0108] In the present invention, the value function estimator is used to evaluate the next action of the robot. Based on the currently estimated potential weighted state information and the output of the learning forward model, the estimated value function is output. The step of evaluating the next action of the robot includes:

[0109] Target analysis: confirm the robot's current position information (i.e., initial state information or currently estimated potential weighted state information) and clarify the next action to be performed;

[0110] Environmental assessment: Based on the environmental data information in the observation status information, check whether there is enough space ahead to perform the next action and whether the ground is stable;

[0111] Dynamic and structural considerations: determine the actuators that need to move (such as joints and drives), the order and amplitude of the actuator movements, to achieve efficient and stable execution;

[0112] Safety verification: Simulate action plans to determine whether the robot remains stable during execution to avoid causing damage to the surrounding environment.

[0113] For further solutions, see Figure 7 When the biped robot is performing a load-bearing operation, after the biped robot effectively lifts a heavy object, the body center of gravity of the biped robot may change instantly, and the balance stability is greatly affected. In order to prevent the biped robot from tipping over during the load-bearing operation, the joint controller of the present invention further includes a standing balance module. The balance control output expression of the standing balance module is:

[0114]

[0115] in,

[0116] Where x0 is the horizontal displacement of the pressure center, is the horizontal velocity of the pressure center, z is the vertical position of the pressure center, g is the gravitational acceleration, x is cp is the ground action point where the robot can step on and stop completely, referred to as CP point, FootEdge1 is the distance from the initial position of the pressure center to the front edge of the support area, FootEdge2 is the distance from the initial position of the pressure center to the rear edge of the support area, the support area refers to the set of CP points, τ 躯干min is the minimum torque produced by the trunk relative to the center of mass, τ 躯干max is the maximum torque produced by the trunk relative to the center of mass, is the maximum negative acceleration of the trunk relative to the center of mass in the Z direction, where the Z direction is the opposite direction of gravity, τ 躯干 is the generalized torque of the trunk joint, x CoP is the location of the center of pressure.

[0117] According to the CP point and the body posture of the biped robot, the standing balance control state is specifically divided into three states: zero state, falling state and stable state. The zero state is the initial state of the biped robot, the horizontal displacement and velocity of the pressure center are both zero, in this state, the stability margin of the biped robot is the largest; the falling state is when the CP point is outside the support area, the body of the biped robot may tilt to the initial state, the pressure center may not have returned to the zero position, the horizontal velocity of the pressure center is not equal to zero, and the stability margin of the robot is a negative value; the stable state is when the CP point is in the support area, the body and legs of the biped robot have tilted to the initial state, the pressure center may not have returned to the zero position, the horizontal velocity of the pressure center is not equal to zero, and the stability margin of the robot is a positive value.

[0118] When the center of gravity of the bipedal robot changes instantly, if the bipedal robot is in a falling state, the CP point is outside the support domain. Since the ground reaction force relative to the center of mass of the robot is not enough to recover the robot, even if the pressure center is adjusted to the foot, the horizontal movement of the center of mass cannot be stopped. Simply adjusting the position of the pressure center is not enough to stop the robot. In this case, the bipedal robot needs as large a trunk torque as possible to bring the pressure center to the foot and lower the height of the center of mass as quickly as possible, that is, maximum capacity control, so that the robot can recover from the tipping state to the stable state before the center of mass leaves the support domain under the condition that the mechanical constraints allow; that is, through the control of the above-mentioned standing balance module, the bipedal robot will reach a stable state, and the bipedal robot will not tip over.

[0119] See also Figure 2 ,The humanoid robot control method is based on the aforementioned humanoid robot control system, including: estimating the current posture information of the robot according to the observed state information;

[0120] According to the observed state information of the previous step, combined with the current posture information of the robot, the potential state information of the dynamics is estimated;

[0121] Based on the current potential state information and the current posture information, according to the Bayesian rule, a corresponding control contribution rate is assigned to each module unit. The control contribution rate is used to configure the weight of the potential state information output by each module unit, and the potential weighted state information is obtained by weighting the potential state information of each module unit.

[0122] Evaluate the robot's next action and select the action with the highest value as the optimal action direction, outputting the estimated value function based on the potential weighted state information;

[0123] The expected controller of each module unit is established through the value function, and the expected controller is used to derive the control output;

[0124] The control contribution rate is also used to configure the weight of the control output exported to each module unit. The final output is obtained by weighting the control output of each module unit to control the robot to perform actions.

[0125] The storage medium stores computer instructions, and when the computer instructions are executed by the processor, the aforementioned humanoid robot control method is executed.

[0126] Embodiment 1 is to take a humanoid biped robot performing a squatting action as an example for explanation. A basic squatting action is provided by a periodic pattern generator. The periodic pattern generator outputs a simple sinusoidal trajectory to the robot's joint controller (such as a proportional derivative PD controller). The frequency of the squatting action is 0.2 Hz. Since the legs of the biped robot will bend and stretch periodically, a complex control law is required to prevent it from tipping over and falling. The humanoid robot control system of the present invention controls the dynamic model of the humanoid robot so that it can better maintain balance during the squatting action. The dynamic model of the humanoid robot is specifically approximated by a double-link robot with a foot model.

[0127] The periodic pattern generator outputs the desired sinusoidal trajectory to the joint controller of the humanoid robot. The input state equations related to the hip, knee, and ankle joints are:

[0128]

[0129] Among them, θ 躯干 is the relative angle of the torso, θ 脚踝 is the relative angle of the ankle, θ 陀螺仪 is the relative angle of the gyroscope, is the relative angular velocity of the trunk, is the relative angular velocity of the ankle, is the relative angular velocity of the gyroscope.

[0130] The output of the forward learning model of the present invention is the relative desired angle between the trunk and the ankle, which is expressed as: in, is the relative desired angle of the torso, is the relative desired angle of the ankle.

[0131] These relative desired angles are added to the required joint angles generated by the periodic pattern generator. To follow the desired joint angles, the torque output of each joint is derived by a proportional derivative PD controller, which is expressed as:

[0132] τ=-K(qq d );

[0133] in, represents the position and velocity of the joint angle, q d is the expected trajectory, τ=[τ 躯干 τ 髋关节 τ 膝关节 τ 脚踝 ] is the torque command of the robot actuator, K is the gain matrix of the proportional derivative PD controller, q 躯干 represents the generalized angle of the trunk joint, q 髋关节 represents the generalized angle of the hip joint, q 膝关节 represents the generalized angle of the knee joint, q 脚踝 represents the generalized angle of the ankle joint, represents the generalized angular velocity of the trunk joint, represents the generalized angular velocity of the hip joint, represents the generalized angular velocity of the knee joint, represents the generalized angular velocity of the ankle joint, τ 躯干 represents the generalized torque of the trunk joint, τ 髋关节 represents the generalized torque of the hip joint, τ 膝关节 represents the generalized torque of the knee joint, τ 脚踝 represents the generalized torque of the ankle joint.

[0134] The desired controller in each module unit aims to make the pitch angle of the gyro sensor mounted on the robot hip close to zero, i.e., θ 陀螺仪 = 0, the quadratic cost function is shown in formula (3), where the state deviation parameter is set to Q i =diag{0,0,1,0,0,1} to minimize the gyro angle and gyro angular velocity; the output deviation parameter is set to R i =diag{0.01,0.01}(i=1,2,3) to minimize the control cost.

[0135] In the process of implementing the squatting action, the control method of the humanoid robot is specifically as follows:

[0136] S1. The forward learning model prepares three module units corresponding to the three critical postures of the squat action, and gives the state information of the robot in the three critical postures. The posture information expression of the robot is:

[0137] Q=[q 躯干 q 髋关节 q 膝关节 q 脚踝 ] T ;

[0138] S2. The state estimator estimates the potential posture information of the robot's linear dynamics model at the next moment based on the robot's initial state information or the currently estimated potential weighted state information and the output information of the learning forward model, specifically including:

[0139] Position state: The forward learning model obtains the current position information of the robot detected by the sensor and sets it as the origin of the space. The state estimator estimates the potential position information of the robot at the next moment.

[0140] Speed ​​state: The forward model is learned to obtain the current movement rate of the robot at different joints detected by the sensor, including the angular velocity of the trunk, hip joint, knee joint and ankle. The expression is:

[0141]

[0142] Combined with the learning forward model, the potential posture state equation of the robot at the next moment is output:

[0143]

[0144] Noise state: Takes into account sensor errors, modeling errors, and noise state to more accurately estimate the true state of the system.

[0145] S3, the responsibility predictor assigns a corresponding control contribution rate (also called responsibility signal) to each module unit according to the Bayesian rule. The control contribution rate is used to configure the weight of the potential state information output by each module unit, and the potential weighted state information of the robot at the next moment is obtained by weighting the potential state information of each module unit;

[0146] The expression of the responsibility signal is:

[0147] Among them, p(x(t)|y(1:t),i) represents the likelihood probability of the i-th module, P(i) represents the prior probability of the i-th module, and the expression of the likelihood probability of the i-th module p(x(t)|y(1:t),i) is:

[0148] p(x(t)∣y(t),i)∝p(y(t)∣x(t|t-1),i)p(x(t|t-1),i),

[0149] therefore,

[0150] Among them, e i (t) = y(t) - H i x i (t|t-1),

[0151]

[0152] d i (t) = x i (t)-{A i x i (t-1)+B i u(t-1)+c i };

[0153] Among them, e i (t) is the innovation error.

[0154] S4. The value function estimator evaluates the next execution action based on the currently estimated potential weighted state information and the output of the learned forward model, and outputs the estimated value function. The steps for evaluating the next execution action are:

[0155] Goal analysis: confirm the current position of the robot (i.e. the initial state information or the currently estimated potential weighted state information) and clarify the next action plan;

[0156] Environmental assessment: Based on the environmental data information in the observation status information, check whether there is enough space in the front space to perform the next action and check whether the ground is stable;

[0157] Dynamic and structural considerations: determine the actuators that need to move (such as joints and drives), the order and amplitude of the actuator movements, to achieve efficient and stable execution;

[0158] Safety verification: Simulate action plans to determine whether the robot is stable during execution to avoid causing damage to the surrounding environment.

[0159] In the specific application process, the learning forward model of the present invention prepares three module units corresponding to the simulated squatting action, gives the state information of the robot in three critical postures, and uses smooth Gaussian random input for φ=0, The environmental data information around the critical posture corresponding to the π phase is sampled, and the regression parameters and deviation parameters of each module unit are determined. In this embodiment, the center of mass height of the humanoid robot in the standing position is about 0.98m, and the natural frequency of the pendulum model represented by the center of mass and the pressure center is 0.504Hz. According to the simulation results, the time constant parameter of time continuity is selected to be 15 times the natural frequency, specifically:

[0160] S5, based on the currently estimated potential weighted state information and the output information of the learning forward model, combined with the value function estimated by the value function estimator output, establish the expected controller of the three module units;

[0161] The control contribution rate of the responsibility predictor is also used to configure the weight of the control output exported to each module unit. The final output is obtained by weighting the control output of each module unit. The expression is:

[0162]

[0163] S6. Based on the final output, the joint controller combines the motion trajectory instructions to generate torque output, and controls each joint of the robot to operate in the corresponding order and amplitude, thereby achieving an efficient and stable squatting action.

[0164] Figure 4 is the phase trajectory of the periodic pattern generator, Figure 5 is the pitch angle trajectory of the gyro sensor, wherein the solid line represents the control result of the humanoid robot control system of the present invention, and the pitch angle of the gyro sensor is approximately maintained at within the range, the robot will not fall down within this range; the dotted line represents the control result of using only the first module unit, and at about t=3s, the robot using only a single module unit will fall backwards; the present invention uses multiple module units to perform a squatting action, successfully balances the humanoid robot during the squatting action, and ensures the stability of the robot during the action; by using the humanoid robot control system of the present invention, the robot can maintain balance and perform a squatting action at a frequency of up to 1.5Hz.

[0165] Example 2, taking the humanoid robot performing a load-bearing operation as an example, in order to realize the load-bearing operation, the present invention uses two module units in the learning forward model, and the two module units correspond to the two states of the humanoid robot being subjected to the payload being absent and payload being present. Similarly, the smoothed Gaussian random input is used again to sample the environmental data information within the set range, and the regression parameters and deviation parameters of each module unit are determined. For the selection of the module unit, it is assumed that the time of the responsibility signal is continuous, and the same time constant parameter as that for realizing the squatting action is selected to perform the load-bearing operation, specifically:

[0166] In this task, each desired controller attempts to balance the pitch angle of the gyroscope. Therefore, this embodiment uses the same quadratic cost function as the squat action as shown in formula (3), where the state deviation parameter is set to Q i =diag{0,0,1,0,0,1}, the output deviation parameter is set to R i =diag{0.01,0.01}, i=1,2.

[0167] In the process of realizing load-bearing operation, the control method of the humanoid robot is specifically as follows:

[0168] S1. Learning the forward model prepares two module units corresponding to the two critical postures of load-bearing operation, and gives the state information of the robot in the two critical postures. The posture information expression of the robot is:

[0169] Q=[q 躯干 q 髋关节 q 膝关节 q 脚踝 ] T ;

[0170] The S2 state estimator estimates the potential posture information of the robot's linear dynamics model at the next moment based on the robot's initial state information or the currently estimated potential weighted state information, combined with the output information of the learning forward model, including:

[0171] Position state: The forward learning model obtains the current position information of the robot detected by the sensor and sets it as the origin of the space. The state estimator estimates the potential position information of the robot at the next moment.

[0172] Speed ​​state: The forward model is learned to obtain the current movement rate of the robot at different joints detected by the sensor, including the angular velocity of the trunk, hip joint, knee joint and ankle. The expression is:

[0173]

[0174] Combined with the learning forward model, the potential posture state equation of the robot at the next moment is output:

[0175]

[0176] Noise state: Takes into account sensor errors, modeling errors, and noise state to more accurately estimate the true state of the system.

[0177] S3, the responsibility predictor assigns a corresponding control contribution rate (also called responsibility signal) to each module unit according to the Bayesian rule. The control contribution rate is used to configure the weight of the potential state information output by each module unit, and the potential weighted state information at the next moment is obtained by weighting the potential state information of each module unit;

[0178] The expression of the responsibility signal is:

[0179] S4. The value function estimator evaluates the next action based on the currently estimated potential weighted state information and the output of the learned forward model, and outputs the estimated value function. The steps for evaluating the next action are:

[0180] Goal analysis: confirm the current position of the robot (i.e. the initial state information or the currently estimated potential weighted state information) and clarify the next action plan;

[0181] Environmental assessment: Based on the environmental data information in the observation status information, check whether there is enough space in the front space to perform the next action and whether the ground is stable;

[0182] Dynamic and structural considerations: determine the actuators that need to move (such as joints and drives), the order and amplitude of the actuator movements to achieve efficient and stable movements;

[0183] Safety verification: Simulate action plans to determine whether the robot is stable during execution to avoid causing damage to the surrounding environment.

[0184] S5, based on the currently estimated potential weighted state information and the output information of the learning forward model, combined with the value function estimated by the value function estimator output, establish the expected controller of the two module units;

[0185] The control contribution rate of the responsibility predictor is also used to configure the weight of the control output exported to each module unit. The final output is obtained by weighting the control output of each module unit. The expression is:

[0186]

[0187] S6. Based on the final output, the joint controller combines the motion trajectory instructions to generate torque output, control the various joints and drivers of the robot to operate in a corresponding order and amplitude, and thus control the robot to achieve load-bearing operation.

[0188] Figure 6 The figure shows the pitch angle of the gyro sensor. A payload is applied to the robot's tray at t = 0 seconds. The solid line represents the control result of the humanoid robot control system of the present invention, and the dotted line represents the control result of a single module unit. The robot with a single module unit falls forward at about t = 3 seconds. The pitch angle of the gyro sensor is successfully maintained at less than 0 seconds when the humanoid biped robot is controlled by the humanoid robot control system proposed by the present invention. In the first stage, the learning forward model selected the first module unit, which controlled the environment well without the load; after the payload was applied, the second module was automatically selected and applied, which successfully controlled the environment when the payload was applied. Obviously, during the humanoid robot's load-bearing operation, the robot cannot successfully maintain balance without redundant module switching.

[0189] The present invention uses a humanoid robot control system with a state estimator to estimate the appropriate number of module units by learning a forward model; the parameters of each module unit can be estimated from the generated data using Gaussian random input, and then the embodiment of the present invention implements a control method based on the humanoid robot control system on the humanoid robot, which can efficiently stabilize the robot when performing squatting movements and load-bearing operations.< / g>

Claims

1. A humanoid robot control system, characterized in that: It includes a learning forward model containing multiple module units, the learning goal of the learning forward model is to minimize the weighted prediction error of each linear dynamics model, which is used to receive control feedback and observation state information, give the state information of the robot under various critical postures, and determine the regression parameters and deviation parameters of each module unit; Each module unit includes: a state estimator, which estimates the potential state information of the robot at the next moment based on the initial state information of the robot or the currently estimated potential weighted state information combined with the output information of the learning forward model; The responsibility predictor allocates the control contribution rate of each module unit according to the Bayesian rule. The control contribution rate is used to configure the weight of the potential state information output by each module unit. The potential weighted state information of the robot at the next moment is obtained by weighting the potential state information of each module unit. The value function estimator is used to evaluate the robot's next action and output the estimated value function based on the current estimated potential weighted state information and the output information of the learned forward model; The expected controller of each module unit is established by using the value function estimated by the value function estimator and the currently estimated potential weighted state information, and the expected controller is used to derive the control output; The control contribution rate of the responsibility predictor is also used to configure the weight of the control output exported to each module unit, and the final output is obtained by the weighted sum of the control outputs of each module unit; The joint controller combines the final output and motion trajectory instructions to generate torque output, control each joint of the robot to operate in the corresponding order and amplitude, and perform the specified action.

2. The humanoid robot control system according to claim 1, characterized in that: In each module unit, the expression of the linear dynamic model is: x(t+1)=A i x(t)+B i u(t)+c i +n(t); The expression of the linear observation model is: y(t)=H i x(t)+v(t); The expression of the quadratic model is: Among them, x is the state information, y is the observed state information, n is the system noise, v is the observed noise, u is the final output of the controller, A i ∈R N×N is the state regression parameter of the linear dynamics model of the i-th module, B i ∈R N×D is the output regression parameter of the linear kinetic model of the i-th module, c i ∈R N is the bias parameter of the i-th module, H i ∈R L×N is the observation matrix of the i-th module, Q i ∈R N×N is the state deviation parameter of the quadratic cost function of the i-th module, R i ∈R D×D is the output deviation parameter of the quadratic cost function of the i-th module, r i (x(t), u(t)) is the quadratic cost function of the i-th module.

3. The humanoid robot control system according to claim 2, characterized in that: The state estimator specifically adopts a linear state estimator, and its expression is: in, is the potential state information estimated by the i-th module, K i are the parameters of the state estimator in the i-th module.

4. The humanoid robot control system according to claim 3, characterized in that: The control contribution rate assigned by the responsibility predictor to each module unit adopts the probability distribution λ i It means that its expression is: Where M is the set of module indicators, p(x(t)|y(1:t), i) is the likelihood probability of the i-th module, and P(i) is the prior probability of the i-th module; The expression of potential weighted state information is: in, is the potential state estimated by the i-th module at time t.

5. The humanoid robot control system according to claim 4, characterized in that: The value function estimator uses a quadratic function to locally estimate the value function, and the expression of the value function is: Among them, P i Solving the Riccati equation below, the expression of the Riccati equation is: 0=P i A i +A i T P i -P i B i R i -1 B i T P i +Q i ; x i v =-(Q i +P i A i ) -1 P i c i ; Among them, Q i ∈R N×N is the state deviation parameter of the quadratic cost function of the i-th module; The control output expression of the expected controller is: The final output expression is: u(t)=∑ i∈M λ i (t)u i (t)。 6. The humanoid robot control system according to claim 3, characterized in that: The potential state information estimated by the state estimator specifically includes: Position state: The forward learning model obtains the current position information of the robot detected by the sensor and sets it as the origin of the space. The state estimator estimates the potential position information of the robot at the next moment. Speed ​​state: The forward model is learned to obtain the current movement rate of the robot at different joints detected by the sensor, and the state estimator estimates the posture state equation of the robot at the next moment; Noisy state: Account for sensor errors, modeling errors, and / or noisy state to more accurately estimate the true state of the system.

7. The humanoid robot control system according to claim 5, characterized in that: The steps of the value function estimator to evaluate the robot's next action include: Target analysis: confirm the robot's current location information and clarify the next action to be performed; Environmental assessment: Based on the environmental data information in the observation status information, check whether there is enough space ahead to perform the next action and whether the ground is stable; Power and structure considerations: determine the actuators that need to move, the order and amplitude of the actuator movements to achieve efficient and stable execution; Safety verification: simulate the action plan to determine whether the robot maintains stability during execution and avoids causing damage to the surrounding environment.

8. The humanoid robot control system according to claim 1, characterized in that: The joint controller also includes a standing balance module, and the balance control output expression of the standing balance module is: in, Where x0 is the horizontal displacement of the pressure center, is the horizontal velocity of the pressure center, z is the vertical position of the pressure center, g is the gravitational acceleration, x is cp is the ground action point where the robot can step on and stop completely, FootEdge1 is the distance from the initial position of the pressure center to the front edge of the support area, the support area refers to the collection of CP points, FootEdge2 is the distance from the initial position of the pressure center to the rear edge of the support area, is the minimum torque produced by the trunk about the center of mass, is the maximum torque produced by the trunk relative to the center of mass, is the maximum negative acceleration of the trunk relative to the center of mass in the Z direction, where the Z direction is the opposite direction of gravity. is the generalized torque of the trunk joint, x CoP is the location of the center of pressure.

9. A humanoid robot control method, based on the humanoid robot control system according to any one of claims 1 to 8, characterized in that: include: By iteratively calculating the responsibility signal and updating the parameters, the learning goal of the learning forward model is achieved, minimizing the weighted prediction error of each linear dynamics model; The state estimator estimates the potential state information of the robot at the next moment based on the robot's initial state information or the currently estimated potential weighted state information combined with the output information of the learning forward model; The control contribution rate of each module unit is allocated according to the Bayesian rule. The control contribution rate is used to configure the weight of the potential state information output by each module unit. The potential weighted state information of the robot at the next moment is obtained by weighting the potential state information of each module unit. Evaluate the robot's next action, output an estimated value function based on the current estimated potential weighted state information and the output information of the learned forward model; The expected controller of each module unit is established through the value function and the currently estimated potential weighted state information, and the expected controller is used to derive the control output; the control contribution rate is also used to configure the weight of the control output derived from each module unit, and the final output is obtained by the weighted sum of the control outputs of each module unit; Combine the final output and motion trajectory instructions to generate torque output, control the robot's joints to run in the corresponding order and amplitude, and perform the specified action.

10. A storage medium, characterized in that Computer instructions are stored, and when the computer instructions are executed by the processor, the control method as claimed in claim 9 is executed.

Images