Biped robot intelligent control method based on attitude synthesis control network

Abstract

The invention provides a biped robot intelligent control method based on a posture synthesis control network. The biped robot intelligent control method comprises the steps that S1, three kinds of 3D human body posture data of walking, jumping and turning are collected, data are marked, and a data set is generated; s2, setting an attitude constraint reward function and a task reward function; s3, constructing a state variation auto-encoder which comprises an encoder, a state network and a decoder, setting a loss function of the state variation auto-encoder, and training the state variation auto-encoder to obtain a pre-training model; and S4, constructing and training a posture synthesis control framework, synthesizing a posture sequence through the posture synthesis control framework, performing biped robot joint adaptation on postures, and deploying the postures into the biped robot to realize intelligent control of the biped robot. According to the method, the utilization rate of the motion data is improved, and the performance of the biped robot in the aspect of coping with complex tasks is improved.

Description

Technical Field

[0001] The invention belongs to the field of robot intelligent control, and in particular relates to an intelligent control method for a biped robot based on a posture synthesis control network. Background Art

[0002] Bipedal humanoid robots in complex environments are an important part of the robot market. Bipedal humanoid robots are a type of bionic robot that has strong operational capabilities in many different service areas such as hotels, transportation, and elderly care. However, the current service scope of humanoid bipedal robots can only be deployed in simple environments. When facing earthquake-stricken areas, fire areas, and even sandy environments, the gait control effect of bipedal robots will be greatly reduced or even unable to work properly. Therefore, research on the application of bipedal robots in complex environments is a research direction urgently needed by the current market.

[0003] At present, imitation learning is one of the mainstream methods to solve this problem. However, imitation learning has great limitations and is heavily dependent on the quality of demonstration. At the same time, the differences in joints, body shape, etc. between bipedal robots and humans must be considered. At present, the control method based on imitating real human motion reduces the dependence on artificiality in controller design, but requires complete motion data engineering. In addition, in data-limited motion control tasks, the current data-based control method overly relies on manually annotated data, the resulting posture is unnatural and the data utilization rate is low. CN113467235B discloses a bipedal robot gait control method and control device, which combines imitation learning and reinforcement learning, but requires a large amount of complete model motion data, and the training process is also relatively complicated. CN116400717A discloses a bipedal robot motion control method based on imitation learning and reinforcement learning, which pre-plans multiple reference trajectories, generates exploratory trajectories using heuristic methods, and trains neural networks to output appropriate action instructions, but the generalization ability for unseen motion tasks is limited. CN117213475A discloses a data processing method, device and storage medium for a bipedal robot, which focuses on the use of multi-sensor data fusion for camera pose estimation, but has the disadvantages of high sensor accuracy and large amount of calculation. CN117215204B discloses a robot gait training method and system based on reinforcement learning, which designs a linear gait observation reward function to train a neural network. However, the design of the reward function is also very challenging, and in-depth analysis of gait characteristics is required to define a reasonable reward target.

[0004] In the research of attention mechanism, variational autoencoder, and reinforcement learning, advanced technologies such as self-attention mechanism and proximal strategy optimization have emerged and used to improve imitation learning, providing new ideas for breaking the bottleneck of imitation learning. By using human posture estimation and intelligent body posture generation methods, the bottleneck of bipedal robot imitation learning can be broken through, and the gait control of bipedal robots can be accurately connected, which has good research feasibility, research prospects and application value. Summary of the invention

[0005] In order to overcome the above technical problems, the purpose of the present invention is to provide an intelligent control method for a bipedal robot based on a posture synthesis control network, which can enable the bipedal robot to autonomously plan motion postures in complex environments to complete target tasks.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] An intelligent control method for a biped robot based on a posture synthesis control network comprises the following steps:

[0008] Step S1: Collect three types of 3D human posture data: walking, jumping and turning, annotate the data and generate a data set;

[0009] Step S2: Setting the posture constraint reward function and the task reward function;

[0010] Step S3: construct a state variational autoencoder, including an encoder, a state network and a decoder, set the loss function of the state variational autoencoder and train the state variational autoencoder to obtain a pre-trained model;

[0011] Step S4: construct and train a posture synthesis control architecture, synthesize posture sequences through the posture synthesis control architecture, adapt the postures to the bipedal robot joints and deploy them into the bipedal robot to achieve intelligent control of the bipedal robot.

[0012] Preferably, step S1 collects three types of 3D human body posture data of walking, jumping and turning, annotates the data and generates a data set, specifically:

[0013] Step S1.1: The human body collects two hours of posture sequence data by wearing a motion capture device, and the posture forms include walking, jumping and turning, while ensuring that the duration of each type of posture sequence data is the same;

[0014] Step S1.2: Generate corresponding human posture sequence capture files for different postures through manual annotation to create a human posture sequence capture dataset.

[0015] Preferably, step S2 sets the posture constraint reward function and the task reward function, specifically:

[0016] Step S2.1: Setting the reward function for walking and turning posture synthesis It is expressed as follows:

[0017]

[0018] in, Represents the square of the distance of the right ankle joint in the Z-axis direction, represents the square of the distance of the left ankle joint in the Z-axis direction, and α represents the penalty coefficient;

[0019] Step S2.2: Setting the reward function for walking and jumping posture synthesis It is expressed as follows:

[0020]

[0021] Where T represents the current movement stage, (0,0,0) represents walking, (1,0,0) represents walking to jumping preparation, (1,1,0) represents jumping up, and (1,1,1) represents jumping down. Represents the square of the distance of the right ankle joint in the Z-axis direction, Represents the square of the distance of the left ankle joint in the Z-axis direction, represents the square of the distance of the hip joint in the Y-axis direction, β1 and β2 represent penalty coefficients;

[0022] Step S2.3: Set the target task reward function r task , 1 if the task is successful, and 0 if the task fails.

[0023] Preferably, step S3 constructs a state variational autoencoder, including an encoder, a state network and a decoder, sets the loss function of the state variational autoencoder and trains the state variational autoencoder to obtain a pre-trained model, specifically:

[0024] Step S3.1: Construct an encoder F consisting of three fully connected layers enc , where each fully connected layer is followed by a linear rectifier unit activation function;

[0025] Step S3.2: The posture data p at time t and time t+1 t and p t+1 Input to encoder;

[0026] The output of the last fully connected layer of the encoder is fed into two independent fully connected layers, outputting the mean μ of the latent vector t and variance σ t ;

[0027] Use the reparameterization technique to adjust the mean μ t and variance σ tSampling generates the latent vector z t , which is expressed as follows:

[0028] z t =μ t +σ t *∈ t

[0029] Among them, ∈ t represents random noise;

[0030] Step S3.3: Construct a state network F consisting of three fully connected layers s , each fully connected layer is followed by a linear rectifier unit activation function;

[0031] F s is followed by an output layer and a normalized exponential activation function;

[0032] Step S3.4: The posture data p at time t t Combined with the potential vector z and put into the state network, the state vector S = [s1,...,s n ];

[0033] Step S3.5: Construct n decoders F consisting of three-layer feedforward neural networks dec , where each linear layer is followed by a linear rectifier unit activation function;

[0034] Step S3.6: By traversing i=(1,...,n), the weights W of all fully connected layers of the i-th encoder are i With s i Perform matrix dot multiplication to perform weighted operations;

[0035] Perform matrix addition operation on all weighted encoder weights to obtain the weight W of the hybrid encoder network mix , calculated as follows:

[0036] W mix =W1⊙s1+...+W n ⊙s n

[0037] By traversing i=(1,...,n), we can set the bias B of all fully connected layers of the i-th encoder i With s i Perform matrix dot multiplication to perform weighted operations;

[0038] Perform matrix addition on all weighted encoder biases to obtain the bias B of the hybrid encoder network mix , calculated as follows:

[0039] Bmix =B1⊙s1+...+B n ⊙s n

[0040] Step S3.7: The posture data p at time t t Combined with the potential vector z, it is sent to the hybrid encoder to generate the predicted posture data at time t+1

[0041] Step S3.8: Through autoregression, Instead of p t+1 As input in subsequent training iterations;

[0042] Step S3.9: Set up the state variational autoencoder F SVAE The loss function L SVAE ,The loss function consists of three parts, namely reconstruction loss, KL divergence and posture similarity penalty, L SVAE The calculation method is as follows:

[0043]

[0044] Among them, f MSE (·) represents the mean square error function, D KL (·) represents the KL divergence function, N(·,·) represents the distribution, η represents the hyperparameter controlling the penalty strength, m represents the number of historical postures, ω i represents the weight of each posture in the posture similarity measurement, sc i Represents the similarity score between the predicted posture and the i-th posture in history;

[0045] Step S3.10: For each type of posture, train a separate F SVAE , thus obtaining three models for running, jumping and turning.

[0046] Preferably, step S4 constructs and trains a posture synthesis control framework, synthesizes a posture sequence through the posture synthesis control framework, and deploys the posture to the biped robot after biped robot joint adaptation to realize the intelligent control of the biped robot, specifically:

[0047] Step S4.1: Establish the Markov decision process for posture synthesis control and express it as Tuple, where represents the state space, represents the action space, represents the state transition probability function, represents the reward function, γ represents the discount factor, and the action space It consists of the mixing ratios {Z,Ω} of the potential samples and poses in the n decoders;

[0048] Given the current posture p t ∈S and action a t The next pose p t+1 The probability density of ∈S is denoted by p, where the actions are represented as:

[0049]

[0050] Step S4.2: Sample a according to the policy network t , Through the corresponding The hybrid decoder in gets the posture

[0051] attitude Mixing ratio Multiply them separately and sum them up to get the final synthetic posture It is expressed as:

[0052]

[0053] Step S4.3: Setting the advantage function A of the reinforcement learning method Proximal Policy Optimization 2 (PPO2) t :

[0054] A t =δ t +(γλ)δ t+1 +(γλ) 2 δ t+2 +...+(γλ) T-t+1 δ T-1

[0055] Among them, λ represents the timing difference error control parameter, δ t represents the temporal difference error at time step t, δ t The calculation formula is:

[0056] δ t =r t +γV φ (s t+1 )-V φ (s t )

[0057] Step S4.4: Set the policy network loss function L for the reinforcement learning method PPO2 π (θ), calculated as follows:

[0058]

[0059] Among them, θ represents the current parameters of the policy network, θ krepresents the parameters of the policy network before time k, π θ (a|s) represents the current strategy, represents the advantage function value of taking action a in state s, ∈ represents the hyperparameter of the difference between the new strategy and the old strategy, clip(x,a,b) represents limiting the value of x to the range [a,b], returning a if x is less than a, returning b if x is greater than b, otherwise returning x;

[0060] Step S4.5: Set the value network loss function φ of the reinforcement learning method PPO2 k+1 , which is expressed as follows:

[0061]

[0062] Among them, D k represents a set of multiple trajectories, |D k |Represents D k The number of trajectories in the equation, T represents the length of each trajectory, V φ (s t ) represents the state s predicted by the value network t The value of represents the cumulative reward at time step t;

[0063] Step S4.6: Train the posture synthesis control architecture F SMC , output the robot posture sequence, adapt the posture joints to the biped robot joints, and realize the intelligent control of the biped robot.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] (1) The present invention compresses the dynamic transformation between the current posture and the next posture into a latent space through a state variational autoencoder, which can generate diversified and high-quality postures. At the same time, through the posture synthesis control architecture, multiple decoders are trained simultaneously to achieve seamless splicing of segmented motion and posture synthesis from different sources, thereby improving generation efficiency and enhancing control effect.

[0066] (2) The present invention enhances the imitation learning capability of the bipedal robot and improves the utilization rate of motion data, thereby enabling the bipedal robot to complete complex tasks through limited segmented motion and imitation learning intelligent control. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a flow chart of a biped robot intelligent control method based on a posture synthesis control network provided by the present invention;

[0068] Figure 2 is a schematic diagram of a state variational autoencoder in one embodiment of the present invention;

[0069] Figure 3 is a schematic diagram of a gesture synthesis control architecture in one embodiment of the present invention;

[0070] Figure 4 This is a schematic diagram of the change in mean square error during the training of the state variational autoencoder for three postures;

[0071] Figure 5 This is a schematic diagram of the change in average rewards obtained by the composite posture and the single posture during the training of the round-trip walking task;

[0072] Figure 6 This is a schematic diagram of the average reward change when using the posture constraint reward function and not using the posture constraint reward function during the training of the round-trip walking task. DETAILED DESCRIPTION

[0073] Unless otherwise specified, any feature disclosed throughout this specification may be replaced by an alternative feature that is equivalent or has a similar purpose. That is, each feature is only an example in a series of equivalent or similar features.

[0074] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be described in more detail below in conjunction with the drawings of the specification. It should be noted that the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0075] Example

[0076] Based on previous work, this paper innovatively proposes an intelligent control method for bipedal robots based on a posture synthesis control network, aiming to help robots achieve natural and smooth movements in complex environments, especially when training data is insufficient. This method combines short, simple action clips into longer, more complex posture sequences, which not only reduces the demand for data and the workload of manual annotation, but also generates high-quality, non-repetitive posture sequences. In addition, it introduces reward function constraints to prevent the robot from making unnatural movements.

[0077] Reference Figure 1 According to the present invention, a biped robot intelligent control method based on a posture synthesis control network includes:

[0078] Step S1 collects three types of 3D human posture data: walking, jumping and turning, annotates the data and generates a data set, specifically:

[0079] Step S1.1: The human body collects two hours of posture sequence data by wearing a motion capture device, and the posture forms include walking, jumping and turning, while ensuring that the duration of each type of posture sequence data is the same.

[0080] Step S1.2: Generate corresponding human posture sequence capture files for different postures through manual annotation to create a human posture sequence capture dataset.

[0081] Step S2 sets the posture constraint reward function and the task reward function, specifically:

[0082] Step S2.1: Setting the reward function for walking and turning posture synthesis It is expressed as follows:

[0083]

[0084] in, Represents the square of the distance of the right ankle joint in the Z-axis direction, represents the square of the distance of the left ankle joint in the Z-axis direction, and α represents the penalty coefficient. In the present invention, α is 0.05.

[0085] Step S2.2: Setting the reward function for walking and jumping posture synthesis It is expressed as follows:

[0086]

[0087] Where T represents the current movement stage, (0,0,0) represents walking, (1,0,0) represents walking to jumping preparation, (1,1,0) represents jumping up, and (1,1,1) represents jumping down. Represents the square of the distance of the right ankle joint in the Z-axis direction, Represents the square of the distance of the left ankle joint in the Z-axis direction, represents the square of the distance of the hip joint in the Y-axis direction, and β1 and β2 represent penalty coefficients.

[0088] Step S2.3: Set the target task reward function r task , 1 if the task is successful, and 0 if the task fails.

[0089] Reference Figure 2 , step S3 constructs a state variational autoencoder, including an encoder, a state network and a decoder, sets the loss function of the state variational autoencoder and trains the state variational autoencoder to obtain a pre-trained model, specifically:

[0090] Step S3.1: Construct an encoder F consisting of three fully connected layers enc , where each fully connected layer is followed by a linear rectifier unit activation function.

[0091] Step S3.2: The posture data p at time t and time t+1 t and p t+1 Input to the encoder. The output of the last fully connected layer of the encoder is sent to two independent fully connected layers, and the mean μ of the potential vector is output. t and variance σ t . Using the reparameterization technique, the mean μ t and variance σ t Sampling generates the latent vector z t , which is expressed as follows:

[0092] z t =μ t +σ t *∈ t

[0093] Among them, ∈ t represents random noise.

[0094] Step S3.3: Construct a state network F consisting of three fully connected layers s , each fully connected layer is followed by a linear rectifier unit activation function. s is followed by an output layer and a normalized exponential activation function.

[0095] Step S3.4: The posture data p at time t t Combined with the potential vector z and put into the state network, the state vector S = [s1,...,s n ].

[0096] Step S3.5: Construct n decoders F consisting of three-layer feedforward neural networks dec , where each linear layer is followed by a linear rectifier unit activation function. In the present invention, n is 6.

[0097] Step S3.6: By traversing i=(1,...,n), the weights W of all fully connected layers of the i-th encoder are i With s i Perform matrix multiplication to perform weighted operation. Perform matrix addition operation on all weighted encoder weights to obtain the weight W of the hybrid encoder network mix , calculated as follows:

[0098] W mix =W1⊙s1+...+W n ⊙s n

[0099] By traversing i=(1,...,n), we can set the bias B of all fully connected layers of the i-th encoderi With s i Perform matrix multiplication to perform weighted operations. Perform matrix addition on all weighted encoder biases to obtain the bias B of the hybrid encoder network. mix , calculated as follows:

[0100] B mix =B1⊙s1+...+B n ⊙s n

[0101] Step S3.7: The posture data p at time t t Combined with the potential vector z, it is sent to the hybrid encoder to generate the predicted posture data at time t+1

[0102] Step S3.8: Through autoregression, Instead of p t+1 As input in subsequent training iterations.

[0103] Step S3.9: Set up the state variational autoencoder F SVAE The loss function L SVAE ,The loss function consists of three parts, namely reconstruction loss, KL divergence and posture similarity penalty, L SVAE The calculation method is as follows:

[0104]

[0105] Among them, f MSE (·) represents the mean square error function, D KL (·) represents the KL divergence function, N(·,·) represents the distribution, η represents the hyperparameter controlling the penalty strength, m represents the number of historical postures, ω i represents the weight of each posture in the posture similarity measurement, sc i Represents the similarity score between the predicted posture and the i-th posture in the history. In the present invention, η is 0.1 and m is 10.

[0106] Step S3.10: For each type of posture, train a separate F SVAE , thus obtaining three training models for walking, jumping and turning. Setting the learning rate to 0.001 and setting 5 different random seeds, we obtain a schematic diagram of the mean square error change during the training of the state variation autoencoder for the three postures, as shown in Figure 4 shown.

[0107] Reference Figure 3Step S4 constructs and trains a posture synthesis control framework, synthesizes posture sequences through the posture synthesis control framework, and deploys the postures to the biped robot after adapting the postures to the biped robot joints to realize the intelligent control of the biped robot, specifically:

[0108] Step S4.1: Establish the Markov decision process for posture synthesis control and express it as Tuple, where represents the state space, represents the action space, represents the state transition probability function, represents the reward function, γ represents the discount factor, and the action space It is composed of the mixing ratio {Z,Ω} of the potential samples and postures in n decoders. In the present invention, γ is taken as 0.99. Given the current posture p t ∈S and action a t The next pose p t+1 The probability density of ∈S is denoted by p, where the actions are represented as:

[0109]

[0110] Step S4.2: Sample a according to the policy network t , Through the corresponding The hybrid decoder in gets the posture attitude Mixing ratio Multiply them separately and sum them up to get the final synthetic posture It is expressed as:

[0111]

[0112] Step S4.3: Setting the advantage function A of the reinforcement learning method Proximal Policy Optimization 2 (PPO2) t :

[0113] A t =δ t +(γλ)δ t+1 +(γλ) 2 δ t+2 +...+(γλ) T-t+1 δ T-1

[0114] Among them, λ represents the timing difference error control parameter, δ t represents the temporal difference error at time step t, δ t The calculation formula is:

[0115] δ t =r t+γV φ (s t+1 )-V φ (s t )

[0116] Step S4.4: Set the policy network loss function L for the reinforcement learning method PPO2 π (θ), calculated as follows:

[0117]

[0118] Among them, θ represents the current parameters of the policy network, θ k represents the parameters of the policy network before time k, π θ (a|s) represents the current strategy, represents the advantage function value of taking action a in state s, ∈ represents the hyperparameter of the difference between the new strategy and the old strategy, clip(x,a,b) represents limiting the value of x to the range [a,b], returning a if x is less than a, returning b if x is greater than b, otherwise returning x. In the present invention, ∈ is 0.2.

[0119] Step S4.5: Set the value network loss function φ of the reinforcement learning method PPO2 k+1 , which is expressed as follows:

[0120]

[0121] Among them, D k represents a set of multiple trajectories, |D k |Represents D k The number of trajectories in the equation, T represents the length of each trajectory, V φ (s t ) represents the state s predicted by the value network t The value of Represents the cumulative reward at time step t.

[0122] Step S4.6: Train the posture synthesis control architecture F SMC , output the robot posture sequence, adapt the posture joints to the biped robot joints, and realize the intelligent control of the biped robot. Set the learning rate to 0.002, set 5 different random seeds, and obtain the average reward change diagram of the training round-trip task with and without the posture constraint reward function, and the average reward change diagram of the training round-trip task with and without the posture constraint reward function, as shown in the figure below. Figure 5 and Figure 6 shown.

[0123] The above embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. For those skilled in the art, without departing from the spirit of the present invention, the present invention may be subjected to several modifications, improvements and equivalent substitutions, which are deemed to fall within the scope of protection of the claims of the present invention.

Claims

1. An intelligent control method for a biped robot based on a posture synthesis control network, characterized in that: include: Step S1: Collect three types of 3D human posture data: walking, jumping and turning, annotate the data and generate a data set; Step S2: Setting the posture constraint reward function and the task reward function; Step S3: construct a state variational autoencoder, including an encoder, a state network and a decoder, set the loss function of the state variational autoencoder and train the state variational autoencoder to obtain a pre-trained model; Step S4: construct and train a posture synthesis control architecture, synthesize posture sequences through the posture synthesis control architecture, adapt the postures to the bipedal robot joints and deploy them into the bipedal robot to achieve intelligent control of the bipedal robot.

2. The intelligent control method of a biped robot based on a posture synthesis control network according to claim 1, characterized in that: The step S1 comprises: Step S1.1: The human body collects two hours of posture sequence data by wearing a motion capture device, and the posture forms include walking, jumping and turning, while ensuring that the duration of each type of posture sequence data is the same; Step S1.2: Generate corresponding human posture sequence capture files for different postures through manual annotation to create a human posture sequence capture dataset.

3. The intelligent control method of a biped robot based on a posture synthesis control network according to claim 1, characterized in that: The step S2 comprises: Step S2.1: Setting the reward function for walking and turning posture synthesis It is expressed as follows: in, Represents the square of the distance of the right ankle joint in the Z-axis direction, represents the square of the distance of the left ankle joint in the Z-axis direction, and α represents the penalty coefficient; Step S2.2: Setting the reward function for walking and jumping posture synthesis It is expressed as follows: Where T represents the current movement stage, (0,0,0) represents walking, (1,0,0) represents walking to jumping preparation, (1,1,0) represents jumping up, and (1,1,1) represents jumping down. Represents the square of the distance of the right ankle joint in the Z-axis direction, Represents the square of the distance of the left ankle joint in the Z-axis direction, represents the square of the distance of the hip joint in the Y-axis direction, β1 and β2 represent penalty coefficients; Step S2.3: Set the target task reward function r task , 1 if the task is successful, and 0 if the task fails.

4. The intelligent control method of a biped robot based on a posture synthesis control network according to claim 1, characterized in that: The step S3 comprises: Step S3.1: Construct an encoder F consisting of three fully connected layers enc , where each fully connected layer is followed by a linear rectifier unit activation function; Step S3.2: The posture data p at time t and time t+1 t and p t+1 Input to encoder; The output of the last fully connected layer of the encoder is fed into two independent fully connected layers, outputting the mean μ of the latent vector t and variance σ t ; Use the reparameterization technique to adjust the mean μ t and variance σ t Sampling generates the latent vector z t , which is expressed as follows: z t =μ t +s t *∈ t Among them, ∈ t represents random noise; Step S3.3: Construct a state network F consisting of three fully connected layers s , each fully connected layer is followed by a linear rectifier unit activation function; F s is followed by an output layer and a normalized exponential activation function; Step S3.4: The posture data p at time t t Combined with the potential vector z and put into the state network, the state vector S = [s1,...,s n ]; Step S3.5: Construct n decoders F consisting of three-layer feedforward neural networks dec , where each linear layer is followed by a linear rectifier unit activation function; Step S3.6: By traversing i=(1,...,n), the weights W of all fully connected layers of the i-th encoder are i With s i Perform matrix dot multiplication to perform weighted operations; Perform matrix addition operation on all weighted encoder weights to obtain the weight W of the hybrid encoder network mix , calculated as follows: IN mix =W1⊙s1+...+W n ⊙s n By traversing i=(1,...,n), we can set the bias B of all fully connected layers of the i-th encoder i With s i Perform matrix dot multiplication to perform weighted operations; Perform matrix addition on all weighted encoder biases to obtain the bias B of the hybrid encoder network mix , calculated as follows: B mix =B1⊙s1+...+B n ⊙s n Step S3.7: The posture data p at time t t Combined with the potential vector z, it is sent to the hybrid encoder to generate the predicted posture data at time t+1 Step S3.8: Through autoregression, Instead of p t+1 As input in subsequent training iterations; Step S3.9: Set up the state variational autoencoder F SVAE The loss function L SVAE ,The loss function consists of three parts, namely reconstruction loss, KL divergence and posture similarity penalty, L SVAE The calculation method is as follows: Among them, f MSE (·) represents the mean square error function, D KL (·) represents the KL divergence function, N(·,·) represents the distribution, η represents the hyperparameter controlling the penalty strength, m represents the number of historical postures, ω i represents the weight of each posture in the posture similarity measurement, sc i Represents the similarity score between the predicted posture and the i-th posture in history; Step S3.10: For each type of posture, train a separate F SVAE , thus obtaining three models for running, jumping and turning.

5. The intelligent control method of a biped robot based on a posture synthesis control network according to claim 1, characterized in that: The step S4 comprises: Step S4.1: Establish the Markov decision process for posture synthesis control and express it as Tuple, where represents the state space, represents the action space, represents the state transition probability function, represents the reward function, γ represents the discount factor, and the action space It consists of the mixing ratios {Z,Ω} of the potential samples and poses in the n decoders; Given the current posture p t ∈S and action a t The next pose p t+1 The probability density of ∈S is denoted by p, where the actions are represented as: Step S4.2: Sample a according to the policy network t , Through the corresponding The hybrid decoder in gets the posture attitude Mixing ratio Multiply them separately and sum them up to get the final synthetic posture It is expressed as: Step S4.3: Setting the advantage function A of the reinforcement learning method proximal policy optimization 2 t : A t =d t +(gl)d t+1 +(cl) 2 d t+2 +...+(cl) T-t+1 d T-1 Among them, λ represents the timing difference error control parameter, δ t represents the temporal difference error at time step t, δ t The calculation formula is: δ t =r t +γV φ (s t+1 )-V φ (s t ) Step S4.4: Set the policy network loss function L for the reinforcement learning method proximal policy optimization 2 π (θ), calculated as follows: Among them, θ represents the current parameters of the policy network, θ k represents the parameters of the policy network before time k, π θ (a|s) represents the current strategy, represents the advantage function value of taking action a in state s, ∈ represents the hyperparameter of the difference between the new strategy and the old strategy, clip(x,a,b) represents limiting the value of x to the range [a,b], returning a if x is less than a, returning b if x is greater than b, otherwise returning x; Step S4.5: Set the value network loss function φ of the reinforcement learning method proximal policy optimization 2 k+1 , which is expressed as follows: Among them, D k represents a set of multiple trajectories, |D k |Represents D k The number of trajectories in the equation, T represents the length of each trajectory, V φ (s t ) represents the state s predicted by the value network t The value of represents the cumulative reward at time step t; Step S4.6: Train the pose synthesis control architecture F SMC , output the robot posture sequence, adapt the posture joints to the biped robot joints, and realize the intelligent control of the biped robot.

Images