Joint control method and apparatus

Abstract

The application discloses a joint control method and device, the method comprises the following steps: predicting the predicted posture information of the robot at the current time based on the historical state data of the robot at at least one historical time, the historical state data is used for characterizing the historical actual posture of the robot at the historical time and the historical action performed by each joint; obtaining the actual posture information of the robot at the current time; determining the posture error feature based on the predicted posture information and the actual posture information, the posture error feature is used for characterizing the abnormal factor feature causing the error of the historical action of each joint of the robot at the last time; determining the action to be performed by each joint of the robot at the current time based on the posture error feature.

Description

Technical Field

[0001] This application relates to the field of control technology, and in particular to a joint control method and apparatus. Background Technology

[0002] Robots often operate in complex and ever-changing environments, facing various uncertainties and potential failure risks. Therefore, how to more effectively control robot movements to enable reliable walking or other actions is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0003] On the one hand, this application provides a joint control method, including:

[0004] Based on the robot's historical state data at at least one historical moment, the robot's predicted posture information at the current moment is predicted. The historical state data is used to characterize the robot's historical actual posture at that historical moment and the historical actions performed by each joint.

[0005] Obtain the robot's actual posture information at the current moment;

[0006] Based on the predicted posture information and the actual posture information, posture error features are determined. The posture error features are used to characterize the abnormal factors that cause errors in the historical actions of each joint of the robot at the previous moment.

[0007] Based on the posture error characteristics, the actions to be performed by each joint of the robot at the current moment are determined.

[0008] In one possible implementation, the joint control method further includes:

[0009] Based on the historical state data of at least one historical moment, determine the joint state mask of the robot;

[0010] The step of determining the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics includes:

[0011] Based on the posture error features and the joint state mask, the actions to be performed by each joint of the robot at the current moment are determined.

[0012] In yet another possible implementation, the joint control method further includes:

[0013] Variational autoencoder processing is performed on the historical state data of the robot at at least one historical moment to obtain state coding features, which are used to characterize at least one of the robot's motion state features and the environmental features in which it is located.

[0014] The prediction of the robot's pose information at the current moment based on the robot's historical state data at at least one historical moment includes:

[0015] Based on the state coding features, the robot's predicted posture information at the current moment is predicted.

[0016] In another possible implementation, determining the attitude error characteristics based on the predicted attitude information and the actual attitude information includes:

[0017] Determine the attitude gap information between the predicted attitude information and the actual attitude information;

[0018] The attitude difference information is encoded to generate attitude error features.

[0019] In yet another possible implementation, the joint control method further includes:

[0020] Variational autoencoder processing is performed on the historical state data of the robot at at least one historical moment to obtain state coding features, which are used to characterize at least one of the robot's motion state features and the environmental features in which it is located.

[0021] Determining the joint state mask of the robot based on the historical state data of at least one historical moment includes:

[0022] Based on the state coding features, the joint state mask of the robot is determined.

[0023] In yet another possible implementation, the joint control method further includes:

[0024] Based on the historical state data of at least one historical moment, the predicted speed of the robot at the current moment is determined;

[0025] The step of determining the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics includes:

[0026] Based on the posture error characteristics and the predicted velocity, the actions to be performed by each joint of the robot at the current moment are determined.

[0027] In another aspect, this application also provides a joint control device, comprising:

[0028] The forward prediction module is used to predict the robot's posture information at the current moment based on the robot's historical state data at at least one historical moment. The historical state data is used to characterize the robot's historical actual posture at the historical moment and the historical actions performed by each joint.

[0029] The error determination module is used to determine the posture error characteristics based on the predicted posture information and the actual posture information of the robot at the current moment. The posture error characteristics are used to characterize the abnormal factors that cause errors in the historical actions of each joint of the robot at the previous moment.

[0030] The decision module is used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics.

[0031] In one possible implementation, the forward prediction module is further configured to: determine the joint state mask of the robot based on the historical state data of the at least one historical moment;

[0032] The decision module is specifically used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error features and the joint state mask.

[0033] In yet another possible implementation, the joint control device further includes:

[0034] The variational autoencoder module is used to perform variational autoencoder processing on the historical state data of the robot at at least one historical moment to obtain state coding features. The state coding features are used to characterize at least one of the robot's motion state features and the environmental features in which it is located.

[0035] The forward prediction module is specifically used to predict at least one of the robot's predicted posture information at the current moment and the joint state mask based on the state coding features.

[0036] In another possible implementation, the joint control device further includes: a variational autoencoder module, used to perform variational autoencoder processing on the historical state data of the robot at at least one historical moment to obtain the predicted speed of the robot at the current moment;

[0037] The decision module is specifically used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics and the predicted speed. Attached Figure Description

[0038] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0039] Figure 1 A flowchart illustrating the joint control method provided in this application;

[0040] Figure 2 Another flowchart illustrating the joint control method provided in this application;

[0041] Figure 3 Another flowchart illustrating the joint control method provided in this application;

[0042] Figure 4 Another flowchart illustrating the joint control method provided in this application;

[0043] Figure 5 A schematic diagram illustrating the implementation principle of the joint control method provided in this application;

[0044] Figure 6 This is an example diagram illustrating one implementation principle of model training in this application;

[0045] Figure 7 A schematic diagram of one possible structure of the joint control device provided in this application;

[0046] Figure 8 This is a schematic diagram of another component structure of the joint control device provided in this application. Detailed Implementation

[0047] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments and is not intended to limit the application. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0048] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0049] like Figure 1 This paper illustrates a flowchart of a joint control method provided in this application. The method of this embodiment can be used to control the movement of each joint in a robot. The method of this embodiment can be applied to the robot's control device. The control device can be a control device built into the robot; or it can be a control device located outside the robot and having a communication connection with the internal controller inside the robot. There are no specific limitations.

[0050] The method in this embodiment may include:

[0051] S101, based on the robot's historical state data at at least one historical moment, predict the robot's predicted posture information at the current moment.

[0052] In this application, the robot has at least one joint. For example, the robot has a torso and at least one limb, which can drive the robot to walk or perform desired actions. The limb may include at least one joint. The robot can be a multi-legged robot, a humanoid robot, or other types of robots. This application does not limit the specific form of the robot.

[0053] The at least one historical moment may include one or more historical moments most recent before the current moment.

[0054] In this application, the historical state data at each historical moment is used to characterize the robot's actual historical posture and the historical actions performed by each joint at that historical moment. For example, the historical state data may include: the robot's actual historical posture at a historical moment and the historical actions performed by each joint. Of course, the historical state data can also characterize the actual historical posture and historical actions in other forms, and there are no limitations on this.

[0055] The historical actual posture at each historical moment is the posture information of the robot at that historical moment, determined based on at least one sensor deployed on the robot. The sensors on the robot may include, but are not limited to, gravity sensors, angular velocity sensors, and angle sensors. For example, the historical actual posture may include, but is not limited to, at least one of the following posture information: the actual movements of each joint in the robot, the robot's gravity projection, angular velocity, direction, and foot position. The actual movements of the joints include the rotation angle between joints and the position moved to. The gravity projection is the gravity data sensed by the gravity sensors on the robot, which is used to characterize the robot's torso tilt angle.

[0056] In this context, for any given historical moment, the historical action executed by the joint is the action that the joint controls to perform at that historical moment. Specifically, the action executed by the joint is the action that the control device determines the action that the joint needs to perform, and then instructs the robot's joint actuator or other controller to drive the joint to perform that action. In this application, the action executed by the joint includes at least one or more of the following: the required rotation angle of the joint in various directions and the position that the joint needs to move to.

[0057] It is understandable that the action performed by the control device on the joint may be the same as the actual action performed by the joint, or the action performed by the joint may be different from the actual action due to joint malfunction or external environmental influences.

[0058] The predicted posture information is based on the robot's historical state data at at least one historical moment, predicting the posture information that the robot should theoretically possess at the current moment. Therefore, the types of posture information included in the historical actual postures of the predicted posture information are the same. For example, the predicted posture information may include at least one of the following: predicted actions of each joint of the robot at the current moment (i.e., predicted actions), predicted gravity projection, predicted angular velocity, and predicted direction.

[0059] S102, obtain the robot's actual posture information at the current moment.

[0060] The robot's current actual posture information is determined by at least one sensor deployed on the robot. Similar to the previous historical actual posture information, this actual posture information may include at least one of the following: the actual movement of each joint of the robot at the current moment, the robot's gravity projection, the robot's angular velocity, and direction.

[0061] S103, based on the predicted attitude information and the actual attitude information, determine the attitude error characteristics.

[0062] The posture error feature is used to characterize the abnormal factors that cause errors in the historical actions of each joint at the previous moment. These errors can include errors in the historical actions determined at the previous moment, as well as deviations in the actual historical actions of each joint. Here, "previous moment" refers to the moment before the current moment.

[0063] Understandably, if the robot's joints are functioning correctly, the external environment does not interfere with the robot, and the model or program used to predict the posture information has no prediction bias, then the predicted posture information and the actual posture information should be similar or identical. Therefore, if there is a deviation between the predicted and actual posture information, it indicates that an abnormal factor caused the robot's previous historical actions to fail to achieve the expected results. Based on this, this application determines posture error characteristics based on the predicted and actual posture information, which can more intuitively reflect the reasons for errors in the historical actions of each joint at the previous moment.

[0064] The abnormal factors characterized by this posture error feature may include, but are not limited to, at least one of the following: joint fault features that cause robot motion errors, external environmental disturbance features, and decision bias features. The joint fault feature may reflect at least one of the following: the presence of a joint fault, the location of the joint fault, and the impact of the joint fault. The external environmental disturbance feature may reflect at least one of the following: the terrain features where the robot is located and the abnormal impact of the terrain features on the robot's movements. The decision bias feature may reflect information such as the bias effects of the program or model module running within the control module.

[0065] S104, based on the posture error characteristics, determines the actions to be performed by each joint of the robot at the current moment.

[0066] The action to be performed by the joint may include at least one of the following: the rotation angle of the joint in at least one coordinate direction, and the spatial position to which the joint needs to move.

[0067] In this application, when determining the actions to be performed by each joint of the robot at the current moment, the posture error characteristic is taken into account. This posture error characteristic can intuitively reflect the specific abnormal factors that caused the deviation of the robot's historical actions at the previous moment, providing a more reliable and intuitive data basis for the robot's fault-tolerant control. This allows for more timely and accurate correction of possible errors in the robot's action decisions. Even in the event of joint failure or external environmental influences, the robot's body posture can be adjusted more reasonably by adjusting the actions of each joint, thereby reducing abnormal situations such as robot falls.

[0068] Understandably, after determining the actions to be performed by each joint of the robot at the current moment, the robot's joint actuators or other controllers can be instructed to drive the robot to perform the corresponding actions based on the actions to be performed by each joint at the current moment. The above steps can be repeated to continue to determine the actions to be performed by the robot at the next moment. In this process, the robot's joint movements can be continuously corrected, thereby achieving closed-loop correction of the robot's joint movements.

[0069] It is understandable that the movements of each joint of the robot are related to the preset movement command information such as the robot's movement direction. Therefore, given that the robot's set movement direction and movement speed are determined, the movements to be performed by each joint of the robot at the current moment can be determined based on this posture error characteristic.

[0070] In one possible implementation, this application can pre-train a decision model using reinforcement learning. Based on this, the decision model can determine the actions to be performed by each joint of the robot according to the posture error characteristics. Of course, there are other possible implementations as well. This application can have multiple specific implementation methods for determining the actions to be performed by each joint of the robot, and there are no specific limitations.

[0071] As described above, before determining the action to be performed by the robot, this application predicts the robot's posture information at the current moment based on historical state data of the robot at at least one historical moment. Based on the predicted posture information and the robot's actual posture information at the current moment, posture error characteristics can be determined. Compared to directly using the original predicted posture information and actual posture information, these posture error characteristics can intuitively reflect the abnormal factors that caused the deviation of the robot's historical actions at the previous moment. This provides a more reliable and intuitive basis for fault perception for the robot's fault-tolerant control, significantly improving the fault-tolerant control capability under abnormal factors such as joint failures or environmental interference. This allows for a more accurate determination of the action to be performed by the robot at the current moment, enabling the robot to walk or complete corresponding actions more reliably.

[0072] It is understandable that in practical applications, the historical state data of the robot at a historical moment and the actual posture information of the robot at the current moment are also key data for determining the robot's next action. Based on this, this application can also determine the action to be performed by each joint of the robot at the current moment based on at least one of the historical state data at at least one historical moment and the actual posture information, combined with the posture error characteristics.

[0073] In any embodiment of this application, there may be multiple ways to determine the attitude difference information, and no limitation is imposed on this.

[0074] In one possible implementation, this application can first determine the attitude difference information between the predicted attitude information and the actual attitude information. For example, the attitude difference information includes: attitude differences between various attitude information between the predicted attitude information and the actual attitude information; or, the attitude difference information is the feature difference between the feature vector of the predicted attitude information and the feature vector of the actual attitude. This attitude difference information can directly reflect the difference between the predicted attitude information and the actual attitude information.

[0075] Based on this, this application can encode the attitude difference information to generate attitude error features. For example, a trained error coding model can be used to encode the attitude difference information to obtain attitude error features.

[0076] Encoding the attitude gap between the predicted and actual attitude information can encode the attitude gap information into features that characterize the abnormal factors of the robot. This allows the attitude gap information to be quantified as a potential feature and used as the feature information for subsequent forward motion prediction, providing data support for robot motion correction and fault-tolerant control.

[0077] In this application, the presence or absence of joint malfunctions in the robot and the location of the malfunctioning joints are relatively important key factors affecting the robot's movement. Therefore, to more accurately achieve fault-tolerant control of the robot and improve the accuracy and reliability of determining robot actions, this application can also determine the joint states of each joint of the robot. The following is a combination of... Figure 2 Please provide an explanation.

[0078] like Figure 2 This illustration shows another flowchart of the joint control method provided in this application. This embodiment may include:

[0079] S201, based on the robot's historical state data at at least one historical moment, predict the robot's predicted posture information at the current moment.

[0080] Among them, historical state data is used to characterize the robot's actual historical posture at that historical moment and the historical actions performed by each joint.

[0081] This step can be found in the previous introduction, and will not be repeated here.

[0082] S202, Based on the historical state data of at least one historical moment, determine the joint state mask of the robot.

[0083] The joint state mask is used to characterize whether any joint of the robot is faulty.

[0084] For example, the number of dimensions of the joint state mask is the same as the number of joints the robot has, and the mask value in each dimension represents whether a joint is faulty. For example, if a joint is faulty, the mask value corresponding to that joint in the joint state mask can be 1; if the joint is not faulty, the mask value corresponding to that joint in the joint state mask is 0.

[0085] In this application, the order of steps S201 and S202 is not limited to... Figure 2 As shown, in practical applications, these two steps can also be performed simultaneously.

[0086] For example, in one instance, this application can determine the robot's predicted pose information and the robot's joint state mask at the current moment using a pre-trained forward prediction model based on the historical state data of at least one historical moment.

[0087] S203, obtain the robot's actual posture information at the current moment.

[0088] S204. Based on the predicted attitude information and the actual attitude information, determine the attitude error characteristics.

[0089] Among them, the posture error feature is used to characterize the abnormal factors that cause errors in the historical actions of each joint of the robot at the previous moment.

[0090] For example, determine the attitude difference information between the predicted attitude information and the actual attitude information, encode the attitude difference information, and generate attitude error features.

[0091] The above steps S203 and S204 can also be found in the description of any other embodiment in this application, and will not be repeated here.

[0092] S205, Based on the posture error characteristics and the joint state mask, determine the actions to be performed by each joint of the robot at the current moment.

[0093] For example, based on the posture error characteristics and the joint state mask, a decision model or other decision algorithm can be used to determine the actions that each joint of the robot needs to perform at the current moment.

[0094] In this embodiment, since the joint state mask can directly reflect whether there is a fault in each joint of the robot and can reflect the specific joint with the fault, on this basis, the posture error feature and the joint state mask can more accurately and intuitively reflect the reasons for the deviation of the historical actions of each joint of the robot, providing richer and more accurate data for the fault-tolerant control of the robot. Moreover, fault-tolerant control based on the joint state mask can prioritize scheduling the normal joints of the robot to complete posture compensation and related action adjustments, which can naturally further improve the fault-tolerant control capability of the robot. Thus, even when the robot experiences sudden joint failures or complex terrain, the actions of each joint of the robot can be reasonably determined to maintain the robot's stable walking and reliable motion execution.

[0095] In any of the above embodiments of this application, there may be a variety of specific implementations for predicting the robot's predicted posture information based on historical state data at at least one historical moment, and no limitation is imposed on this.

[0096] In one possible implementation, this application can perform variational autoencoding on the robot's historical state data at at least one historical moment to obtain state-coded features. Based on these state-coded features, the robot's predicted posture information at the current moment is predicted.

[0097] The state coding features are used to characterize at least one of the robot's motion state features and environmental features. Motion state features can include characteristics that influence the robot's motion state, such as its motion trends and inertial characteristics. Environmental features can at least characterize the terrain features where the robot is located; however, they can also characterize environmental features such as the ground material and weather conditions, without any specific limitations.

[0098] By performing variational autoencoding on historical state data at at least one historical moment, the limited information of historical state data at at least one historical moment can be encoded into a potential high-dimensional feature space. This enables the transformation of the original state data into high-dimensional, continuous motion features containing more information through representation learning. The encoded state features can reflect the robot's motion trends and inertial characteristics at different time scales, as well as the environmental characteristics of the robot. As a result, the predicted posture information of the robot at the current moment can be predicted more accurately based on the state encoded features.

[0099] Similarly, in cases where it is necessary to predict the joint state mask of a robot, in order to more accurately predict the joint state mask of the robot, this application can also perform variational autoencoding on the historical state data of the robot at at least one historical moment; and determine the joint state mask of the robot based on the obtained state encoding features.

[0100] If both the prediction of the joint state mask and the prediction of the pose information require the use of state coding features, then only one variational autoencoder processing is needed on the historical state data.

[0101] To facilitate understanding, let's take one case as an example, combined with... Figure 3 Please provide an explanation. For example... Figure 3 This illustration shows another flowchart of the joint control method provided in this application. The method in this embodiment may include:

[0102] S301, perform variational autoencode processing on the robot's historical state data at at least one historical moment to obtain state coding features.

[0103] Among them, the state coding feature is used to characterize at least one of the robot's motion state features and the environmental features in which it is located.

[0104] For example, a trained variational autoencoder can be used to encode the historical state data at at least one historical moment to obtain state coding features. This variational autoencoder can be trained using sample historical state data obtained under different terrain and other environmental features. This allows the variational autoencoder to extract the robot's motion trends, inertial characteristics, and environmental features reflected in the historical state data at different time scales by performing variational autocoding on the historical state data at at least one historical moment.

[0105] S302, Based on this state coding feature, determine the robot's predicted posture information and the robot's joint state mask at the current moment.

[0106] For example, by using a trained forward prediction model to process the state encoding features, the predicted posture information of the robot at the current moment and the joint state mask of the robot can be obtained.

[0107] In this embodiment, the predicted pose information and joint state mask are determined simultaneously based on state coding features. In practical applications, the predicted pose information and joint state mask can also be determined in two steps, without any specific restrictions.

[0108] S303, obtain the robot's actual posture information at the current moment.

[0109] S304. Based on the predicted attitude information and the actual attitude information, determine the attitude error characteristics.

[0110] The posture error feature is used to characterize the abnormal factors that cause errors in the historical movements of the robot's joints at the previous moment.

[0111] S305, based on the posture error characteristics and joint state mask, determine the actions to be performed by each joint of the robot at the current moment.

[0112] In this embodiment, in order to improve the accuracy of the robot's fault-tolerant control and thus improve the reliability of the determined actions, this application may also combine at least one of the actual posture information, historical state data at at least one historical moment and state coding features, as well as the posture error features and joint state mask, to determine the actions to be performed by each joint of the robot at the current moment.

[0113] In one possible implementation, in any of the above embodiments of this application, in order to accurately determine the action to be performed by the robot, this application can also obtain the robot's speed at the current moment. However, since the robot's own speed cannot be directly obtained using sensors, this application can also determine the predicted speed of the robot at the current moment based on the historical state data of at least one historical moment. Accordingly, the actions to be performed by each joint of the robot at the current moment can be determined based on the posture error characteristics and the predicted speed.

[0114] Furthermore, by combining at least one of historical state data and actual posture information, and based on the posture error characteristics and predicted speed, the actions to be performed by each joint of the robot at the current moment can be determined.

[0115] The joint control method of this application will be described below with reference to one possible scenario. For example... Figure 4 This illustration shows another implementation flow diagram of the joint control method provided in this application. The method in this embodiment may include:

[0116] S401, Obtain historical state data of the robot at at least one historical moment.

[0117] S402, perform variational autoencoding on the robot's historical state data at at least one historical moment to obtain state coding features and the robot's predicted speed at the current moment.

[0118] The state coding feature is used to characterize at least one of the robot's motion state features and the features of its environment.

[0119] For example, a trained variational autoencoder can be used to encode the historical state data of at least one historical moment to obtain the state coding features and the prediction speed.

[0120] S403, based on this state coding feature, determine the robot's predicted pose information and the robot's joint state mask at the current moment.

[0121] For example, by processing the state-encoded features using a forward prediction model, the predicted pose information of the robot at the current moment and the joint state mask of the robot can be obtained. This forward prediction model can be a multilayer perceptron or other forms of intelligent model, without restriction.

[0122] In this embodiment, the predicted pose information and joint state mask are determined simultaneously based on state coding features. In practical applications, the predicted pose information and joint state mask can also be determined in two steps, without any specific restrictions.

[0123] S404, obtain the robot's actual posture information at the current moment.

[0124] S405, determine the attitude gap information between the predicted attitude information and the actual attitude information.

[0125] S406, the attitude difference information is encoded to generate attitude error features.

[0126] The posture error feature is used to characterize the abnormal factors that cause errors in the historical movements of the robot's joints at the previous moment.

[0127] For example, the pose difference information can be encoded using a trained error coding model to generate encoded pose error features.

[0128] S407. Based on the posture error characteristics, joint state mask, historical state data, state coding characteristics, and predicted speed, determine the actions to be performed by each joint of the robot at the current moment.

[0129] For example, based on the posture error characteristics, joint state masks, historical state data, state coding characteristics, and predicted speed, a decision model can be used to determine the actions to be performed by each joint of the robot at the current moment. For instance, a Fourier transform can be performed on the historical state data at at least one historical time point. The transformed state data, along with the aforementioned posture error characteristics, joint state masks, state coding characteristics, and predicted speed, can be used as state data. The relationship between different state data and action behaviors in the decision model can then be used to determine the actions to be performed by each joint.

[0130] Historical state data, state coding features, and predicted speed all affect the robot's motion behavior and provide reliable data support for determining the actions to be performed by each joint of the robot at the current moment. Attitude error features and joint state masks can provide reliable data support for fault-tolerant control of the robot in the process of determining actions. Thus, in the process of determining the actions of each joint of the robot, not only can the movement of normal joints be taken into account, but the impact of faulty joints and the environment on the overall movement of the robot can also be dynamically estimated, thereby reasonably determining the actions of each joint of the robot and improving the reliability and stability of the robot's actions.

[0131] To intuitively understand the implementation of this application's solution, the following example illustrates how fault-tolerant control is achieved based on the robot's historical state data and various models, ultimately determining the actions to be performed by each joint of the robot. Figure 5 A schematic diagram illustrating one implementation principle framework of the solution proposed in this application is shown.

[0132] Depend on Figure 5It can be seen that after using a variational autoencoder to process the robot's historical state data at at least one historical moment, state coding features and the robot's predicted speed at the current moment will be generated.

[0133] The state-encoded features are input into the forward prediction model, which can then process these features to predict the robot's current pose and joint state mask.

[0134] The posture error characteristics can be obtained by subtracting the predicted posture information from the robot's actual posture information at the current moment and inputting the resulting posture difference information into the error coding module.

[0135] Based on the above, the predicted velocity and state features output by the variational autoencoder are also input into the decision model, and the joint state mask output by the forward prediction model and the posture error features output by the error coding module are also input into the decision model, so that the decision model can determine the actions to be performed by each joint of the robot based on these data.

[0136] In this embodiment, the variational autoencoder, the forward prediction model, and the error coding model are trained based on the historical state data of the simulated robot at at least one sample time point in the simulation scenario. The training objective is to determine that the sample posture prediction information of the simulated robot is consistent with the actual sample posture information of the simulated robot.

[0137] These models can be continuously trained in various simulation scenarios. Each simulation scenario corresponds to a map of a terrain where the robot is located and its position on the map. Therefore, in different simulation scenarios, the process of the robot walking or performing target behaviors in different locations on different terrain maps can be simulated.

[0138] like Figure 6 An example diagram of the training model in this application is shown.

[0139] Depend on Figure 6 As can be seen, during the training process, the historical state data of the robot samples obtained in the simulation scenario needs to be processed according to the process of processing historical state data in this embodiment. The sample error coding features obtained by processing through variational autoencoder, forward prediction model and error coding model will be input into decision model. Decision model can determine the action to be performed by the robot at the current moment and control the robot's joints to perform actions in the simulation scenario.

[0140] Based on this, the mean squared error (MSE) is calculated by combining the sample posture information predicted by the forward prediction model with the actual sample posture information of the simulated robot. The calculated MSE value is used as the loss function value. A self-supervised learning strategy is adopted, and the parameters of these models can be continuously adjusted. The specific process is not restricted.

[0141] In the above self-supervised learning process, the actions of each joint and the actual actions of each joint of the robot can be determined based on the decision model. Reinforcement learning is used to continuously train the decision model, and the specific reinforcement learning process is not restricted.

[0142] Furthermore, this application also provides a joint control device. For example... Figure 7 This diagram illustrates a possible structural composition of the joint control device provided in this application. The device in this embodiment may include:

[0143] The forward prediction module 701 is used to predict the robot's posture information at the current moment based on the robot's historical state data at at least one historical moment. The historical state data is used to characterize the robot's historical actual posture at a historical moment and the historical actions performed by each joint.

[0144] The error determination module 702 is used to determine the posture error characteristics based on the predicted posture information and the robot's actual posture information at the current moment. The posture error characteristics are used to characterize the abnormal factors that cause errors in the historical actions of each joint of the robot at the previous moment.

[0145] The decision module 703 is used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics.

[0146] In one possible implementation, the forward prediction module is further used to: determine the robot's joint state mask based on the historical state data of the at least one historical moment.

[0147] Accordingly, the decision module is specifically used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics and the joint state mask.

[0148] In another possible implementation, in any of the above embodiments, the joint control device further includes: a variational autoencoder module, used to perform variational autoencoder processing on the historical state data of the robot at at least one historical moment to obtain state coding features, the state coding features being used to characterize at least one of the robot's motion state features and the environmental features in which it is located.

[0149] Correspondingly, the forward prediction module is specifically used to predict at least one of the robot's predicted pose information and joint state mask at the current moment based on state coding features.

[0150] In another possible implementation, in any of the above embodiments of the device, the device further includes: a variational autoencoder module, used to perform variational autoencoder processing on the historical state data of the robot at at least one historical moment to obtain the predicted speed of the robot at the current moment.

[0151] Accordingly, this decision-making module is specifically used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics and the predicted speed.

[0152] For example, in one possible situation, such as Figure 8 A schematic diagram of another component structure of the joint control device provided in this application is shown.

[0153] contrast Figure 7 and Figure 8 It is known that the joint control device also adds a variational autoencoder module 704, which is used to perform variational autoencoder processing on the historical state data of the robot at at least one historical moment to obtain state coding features and the robot's predicted speed at the current moment. The state coding features are used to characterize at least one of the robot's motion state features and the environmental features in which it is located.

[0154] Correspondingly, the forward prediction module 701 is specifically used to predict at least one of the state coding features generated by the variational autoencoder module 704, the robot's predicted posture information at the current moment, and the joint state mask.

[0155] The decision module 703 is specifically used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics determined by the error determination module 702 and the predicted speed determined by the variational autoencoder module 704.

[0156] In yet another possible implementation, the error determination module includes:

[0157] The gap calculation submodule is used to determine the attitude gap information between the predicted attitude information and the actual attitude information;

[0158] The error encoding module is used to encode the attitude difference information to generate attitude error features.

[0159] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the joint control methods provided in this application.

[0160] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the joint control methods provided in this application.

[0161] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0163] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0164] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A joint control method, comprising: Based on the robot's historical state data at at least one historical moment, the robot's predicted posture information at the current moment is predicted. The historical state data is used to characterize the robot's historical actual posture at that historical moment and the historical actions performed by each joint. Obtain the robot's actual posture information at the current moment; Based on the predicted posture information and the actual posture information, posture error features are determined. The posture error features are used to characterize the abnormal factors that cause errors in the historical actions of each joint of the robot at the previous moment. Based on the posture error characteristics, the actions to be performed by each joint of the robot at the current moment are determined.

2. The joint control method according to claim 1 further includes: Based on the historical state data of at least one historical moment, determine the joint state mask of the robot; The step of determining the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics includes: Based on the posture error features and the joint state mask, the actions to be performed by each joint of the robot at the current moment are determined.

3. The joint control method according to claim 1 or 2 further includes: Variational autoencoder processing is performed on the historical state data of the robot at at least one historical moment to obtain state coding features, which are used to characterize at least one of the robot's motion state features and the environmental features in which it is located. The prediction of the robot's pose information at the current moment based on the robot's historical state data at at least one historical moment includes: Based on the state coding features, the robot's predicted posture information at the current moment is predicted.

4. The joint control method according to claim 1 or 2, wherein determining the posture error characteristics based on the predicted posture information and the actual posture information includes: Determine the attitude gap information between the predicted attitude information and the actual attitude information; The attitude difference information is encoded to generate attitude error features.

5. The joint control method according to claim 2, further comprising: Variational autoencoder processing is performed on the historical state data of the robot at at least one historical moment to obtain state coding features, which are used to characterize at least one of the robot's motion state features and the environmental features in which it is located. Determining the joint state mask of the robot based on the historical state data of at least one historical moment includes: Based on the state coding features, the joint state mask of the robot is determined.

6. The joint control method according to claim 1, further comprising: Based on the historical state data of at least one historical moment, the predicted speed of the robot at the current moment is determined; The step of determining the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics includes: Based on the posture error characteristics and the predicted velocity, the actions to be performed by each joint of the robot at the current moment are determined.

7. A joint control device, comprising: The forward prediction module is used to predict the robot's posture information at the current moment based on the robot's historical state data at at least one historical moment. The historical state data is used to characterize the robot's historical actual posture at the historical moment and the historical actions performed by each joint. The error determination module is used to determine the posture error characteristics based on the predicted posture information and the actual posture information of the robot at the current moment. The posture error characteristics are used to characterize the abnormal factors that cause errors in the historical actions of each joint of the robot at the previous moment. The decision module is used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics.

8. The joint control device according to claim 7, wherein the forward prediction module is further configured to: determine the joint state mask of the robot based on the historical state data of the at least one historical moment; The decision module is specifically used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error features and the joint state mask.

9. The joint control device according to claim 7 or 8, further comprising: The variational autoencoder module is used to perform variational autoencoder processing on the historical state data of the robot at at least one historical moment to obtain state coding features. The state coding features are used to characterize at least one of the robot's motion state features and the environmental features in which it is located. The forward prediction module is specifically used to predict at least one of the robot's predicted posture information at the current moment and the joint state mask based on the state coding features.

10. The joint control device according to claim 7 or 8, further comprising: A variational autoencoder module is used to perform variational autoencoder processing on the historical state data of the robot at at least one historical moment to obtain the predicted speed of the robot at the current moment. The decision module is specifically used to determine the actions to be performed by each joint of the robot at the current moment based on the posture error characteristics and the predicted speed.

Images