Machine dog control method and device, electronic equipment and computer readable medium

Abstract

The application discloses a robot dog control method and device, electronic equipment and computer readable medium, wherein the control method comprises: in response to a balance prediction request, querying input data corresponding to the balance prediction request; inputting the input data into a running state prediction model to make the running state prediction model output a predicted running state of a next control frame; and correcting the expected running state of the next control frame according to a comparison result of the predicted running state and the expected running state of the next control frame. The application has the advantages that the control instruction of the robot dog with a load can be effectively corrected through a neural network model prediction and a comparison algorithm, thereby effectively overcoming the problem of imbalance of the robot dog walking caused by the load.

Description

Technical Field

[0001] This application relates to the field of robot control, and more specifically, to a robot dog control method, apparatus, electronic device, and computer-readable medium. Background Technology

[0002] A robot dog (quadruped robot) is an intelligent device that achieves autonomous movement through biomimetic design. Inspired by the quadrupedal locomotion of animals, it can move flexibly across various terrains. Equipped with advanced sensors and processors, it can perceive environmental changes in real time and make corresponding adjustments, thus achieving efficient autonomous navigation and task execution. Robot dogs have wide applications in many fields. In the consumer sector, they can serve as intelligent companion devices, providing users with interactive and entertainment functions; in the industrial sector, they are often used for inspection tasks, monitoring equipment and the environment with cameras and sensors to promptly identify potential problems. Furthermore, robot dogs can also be used in rescue and other fields, performing high-risk tasks such as detection and transportation, fully demonstrating their excellent adaptability and versatility.

[0003] In related technical fields, the balance control of robotic dogs typically relies on single-rigid-body dynamics models. Using model predictive control (MPC), the system can effectively track the trajectory of the robotic dog's actual center of mass and its desired center of mass. Simultaneously, efforts are made to maintain the zero-moment point (ZMP) at the center of the supporting polygon to ensure the robot's stability during movement. This method not only improves the robot's smoothness but also enhances its adaptability to complex terrain.

[0004] However, this is often based on knowing the exact position of the robot dog's center of mass in a single rigid body dynamics model. When the robot dog is under load, the load and the robot dog's center of mass become uncertain. If control is still based on the robot dog's own center of mass position, an imbalance may occur.

[0005] To address the issue of center of gravity shift caused by load, related technologies estimate the load weight by measuring the load changes (encoder current changes) of the robot dog's encoder. This allows the robot dog to walk in a lower posture under heavy loads, hoping to solve the imbalance problem caused by the increased center of gravity. However, increased road resistance (such as when climbing hills) can also cause encoder load changes. Therefore, without being able to measure the impact of the load weight alone, effective corrective control is not possible.

[0006] In other related technologies, the shape and mass of the load are estimated by adding additional sensors, such as a camera that takes pictures of the back, thereby re-determining the center of gravity and related correction information. There are also schemes that use pressure sensors to detect weight for correction. However, these related technologies will undoubtedly increase hardware costs, and they can only make a rough estimate from the characterization of shape or weight, and cannot achieve precise and effective control.

[0007] There is currently no effective solution to the aforementioned technical problems. Summary of the Invention

[0008] The summary section of this application is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0009] Some embodiments of this application propose methods, apparatuses, electronic devices, and computer-readable media to address the technical problems mentioned in the background section above.

[0010] As a first aspect of this application, some embodiments of this application provide a robot dog control method, including: responding to a balance prediction request, querying input data corresponding to the balance prediction request; inputting the input data into a running state prediction model to cause the running state prediction model to output a predicted running state of the next control frame; correcting the expected running state of the next control frame according to a comparison result of the predicted running state and the expected running state of the next control frame; wherein, the step of correcting the expected running state of the next control frame according to the comparison result of the predicted running state and the expected running state of the next control frame includes: comparing ZMP position data in the predicted running state and the expected running state of the next control frame; determining whether it is necessary to correct the expected running state of the next control frame according to the comparison result of the ZMP position data in the predicted running state and the expected running state of the next control frame; when it is determined that it is necessary to correct the expected running state of the next control frame, obtaining the changing trend of the expected running state of the current control frame and the actual running state of the previous control frame, and using the gradient value to reversely correct the expected running state of the next control frame.

[0011] Optionally, in some embodiments of this application, the running state prediction model is a neural network model based on a temporal convolutional network.

[0012] Optionally, in some embodiments of this application, the input data of the running state prediction model is a historical sequence of running data of the actual running state of the previous N control frames.

[0013] Optionally, in some embodiments of this application, the actual operating data includes: encoder position data, motor current data, IMU data, ZMP position data, and support polygon data.

[0014] Optionally, in some embodiments of this application, the predicted running state, the expected running state, and the actual running state have the same data type and order.

[0015] Optionally, in some embodiments of this application, comparing the ZMP position data in the predicted operating state and the expected operating state of the next control frame includes: determining whether the distance between the ZMPs in the predicted operating state and the expected operating state of the next control frame is greater than a first preset threshold; and / or determining whether the ratio of the distance between the ZMP and the supporting polygon in the predicted operating state to the distance between the ZMP and the supporting polygon in the expected operating state of the next control frame is greater than a second preset threshold; and / or determining whether the ZMPs in the predicted operating state and the expected operating state of the next control frame belong to the same cluster; and / or determining whether the centroids of the propagation in the predicted operating state and the expected operating state of the next control frame belong to the same cluster.

[0016] Optionally, in some embodiments of this application, the desired operating state is output by the model predictive control algorithm inherent in the robot dog.

[0017] As a second aspect of this application, some embodiments of this application provide a robot dog control device, mainly comprising: a query module, used to query input data corresponding to a balance prediction request in response to a balance prediction request; a prediction module, used to input the input data into a running state prediction model so that the running state prediction model outputs a predicted running state of the next control frame; and a correction module, used to correct the expected running state of the next control frame based on a comparison result between the predicted running state and the expected running state of the next control frame.

[0018] As a third aspect of this application, some embodiments of this application provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0019] As a fourth aspect of this application, some embodiments of this application provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any implementation of the first aspect above.

[0020] The beneficial effect of this application is that it can effectively correct the control commands of the robot dog when it is under load by using neural network model prediction and comparison algorithm, thereby effectively overcoming the problem of robot dog walking unbalanced due to load. Attached Figure Description

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application.

[0022] Furthermore, throughout the accompanying drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the elements are not necessarily drawn to scale.

[0023] In the attached diagram:

[0024] Figure 1 This is a schematic diagram of a robot dog provided according to some embodiments of this application;

[0025] Figure 2 This is a schematic diagram of the control architecture of a robot dog provided according to some embodiments of this application;

[0026] Figure 3 This is a schematic diagram of some hardware modules of a robot dog provided according to some embodiments of this application;

[0027] Figure 4 This is a schematic diagram of the center of mass position of a robot dog in an unloaded state, provided according to some embodiments of this application;

[0028] Figure 5 This is a schematic diagram of the overall (machine dog and load) centroid position of the robot dog in an unloaded state, according to some embodiments of this application.

[0029] Figure 6 This is a schematic diagram of the leg joints of a robot dog provided according to some embodiments of this application;

[0030] Figure 7 This is a schematic diagram of the grid space established around the robot dog according to some embodiments of this application;

[0031] Figure 8 It is based on Figure 7 The diagram shows a clustering of centroid locations within a grid space.

[0032] Figure 9 This is a schematic diagram of the main steps of a robot dog control method provided according to some embodiments of this application;

[0033] Figure 10This is a schematic diagram of the input and output of a centroid position prediction model provided according to some embodiments of this application;

[0034] Figure 11 This is a schematic diagram of the structure of a temporal convolutional network provided according to some embodiments of this application;

[0035] Figure 12 This is a schematic diagram of the structure of a single causal convolutional layer in a temporal convolutional network provided according to some embodiments of this application;

[0036] Figure 13 This is a schematic diagram of the input data for a centroid position prediction model provided according to some embodiments of this application;

[0037] Figure 14 This is a schematic diagram of a robot dog control device provided according to some embodiments of this application;

[0038] Figure 15 This is a schematic diagram of the structure of an electronic device provided according to some embodiments of this application.

[0039] Meaning of the reference numerals in the diagram:

[0040] 100. Robot dog;

[0041] 110. Base;

[0042] 120. Hip joint lateral movement motor;

[0043] 130. Hip joint pitch motor;

[0044] 140. Knee joint motor;

[0045] 150. Thigh;

[0046] 160. Calf;

[0047] 170. Transmission belt;

[0048] 180. Transmission pulley;

[0049] 200. Robot dog control device;

[0050] 210. Query module;

[0051] 220. Prediction module;

[0052] 230. Correction Module;

[0053] 300, load;

[0054] 800. Electronic equipment;

[0055] 801. Processing device;

[0056] 802, ROM;

[0057] 803, RAM;

[0058] 804, bus;

[0059] 805, I / O interface;

[0060] 806. Input device;

[0061] 807. Output device;

[0062] 808. Storage device;

[0063] 809. Communication device;

[0064] C1, Base axis;

[0065] C2, Thigh axis;

[0066] C3, Lower leg axis;

[0067] C4, Yaw axis;

[0068] S, supporting plane;

[0069] G1, Body center of mass;

[0070] G2, load centroid;

[0071] G3, Overall center of mass. Detailed Implementation

[0072] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0073] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0074] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0075] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0076] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0077] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0078] For ease of explanation of the control methods and other technical solutions of this application, please refer to... Figure 1 As shown, the embodiments of this application provide a typical structural scheme of a robot dog 100. The control scheme of this application is suitable for this structural scheme of the robot dog 100, but is not limited to this structural scheme of the robot dog 100.

[0079] Reference Figure 1 As shown, the robot dog 100 provided in this embodiment includes: a base 110, a hip joint yaw encoder, a hip joint pitch encoder, a knee joint encoder, a thigh 150, a lower leg 160, a drive belt 170, and a drive belt pulley 180.

[0080] The base 110 can be equipped with power supply, controller, IMU (Inertial Measurement Unit) and other devices. The base 110 can also serve as the main platform for carrying load 300 (such as goods or other equipment).

[0081] The base 110 is equipped with four hip joint yaw encoders, which can control the yaw angle of the thigh 150, that is, control the thigh 150 to be in an inward or outward yaw angle. Since the knee joint does not have a separate yaw joint and encoder in this solution, the hip joint yaw encoder can actually control the yaw angle of the entire leg (thigh 150 and calf 160 form a whole).

[0082] Hip joint pitch encoders are also installed at each of the four hip joints. These encoders control the pitch angle of the thigh 150. It should be noted that since the hip joint pitch encoders rotate with the hip joint yaw encoders, the adjustment here is not the absolute pitch angle (relative to the horizontal plane), but rather the angle of the thigh 150 relative to the hip joint yaw axis C4. The hip joint pitch encoders are mainly used to drive the thigh 150, thereby changing the posture (as a standing leg) and stride (as a swinging leg) of the robot dog 100.

[0083] Considering factors such as counterweight, the knee joint encoder is actually installed at the hip joint. It drives the pulley 180 at the knee joint (the joint where the thigh 150 and the lower leg 160 are rotatably connected) via the transmission belt 170. This can change the angle of the lower leg 160 relative to the thigh 150. The knee joint encoder is mainly used to drive the lower leg 160, thereby changing the posture (as a standing leg) and stride (as a swinging leg) of the robot dog 100.

[0084] Reference Figure 6 As shown, from the side view plane (left view plane or right view plane, i.e.) Figure 6 From the perspective of the paper (the plane of view), the angle α between the base axis C1 (or the yaw axis C4) and the thigh axis C2, and the angle β between the thigh axis C2 and the lower leg axis C3, can be controlled by comprehensively controlling the angle θ between the base axis C1 and the support plane S (a virtual plane), where angles α, β, and θ are all pitch angles; additionally, from the front view plane (or the rear view plane, not shown in the figure, i.e., vertical) Figure 6 Regarding the plane of the paper and the deflection axis, the degree of inward and outward tilt of the thigh 150 can be controlled by the angle between the thigh axis C2 and the supporting plane S (not shown in the figure), which is the yaw angle. It can be understood that the hip joint yaw encoder, hip joint pitch encoder, and knee joint encoder can not only realize the walking leg swing action, but also realize the adjustment of the walking posture of the robot dog 100, that is, control the expected position of the center of gravity of the robot dog 100.

[0085] The specific driving methods and the structure of the hip joint yaw encoder, hip joint pitch encoder, and knee joint encoder are technical solutions familiar to those skilled in the art and are not improvements of this application, so they will not be described in detail here.

[0086] It should be noted that the above angles can be detected by additional angle sensors, such as the hip joint yaw encoder, hip joint pitch encoder, or the encoder built into the knee joint encoder.

[0087] Reference Figure 2 As shown, Figure 2 A typical architecture of the control system of the robot dog 100 provided by some embodiments of this application is shown.

[0088] The control system mainly includes: an instruction setter, a planner, a controller, an estimator, an API interface, and a hardware layer.

[0089] The command unit is mainly used to implement upper-level planning, such as path planning based on user instructions and obstacle avoidance requirements (obtained based on vision or LiDAR), and then generating motion instructions and gait phase instructions based on the path planning. Among them, motion instructions are used to control the overall movement direction and speed of the robot dog 100; gait phase instructions are used to control the leg gait mode and specific gait phase of the robot dog 100.

[0090] Gait patterns generally include: amble, trot, bound, pace, pronk, etc.

[0091] The specific gait patterns and implementation schemes are technical solutions familiar to those skilled in the art and are not improvements of this application, so they will not be described in detail here.

[0092] Understandably, the magnitude of the change in the center of gravity of the robot dog 100 varies under different gait modes, and the requirements for balance control also differ.

[0093] The module responsible for motion commands in the controller outputs relevant data on the speed and angular velocity of the robot dog 100, while the module responsible for gait phase commands in the controller outputs relevant data on the state and phase of the robot dog 100's legs; that is, the controller is used to output the requirements for the overall and leg movements of the robot dog 100.

[0094] The planner receives data sent by the controller and then decomposes it into centroid trajectory planning, landing point planning, and swing leg trajectory planning according to the controller's top-level planning requirements.

[0095] The centroid referred to in the centroid trajectory planning here refers to the centroid of the robot dog 100 itself, defined below as the body centroid G1. It should be noted that, according to the robot dog 100 centroid trajectory planning algorithm, the body centroid G1 is not the actual centroid of the robot dog 100 in various gait states. This is because the robot dog 100 centroid trajectory planning algorithm is based on assuming the robot dog 100 is a single rigid body dynamic model, that is, assuming the robot dog 100's legs are massless light rods, and representing the robot dog 100 as a whole using a rigid body whose centroid and moment of inertia do not change. This is also why the knee joint encoder is also installed at the hip joint, in order to minimize the weight of the device at the knee joint.

[0096] The planner generates the base 110 trajectory and foot trajectory based on the planned center of mass trajectory, landing point, and swing leg trajectory. The base 110 trajectory represents the motion trajectory of the base 110, and the foot trajectory represents the motion trajectory of the lower leg 160 end (which can also be considered equivalent to the foot end). The planner sends the relevant data of the base 110 trajectory and foot trajectory to the controller.

[0097] The controller decomposes the data based on the trajectory of the base 110 and the foot trajectory into standing leg control and swing leg control. The standing leg refers to the leg used for support, and the swing leg refers to the leg that is suspended in the air. It can be understood that in different gait modes, a portion of the leg can be used as the standing leg and a portion as the swing leg, or both can be used as the standing leg or the swing leg.

[0098] For example, in a diagonal gait, the diagonal leg can function as either the standing leg or the swinging leg; in a bouncy gait, all legs can function as either the standing leg or the swinging leg.

[0099] The controller can convert the standing leg control and swing leg control into parameters such as joint torque, angle and speed of each joint, and then send them to the API interface; the API interface will standardize the received data and then convert it into control parameters to be sent to various hardware, such as the torque value sent to the microcontroller that drives the encoder.

[0100] Reference Figure 3 As shown, the hardware layer mainly includes actuators and sensors. The actuators mainly include hip joint yaw encoders, hip joint pitch encoders, and knee joint encoders. The sensors can include IMUs, and the hip joint yaw encoders, hip joint pitch encoders, and knee joint encoders embedded in them can also be used as sensors. Because the hip joint yaw motor 120, hip joint pitch motor 130, and knee joint motor 140 have a fixed transmission relationship with their corresponding joints, as well as the thigh 150 and lower leg 160, the rotational position of the corresponding joints, as well as the rotational position of the thigh 150 and lower leg 160, can be determined by detecting the rotational position of the motor shaft. In this way, the hip joint yaw encoders, hip joint pitch encoders, and knee joint encoders, which are originally designed to drive the motors, can be reused.

[0101] It should be noted that the data from the API interface is not sent directly to the hip yaw motor 120, hip pitch motor 130, and knee motor 140, but is sent to the microcontroller or other logic control device that controls them. These logic control devices generate corresponding drive signals based on the data, and then drive the semiconductor switches in the drive circuits of the hip yaw motor 120, hip pitch motor 130, and knee motor 140 through the drive signals, thereby ultimately controlling the operation of the hip yaw motor 120, hip pitch motor 130, and knee motor 140.

[0102] The estimator, a crucial component of the robot dog's 100 control system, primarily predicts and calculates upcoming states. Generally, the estimator performs kinematic estimation of the base 110, terrain estimation, and foot contact estimation. Base 110 kinematic estimation predicts the base's trajectory; terrain estimation predicts routes and obstacles; and foot contact estimation predicts foot position and whether it provides effective support. The estimator effectively achieves dynamic control by feeding the predicted data back to the controller and planner. The estimator can obtain data (or converted signals) from various sensors via the API interface.

[0103] The desired operating state in the control method of this application is estimated by the estimator based on existing control algorithms (such as the model predictive control algorithm described below). Before the improvement of this application, the estimator directly transmitted the estimation result to the instruction controller and planner for pre-control. The actual operating state of this application can be considered as the actual operating condition represented by the actual detection data output by the API interface.

[0104] The specific implementations of the aforementioned control system and its instruction setter, planner, controller, estimator, API interface, and hardware layer are technical solutions familiar to those skilled in the art and will not be elaborated upon here.

[0105] The hardware structure and control system architecture on which the control method provided in this application depends have been described above. The control method of this application will be described in detail below in conjunction with the above content.

[0106] Based on the aforementioned hardware structure and control system architecture, the core idea of ​​the current robot dog 100's balance control is to adjust the leg state in real time to ensure that the projection and motion state of the body's center of mass G1 meet the stability conditions.

[0107] As a specific implementation scheme, Model Predictive Control (MPC) is generally used for control implementation. Model Predictive Control can establish a kinematic model based on the single rigid body assumption, thereby predicting the motion state of the robot dog 100 and thus achieving dynamic control.

[0108] Specifically, the model predictive control used by the robot dog 100 requires an internal mathematical model describing the system's dynamics to predict the future. For the robot dog 100, this is usually based on a simplified dynamic model (such as a single rigid body model) to predict the future trajectory of the body's center of mass G1, body posture, and ZMP (Zero Moment Point) based on the current state and assumed control inputs; then, the body's center of mass G1 or ZMP is controlled within the rectangular area (i.e., the support polygon) defined by the foot of the standing leg.

[0109] The specific method of using model predictive control algorithm combined with ZMP control algorithm to balance the robot dog 100 is a technical solution familiar to those skilled in the art and will not be elaborated here. The control method of this application mainly addresses how to correct the overall balance between the robot dog 100 and the load 300 when the addition of load 300 causes the robot dog 100 and the load 300 to deviate from the body's center of mass G1, thereby satisfying the overall balance between the robot dog 100 and the load 300.

[0110] Reference Figure 4 and Figure 5 As shown, after the load 300 is added to the base 110 of the robot dog 100, the overall centroid G3 of the robot dog 100 and the load 300 changes due to the presence of the load centroid G2.

[0111] According to the traditional solution, the model predictive control algorithm should be updated to model the overall centroid G3 as the rigid body centroid assumed by the model. However, due to the diversity of load 300 and whether the robot dog 100 and load 300 as a whole satisfy the rigid body assumption, the model building difficulty is increased. Therefore, the traditional remodeling method cannot solve the above problems.

[0112] The core idea of ​​the robot dog 100 control method in this application is that the change in the center of mass caused by the load 300 cannot exceed the balancing capability of the robot dog 100 hardware. Therefore, the overall center of mass G3 can only be balanced if the range is within a reasonable space. The balance control correction in this application does not attempt to obtain the position of the overall center of mass G3 and update the model predictive control algorithm. Instead, it compares the predicted operating state (reflecting the impact of the load 300) with the expected operating state (the result of the inherent balance control) to determine whether the hardware capability of the robot dog 100 and the original model predictive control algorithm still have room for adaptive adjustment. If so, control is still based on the original model predictive control algorithm (indicating that the load 300 has little impact). If the trend of the original model predictive control algorithm tends to be unbalanced, then correction control is performed.

[0113] Reference Figure 9 As shown, the control method for the robot dog 100 provided in this application mainly includes the following steps:

[0114] S1: In response to the balance prediction request, query the input data corresponding to the balance prediction request.

[0115] S2: Input the input data into the running state prediction model so that the running state prediction model outputs the predicted running state of the next control frame.

[0116] S3: Based on the comparison between the predicted operating state and the expected operating state of the next control frame, correct the expected operating state of the next control frame.

[0117] In some embodiments of this application, the running state prediction model is a neural network model based on a temporal convolutional network.

[0118] In some embodiments of this application, the input data of the running state prediction model is a historical sequence of running data of the actual running state of the previous N control frames.

[0119] In some embodiments of this application, the actual operating data includes: encoder position data, motor current data, IMU data, ZMP position data, and support polygon data (generally quadrilaterals).

[0120] In some embodiments of this application, the data types and order of the predicted running state, the expected running state, and the actual running state are the same.

[0121] As can be seen from the above, in some embodiments of this application, the expected operating state is output by the model predictive control algorithm inherent in the robot dog 100.

[0122] Reference Figures 10 to 12 As shown below, the runtime prediction model (i.e., a temporal convolutional network model) of this application will be described in detail.

[0123] The temporal convolutional network model of this application includes an input layer, an output layer, and M cascaded residual modules. Each residual module includes two sets: a Dropout layer (or regularization layer), an activation function layer (the activation function is ReLU), a weight normalization layer, and a dilated causal convolutional layer (or extended causal convolutional layer).

[0124] Reference Figure 12 As shown, each dilated causal convolutional layer includes an output layer, a hidden layer, and an input layer. The structure and transmission mechanism of the dilated causal convolutional layer give it two prominent features: causal convolution and synchronous computation.

[0125] Causal convolution is the cornerstone of temporal convolutional network models. By performing unidirectional convolution on the input sequence, it ensures that the output at time t depends only on the input at time t and before, never "seeing" future data; this aligns with physical causality. Unlike RNNs / LSTMs, which must compute sequentially over time steps, temporal convolutional network models can process the entire input sequence in parallel, resulting in significant advantages in training and inference speed, crucial for the real-time control of the robot dog 100. Furthermore, temporal convolutional network models insert "holes" between elements of the standard convolutional kernel, allowing the network to exponentially expand its receptive field without increasing parameters or computational cost. For example, stacked layers with dilation coefficients d of 1, 2, 4, 8... allow higher-level neurons to "see" the very distant past, essential for capturing long-range dependencies such as the gait cycle of the robot dog 100.

[0126] The temporal convolutional network model in this application can be implemented based on PyTorch.

[0127] Reference Figure 13 As shown, as a specific scheme, the input data of the operation status prediction model of this application is a data matrix, where each row of the data matrix represents a different historical control frame, and each column of the data matrix represents a different data item.

[0128] For example, the first row of data is a series of data from the most recent control frame (also known as the control step size), such as: left anterior hip joint yaw encoder data, right anterior hip joint yaw encoder data, left posterior hip joint yaw encoder data, right posterior hip joint yaw encoder data, left anterior hip joint yaw motor 120 current data, right anterior hip joint yaw motor 120 current data, left posterior hip joint yaw motor 120 current data, right posterior hip joint yaw motor 120 current data, left posterior hip joint yaw motor 120 current data, right posterior hip joint yaw motor 120 current data, left anterior knee joint encoder data, right anterior knee joint encoder data, left posterior knee joint encoder data, right posterior knee joint encoder data, left anterior knee joint motor 140 current data, right anterior knee joint motor 140 current data, left posterior knee joint motor 140 current data, right posterior knee joint motor 140 current data, IMU X-axis acceleration, IMU Y-axis acceleration, IMU... Z-axis acceleration, IMU X-axis angular velocity, IMU Y-axis angular velocity, IMU Z-axis angular velocity, X-axis coordinate data of body centroid G1, Y-axis coordinate data of body centroid G1, Z-axis coordinate data of body centroid G1, X-axis coordinate of ZMP, Y-axis coordinate of ZMP, Z-axis coordinate of ZMP, X-axis coordinate of the first corner point of the supporting polygon, Y-axis coordinate of the first corner point of the supporting polygon, Z-axis coordinate of the first corner point of the supporting polygon, X-axis coordinate of the second corner point of the supporting polygon, Y-axis coordinate of the second corner point of the supporting polygon, Z-axis coordinate of the second corner point of the supporting polygon, X-axis coordinate of the third corner point of the supporting polygon, Y-axis coordinate of the third corner point of the supporting polygon, Z-axis coordinate of the third corner point of the supporting polygon, X-axis coordinate of the fourth corner point of the supporting polygon, Y-axis coordinate of the fourth corner point of the supporting polygon, Z-axis coordinate of the fourth corner point of the supporting polygon. That is... Figure 13 The data matrix shown has 40 columns; N in the first N frames of control can take an integer value between 100 and 200, with 100 being preferred. Figure 13 The data matrix shown has 100 rows.

[0129] The output data of the operational state prediction model can also be a data matrix, with columns identical to the input matrix. In this application, the output data consists of only one row, representing the state data matrix for the next control frame. While it's possible to predict multiple frames, meaning the output matrix could have multiple rows, this is unsuitable for the high-speed response requirements of the robot dog 100. Therefore, the output data matrix in this application consists of only one row.

[0130] The training of the operational status prediction model in this application can be achieved by slicing the real operational status in historical data to obtain a sufficient training set and validation set. Since the control frequency of the robot dog 100 is relatively high, sufficient training data can be obtained.

[0131] The purpose of using the running state prediction model is to establish a prediction model independent of the estimator, thereby obtaining an independent predicted running state, and then comparing the predicted running state with the estimator's expected running state.

[0132] In some embodiments of this application, step S3 may include:

[0133] S31: Compare the predicted running state with the ZMP position data in the expected running state of the next control frame.

[0134] S32: Based on the comparison results between the predicted operating state and the expected operating state in the next control frame, determine whether it is necessary to correct the expected operating state of the next control frame.

[0135] S33: When it is determined that the expected operating state of the next control frame needs to be corrected, the changing trend of the expected operating state of the current control frame and the actual operating state of the previous control frame is obtained, and the expected operating state of the next control frame is corrected in reverse using the gradient value.

[0136] Step S31 may specifically include the following sub-steps:

[0137] S311: Determine whether the distance between the predicted running state and the ZMP in the expected running state of the next control frame is greater than a first preset threshold.

[0138] S312: Determine whether the ratio of the distance between the ZMP and the supporting polygon in the predicted running state to the distance between the ZMP and the supporting polygon in the expected running state of the next control frame is greater than a second preset threshold.

[0139] S313: Determine whether the ZMP in the predicted running state and the expected running state in the next control frame belong to the same cluster;

[0140] S314: Determine whether the ontological centroids in the predicted running state and the expected running state in the next control frame belong to the same cluster.

[0141] Steps S311 to S314 can be executed sequentially, or only a portion of them can be executed.

[0142] Specifically, comparing the ZMP position data in the current predicted operating state with the expected operating state in the next control frame mainly involves comparing whether the deviation between the ZMP position in the current predicted operating state and the ZMP position in the expected operating state in the next control frame is acceptable.

[0143] As an optional approach, the distance between the ZMP position of the current predicted running state and the ZMP position in the expected running state of the next control frame can be calculated. If the distance is greater than a first preset threshold, it is determined that the expected running state given by the estimator needs to be corrected.

[0144] As a further solution, to avoid the limitations of judging solely based on distance, the distance between the ZMP position in the current expected operating state and the supporting polygon in the current predicted operating state can be calculated. Then, the distance between the ZMP position in the expected operating state of the next control frame and the supporting polygon in the expected operating state of the next control frame can be calculated. If the ratio of the former to the latter is less than a second preset threshold, it is determined that no correction is needed for the expected operating state of the next control frame given by the estimator. If it is greater than the threshold, the expected operating state given by the estimator needs to be corrected.

[0145] As a further step, refer to Figure 7 and Figure 8 As shown, a grid space (or a 3D coordinate system) is established around the robot dog 100. Then, all ZMP position data for the expected, desired, and actual operating states during the control process are stored. Corresponding coordinate points are generated in the grid space based on the ZMP position data, and then K-means clustering is performed. Comparing the ZMP position data in the predicted operating state with the expected operating state of the next control frame determines whether the coordinate points corresponding to the ZMP position data in the current predicted operating state and the expected operating state of the next control frame are in the same cluster after clustering. If they are, it is determined that no correction is needed for the expected operating state of the next control frame given by the estimator; otherwise, the expected operating state of the next control frame given by the estimator needs to be corrected.

[0146] Based on this, coordinates of all ontological centroids G1 in the expected, desired, and actual operating states are generated in the grid space, and K-means clustering is performed on them as well. It is then determined whether the coordinates of the ontological centroids G1 in the current predicted operating state and the expected operating state of the next control frame are in the same cluster. If they are, it is determined that no correction is needed for the expected operating state of the next control frame given by the estimator; otherwise, the expected operating state of the next control frame given by the estimator needs to be corrected. This further ensures the accuracy of the correction intervention.

[0147] Of course, all of the above judgment conditions can be treated as parallel conditions. That is, only when all of them are satisfied is it determined that no correction is needed to the expected operating state given by the estimator, and if any one of them is not satisfied, it is determined that the expected operating state of the next control frame given by the estimator needs to be corrected.

[0148] The correction method in this application involves comparing the changing trends of some preset data items (preset column comparison) in the desired operating state and the actual operating state of the previous control frame (ideally, the current frame would be compared, but due to detection delay and to improve feedback efficiency, the previous control frame is chosen here) with the opposite trend direction to perform reverse correction on the gradient value (absolute value) of the corresponding data item. For example, if the left anterior hip joint yaw encoder data is 10 in the desired operating state and 9 in the actual operating state of the previous control frame, and the gradient value of this item is 0.5, then its changing trend is an increase of 1. Therefore, during correction, the gradient value of 0.5 is subtracted in the opposite direction, resulting in a final corrected value of 9.5.

[0149] As a specific solution, since the balance problem is mainly caused by posture, and in order to reserve sufficient margin for the hardware under the core concept of this application, for the above-mentioned items, only the items that affect posture can be modified, while the items that affect strength can be modified. For example, the Y-axis coordinate data and Z-axis coordinate data of the body centroid G1 can be modified for the left front hip joint yaw encoder data, right front hip joint yaw encoder data, left rear hip joint yaw encoder data, right rear hip joint yaw encoder data, left front knee joint encoder data, right front knee joint encoder data, left rear knee joint encoder data, right rear knee joint encoder data, body centroid G1, and ZM can be modified for the above-mentioned items. The X-axis coordinates of P, the Y-axis coordinates of ZMP, the Z-axis coordinates of ZMP, the X-axis coordinates of the first corner point of the supporting polygon, the Y-axis coordinates of the first corner point of the supporting polygon, the Z-axis coordinates of the first corner point of the supporting polygon, the X-axis coordinates of the second corner point of the supporting polygon, the Y-axis coordinates of the second corner point of the supporting polygon, the Z-axis coordinates of the second corner point of the supporting polygon, the X-axis coordinates of the third corner point of the supporting polygon, the Y-axis coordinates of the third corner point of the supporting polygon, the Z-axis coordinates of the third corner point of the supporting polygon, the X-axis coordinates of the fourth corner point of the supporting polygon, the Y-axis coordinates of the fourth corner point of the supporting polygon, and the Z-axis coordinates of the fourth corner point of the supporting polygon are corrected, while the motor current is not corrected.

[0150] As an optional solution, if the comparison result still does not meet the requirement of no correction after a preset number of corrections, the aforementioned gradient correction can be applied to the remaining items. The items to be gradient corrected can be added gradually, either in a preset order or a randomly generated sequence. The gradient values ​​for different items can be set according to their characteristics, and different gradient values ​​need to be set for different time-state modes to improve the correction's response speed to different time-states and avoid the correction gradient affecting the final balance effect.

[0151] Reference Figure 14As shown, as another aspect of this application, the robot dog control device 200 provided in the embodiments of this application mainly includes: a query module 210, a prediction module 220, and a correction module 230.

[0152] The query module 210 is used to query the input data corresponding to the balance prediction request in response to the balance prediction request; the prediction module 220 is used to input the input data into the operation state prediction model so that the operation state prediction model outputs the predicted operation state of the next control frame; the correction module 230 is used to correct the expected operation state of the next control frame according to the comparison result between the predicted operation state and the expected operation state of the next control frame.

[0153] Reference Figure 15 As shown, the electronic device 800 may include a processing device 801 (e.g., a central processing unit, a graphics processor, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM 802) or a program loaded from a storage device 808 into a random access memory (RAM 803). The RAM 803 also stores various programs and data required for the operation of the electronic device 800. The processing device 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface (hereinafter referred to as I / O interface 805) is also connected to the bus 804.

[0154] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although... Figure 15 An electronic device 800 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 15 Each box shown can represent a device or multiple devices as needed.

[0155] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, it performs the functions defined in the methods of some embodiments of this disclosure.

[0156] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium may be, for example, (but not limited to) an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM 803), read-only memory (ROM 802), erasable programmable read-only memory (EPROM 802 or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM 802), optical storage device, magnetic storage device, or any suitable combination thereof.

[0157] In some embodiments of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0158] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0159] The aforementioned computer-readable medium may be included in the aforementioned electronic device 800, or it may exist independently without being assembled into the electronic device 800. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device 800, cause the electronic device 800 to: respond to a balance prediction request by querying the input data corresponding to the balance prediction request; input the input data into a running state prediction model to cause the running state prediction model to output a predicted running state for the next control frame; and correct the expected running state of the next control frame based on a comparison between the predicted running state and the expected running state of the next control frame.

[0160] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to any type of network, including a local area network (LAN) or a wide area network (WAN), such as LAN or WAN X, or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function.

[0162] It should also be noted that in some alternative implementations, the functions marked in the box may occur in a different order than those marked in the attached figures.

[0163] For example, two consecutively represented blocks can actually be executed in substantially parallel order, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, as well as combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0164] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a query unit, a prediction unit, and a correction unit. The names of these units do not necessarily limit the specific unit; for example, a query unit may also be described as "a unit for querying corresponding data."

[0165] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), etc.

[0166] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for controlling a robot dog, characterized in that, include: In response to a balance prediction request, query the input data corresponding to the balance prediction request; The input data is fed into the running state prediction model so that the running state prediction model outputs the predicted running state of the next control frame; The expected operating state of the next control frame is corrected based on the comparison result between the predicted operating state and the expected operating state of the next control frame. The step of correcting the expected operating state of the next control frame based on the comparison result between the predicted operating state and the expected operating state of the next control frame includes: Compare the predicted operating state with the ZMP position data in the expected operating state of the next control frame; Based on the comparison result between the predicted operating state and the expected operating state in the next control frame, it is determined whether the expected operating state in the next control frame needs to be corrected. When it is determined that the expected operating state of the next control frame needs to be corrected, the changing trend of the expected operating state of the current control frame and the actual operating state of the previous control frame is obtained, and the expected operating state of the next control frame is corrected in reverse using the gradient value.

2. The robot dog control method according to claim 1, characterized in that, The operational status prediction model is a neural network model based on a temporal convolutional network.

3. The robot dog control method according to claim 2, characterized in that, The input data for the operation state prediction model is a historical sequence of operation data of the actual operation state of the previous N control frames.

4. The robot dog control method according to claim 3, characterized in that, The actual operating data includes: encoder position data, motor current data, IMU data, ZMP position data, and support polygon data.

5. The robot dog control method according to claim 4, characterized in that, The predicted operating state, the expected operating state, and the actual operating state have the same data type and order.

6. The robot dog control method according to claim 5, Its features are, The step of comparing the predicted operating state with the expected operating state in the next control frame includes: Determine whether the distance between the predicted running state and the ZMP in the expected running state of the next control frame is greater than a first preset threshold. And / or, determine whether the ratio of the distance between the ZMP and the supporting polygon in the predicted running state to the distance between the ZMP and the supporting polygon in the expected running state of the next control frame is greater than a second preset threshold. And / or, determine whether the predicted running state and the ZMP in the expected running state of the next control frame belong to the same cluster; And / or, determine whether the ontology centroids in the predicted running state and the expected running state in the next control frame belong to the same cluster.

7. The robot dog control method according to any one of claims 1 to 6, characterized in that, The desired operating state is output by the model predictive control algorithm inherent in the robot dog.

8. A robot dog control device, characterized in that, include: The query module is used to query the input data corresponding to the balance prediction request in response to the balance prediction request; The prediction module is used to input the input data into the running state prediction model so that the running state prediction model outputs the predicted running state of the next control frame; The correction module is used to correct the expected operating state of the next control frame based on the comparison result between the predicted operating state and the expected operating state of the next control frame. The step of correcting the expected operating state of the next control frame based on the comparison result between the predicted operating state and the expected operating state of the next control frame includes: Compare the predicted operating state with the ZMP position data in the expected operating state of the next control frame; Based on the comparison result between the predicted operating state and the expected operating state in the next control frame, it is determined whether the expected operating state in the next control frame needs to be corrected. When it is determined that the expected operating state of the next control frame needs to be corrected, the changing trend of the expected operating state of the current control frame and the actual operating state of the previous control frame is obtained, and the expected operating state of the next control frame is corrected in reverse using the gradient value.

9. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that, The computer-readable medium stores a computer program, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images