A kind of whole body control method and system of legged robot based on graph model

Abstract

The application provides a kind of full-body control method and system of leg type operating robot based on graph model, belongs to robot control field, method includes: with the perception unit of robot and actuating mechanism establish node set, with the physical structure connection relationship of robot establish edge set, with the tensor splicing of multi-source perception data obtain the feature vector of node, the feature vector of node is made into the feature matrix of node, graph model is constructed with node set, edge set and the feature matrix of node;Graph model is input into permutation invariant graph encoder and is encoded, to obtain fusion feature;The task to be controlled is decomposed into subtask using high-level planner in strategy network, in each subtask, determine trigger condition according to fusion feature, according to trigger condition, output corresponding instruction;According to corresponding instruction, using low-level controller outputs corresponding action to control robot.The application realizes the internal collaborative optimization of movement and operation by dynamically aggregating the heterogeneous features of each node.

Description

Technical Field

[0001] This invention belongs to the field of robot control, specifically relating to a whole-body control method and system for a legged manipulator based on a graph model. Background Technology

[0002] Currently, quadruped robots, with their superior terrain adaptability, have become ideal mobile platforms in complex scenarios such as field exploration, disaster relief, and industrial inspection. With the integration of multi-degree-of-freedom robotic arms, these platforms have been further expanded into legged manipulators that combine mobility and manipulation functions. In unstructured scenarios, they can perform tasks such as opening doors, carrying materials, and maintaining equipment, thus improving the robot's ability to operate under extreme conditions.

[0003] In recent years, Graph Neural Networks (GNNs) have achieved significant results in various fields such as point cloud processing, molecular structure analysis, and social network mining due to their superior capabilities in topological relationship modeling and heterogeneous information fusion. However, in the field of whole-body control of multi-degree-of-freedom robots, the application of GNNs is still in the initial exploratory stage, and a mature technical solution has not yet been formed.

[0004] Data-driven learning methods have significantly enhanced the motion performance of quadruped robots, enabling them to move stably in complex terrain. Thanks to advancements in intelligent grasping technology, legged manipulators, with their combined mobility and maneuverability in complex environments, have become a research hotspot in the field of robotics. Currently, research on the whole-body control of legged manipulators mainly follows two technological routes:

[0005] (1) Modular decomposition strategy: This type of method decomposes the robot system into two independent modules, motion and operation, which are trained and optimized separately, and finally the overall control is achieved through a higher-level strategy. Although this method has achieved results in specific tasks, the independent optimization at each stage leads to insufficient coordination between motion and operation, which greatly limits the robot's ability to perform fine operations in complex tasks.

[0006] (2) End-to-end learning methods: These methods use imitation learning or single-agent reinforcement learning to learn control strategies by mapping states to joint movements end-to-end. In addition, another type of research uses trajectory optimization methods to generate reference trajectories and combines them with deep reinforcement learning to achieve target state tracking control. However, due to the strong dynamic coupling between robot body motion and upper limb manipulation, existing end-to-end methods based on single-policy models often struggle to fully model and optimize the whole-body system, resulting in insufficient generalization ability and robustness in complex tasks.

[0007] The above methods have made significant progress in improving the motion and manipulation capabilities of legged robots, but problems such as insufficient motion and manipulation coordination and unstable whole-body coordination performance still exist, and there is an urgent need to explore more efficient whole-body control methods. Summary of the Invention

[0008] To address the shortcomings of existing technologies, a whole-body control method and system for legged manipulators based on graph models is proposed.

[0009] In a first aspect, the present invention provides a method for whole-body control of a legged manipulator based on a graph model, comprising:

[0010] A node set is established using the robot's sensing units and actuators, an edge set is established using the physical structural connections of the robot, and the feature vectors of the nodes are obtained by concatenating tensors of multi-source sensing data. The feature vectors of the nodes are then combined to form the feature matrix of the nodes, and a graph model is constructed using the node set, edge set, and feature matrix of the nodes.

[0011] The graph model is input into a permutation-invariant graph encoder for encoding to obtain fused features;

[0012] The high-level planner in the policy network is used to decompose the task to be controlled into subtasks. In each subtask, the triggering condition is determined based on the fusion characteristics, and the corresponding instruction is output according to the triggering condition.

[0013] Based on the corresponding instructions, the lower-level controller outputs the corresponding actions to control the robot.

[0014] The establishment of a node set based on the robot's sensing units and actuators includes:

[0015] The robot's inertial measurement unit is used as the body node, the robot's leg joints are used as motion nodes, and the robot's robotic arm joints are used as operation nodes.

[0016] Create a node set for the body node, motion node, and operation node.

[0017] The establishment of edge sets based on the physical structural connections of the robot includes:

[0018] Construct a topological adjacency matrix based on the physical connection structure between any two nodes in the node set;

[0019] Calculate the feature cosine similarity between any two nodes in the node set.

[0020] Select the k nodes with the highest feature cosine similarity as neighbor nodes, set the neighbor nodes to 1 and the rest nodes to 0, and obtain the sparse feature adjacency matrix.

[0021] The topological adjacency matrix and the sparse feature adjacency matrix are fused to obtain the final adjacency matrix, which is then used as the edge set.

[0022] The construction of a topological adjacency matrix based on the physical connection structure between any two nodes in the node set includes:

[0023] Based on the physical connection structure between any two nodes in the node set, a basic topological adjacency matrix is ​​constructed, and the calculation formula is as follows:

[0024] ;

[0025] Where A is the basic topological adjacency matrix. Given any element in the topological adjacency matrix based on n, where n is the number of nodes;

[0026] Based on the basic topological adjacency matrix, an identity matrix and weight coefficients are introduced to construct a topological adjacency matrix with added self-loops, calculated as follows:

[0027] ;

[0028] in, It is a topological adjacency matrix. These are the weighting coefficients of the identity matrix. It is an identity matrix.

[0029] The feature matrix of a node is composed of the feature vector of each node, and the feature vector is calculated as follows:

[0030] ;

[0031] in, Let v be the feature vector of the v-th node. The global point cloud features extracted by the point cloud network are denoted by [symbol]. This indicates a feature concatenation operation. The acceleration measured by the inertial measurement unit. Angular velocity measured by the inertial measurement unit. For the fuselage attitude quaternion, Let be the single joint angle of the v-th node. Let v be the angular velocity of the v-th node. The torque at the v-th node, For the ontology node, For the j-th moving node, This is the kth operation node.

[0032] The topological adjacency matrix and the sparse feature adjacency matrix are fused to obtain the final adjacency matrix, calculated as follows:

[0033] ;

[0034] in, This is the final adjacency matrix. Let be the degree matrix of the topological graph. Let be the degree matrix of the feature map. It is a topological adjacency matrix. For the characteristic adjacency matrix, The weights for merging the two graphs.

[0035] The process of inputting the graph model into a permutation-invariant graph encoder to obtain fused features includes:

[0036] Expand the feature vector of each node to construct an initial feature matrix, and use the initial feature matrix as the input of the graph encoder;

[0037] Based on the shared weights of each layer, a three-level progressive feature extraction method is used to obtain the feature matrix of each layer of the graph network.

[0038] Based on the feature matrix of the third-layer graph network, permutation-invariant global pooling is performed to obtain the fused features;

[0039] The fused features at different times are spliced ​​together to form a reinforcement learning state space, which is then used as the output of the graph encoder.

[0040] Based on the shared weights of each layer, a three-level progressive feature extraction method is used to obtain the feature matrix of each layer of the graph network. The calculation formula is as follows:

[0041] ;

[0042] in, For the first Feature matrix of layer nodes For the first Layer feature dimension For the shared weights of layer l, For the first Layer feature dimension For the bias of the l-th layer, This is the final adjacency matrix. for The activation function, where n is the number of nodes.

[0043] The fused features are obtained by performing permutation-invariant global pooling based on the feature matrix of the third-layer graph network, as calculated by the following formula:

[0044] ;

[0045] in, This is the feature matrix of the third layer graph network. yes The List, Let be the infinite norm of the vector. This is a feature of fusion.

[0046] Secondly, the present invention also provides a whole-body control system for a legged manipulator based on a graph model, comprising:

[0047] The graph model construction module is used to establish a node set based on the robot's sensing units and actuators, establish an edge set based on the physical structure connection relationship of the robot, obtain the feature vector of the node by concatenating tensors of multi-source sensing data, form the feature matrix of the node by combining the feature vector of the node, and construct the graph model with the node set, edge set and feature matrix of the node.

[0048] The graph encoding module is used to input the graph model into the permutation-invariant graph encoder for encoding to obtain fused features;

[0049] The instruction output module is used to decompose the task to be controlled into subtasks using the high-level planner in the policy network. In each subtask, the triggering condition is determined based on the fusion characteristics, and the corresponding instruction is output based on the triggering condition.

[0050] The motion output module is used to output the corresponding actions of the robot using the low-level controller according to the corresponding instructions.

[0051] Thirdly, this application proposes an electronic device, comprising: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to execute the aforementioned graph model-based whole-body control method for a legged manipulator.

[0052] Fourthly, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the aforementioned graph model-based whole-body control method for a legged robot.

[0053] Fifthly, this application proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned graph model-based whole-body control method for a legged manipulator.

[0054] Beneficial effects:

[0055] This application proposes a whole-body control method and system for a legged manipulator based on a graph model, with the following advantages:

[0056] (1) A unified graph structure modeling method is used to achieve intrinsic synergistic optimization of motion and operation by dynamically aggregating the heterogeneous features of the body motion, the robotic arm operation and the environmental perception nodes. This design can effectively suppress the body attitude instability caused by the load disturbance of the robotic arm and reduce the communication delay caused by the state synchronization between modules, thereby improving the robustness and real-time performance of the system in unstructured scenarios.

[0057] (2) By utilizing the neighborhood propagation mechanism of graph neural networks, local features such as the leg joint state and the end-effector operation pose are diffused along physical connections to global nodes, thereby achieving coordinated control of leg movement and robotic arm operation.

[0058] (3) A permutation-invariant graph neural network is introduced to solve the problem of traditional graph networks relying on the order of node input. By designing convolutional layers with shared weights and permutation-invariant global pooling operations, it is ensured that the model output features and control strategies remain consistent when the node order changes due to component replacement, system debugging, etc. At the same time, by combining a dual-space adjacency matrix and a multi-channel feature fusion mechanism, the complexity of training samples is reduced, eliminating the need for repeated learning for different node orders and improving the model training efficiency and generalization ability.

[0059] (4) The hierarchical strategy network decomposes complex tasks into specific sub-tasks, improving the system's ability to solve long-term decision-making tasks. The high-level planner generates target positions and operation instructions, and the low-level controller controls the motor to complete the specific tasks according to the instructions. Attached Figure Description

[0060] Figure 1 A flowchart of a whole-body control method for a legged manipulator robot based on a graph model according to an embodiment of the present invention;

[0061] Figure 2 A schematic flowchart of a whole-body control method for a legged manipulator based on a graph model according to an embodiment of the present invention;

[0062] Figure 3 A schematic diagram of the graph model structure modeling of an embodiment of the present invention;

[0063] Figure 4 The three-level progressive feature extraction architecture of graph convolutional layers in this embodiment of the invention;

[0064] Figure 5 A schematic diagram of the permutation-invariant global pooling operation in an embodiment of the present invention;

[0065] Figure 6 Flowchart of the strategy algorithm in this embodiment of the invention;

[0066] Figure 7 Simulation model of a legged manipulator robot according to an embodiment of the present invention;

[0067] Figure 8 A schematic diagram of the whole-body control system of a legged manipulator based on a graph model according to an embodiment of the present invention. Detailed Implementation

[0068] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0069] This application proposes a graph-based method and system for full-body control of legged manipulators. The aim is to develop a unified graph-based full-body control method to address the modular coupling problem between motion and manipulation in legged manipulators, while overcoming the limitations of traditional graph neural networks that rely on node order. Unlike traditional methods, this method constructs the quadrupedal mobile platform and robotic arm as a unified graph model. Graph nodes represent the robot's sensing units and actuators (e.g., leg joints, inertial measurement units, robotic arm motors); edges represent the physical structural connections of the robot, reflecting the dynamic coupling relationships between nodes (e.g., the rigid connection between the robotic arm and the body, the rotational joints between the legs and the body). This unified modeling enables the system to more effectively coordinate full-body movements, improving system stability and adaptability.

[0070] Specifically, this method achieves intrinsic co-optimization of motion and operation by dynamically aggregating local node information. Simultaneously, it utilizes a permutation-invariant graph neural network to share node encoding weights and message passing weights, ensuring that node order adjustments do not affect model output. Compared to existing research, this method effectively suppresses center-of-gravity shifts caused by robotic arm gripping disturbances, reduces decision-making delays resulting from the fusion of motion and operation modules, and improves the stability and real-time performance of the collaborative system. Furthermore, this method eliminates node order dependencies, reduces the sample complexity of graph network training, and enhances system stability. The technical contributions of this method can be summarized as follows:

[0071] (1) A whole-body modeling paradigm for legged robots was proposed, achieving for the first time a unified modeling of quadrupedal movement and robotic arm operation;

[0072] (2) To address the problem of a surge in parameters and high model deployment costs caused by the explosion of node combinations in multi-node systems, a permutation-invariant graph network design is introduced to reduce model complexity and lay the foundation for the migration and application of graph neural networks to more complex multi-joint systems such as humanoid robots.

[0073] (3) Develop a whole-body control strategy based on the model to generate stable foot movement trajectories and robotic arm operation trajectories;

[0074] (4) Verify the generalization ability of the architecture in a simulation environment and compare it with existing baseline methods to evaluate the performance of the algorithm.

[0075] The objective of this application is to develop a full-body control framework for legged manipulators in complex environments, achieving multimodal perception data fusion through a permutation-invariant graph neural network and hierarchical reinforcement learning. The inputs to this control framework include environmental point clouds acquired by a depth camera, acceleration and angular velocity measured by an inertial measurement unit (IMU), and joint pose information fed back from motors. After feature extraction and preprocessing, the raw perception data is mapped and concatenated into feature vectors for each node in the graph, which are then input into the graph neural network. The permutation-invariant graph neural network iteratively aggregates the features of the node itself and its neighborhood, learning the fused high-level features of the nodes. These features are then passed as global state information to the policy network.

[0076] The policy network employs a hierarchical reinforcement learning architecture. The high-level planner is responsible for decomposing the robot's overall task into a sequence of sub-tasks. Its input is the encoded fused features, and its output is the motion target point and grasping signal. The low-level controller focuses on executing the motion control of the quadrupedal base and the operation of the end effector. This module uses trainable policy learning algorithms, such as PPO (Proximal Policy Optimization) and SAC (Soft Actor-Critic), to control the joint actuators to complete basic movements. Its input is the motion target point and grasping signal from the high-level planner, and its output is the joint angle.

[0077] In summary, this application proposes a graph model-based method and system for the whole-body control of a legged robot, which has significant application potential and research value in the field of autonomous operation in unstructured environments. The following sections will focus on elaborating the technical solutions and implementation details of this framework, specifically the graph model construction, graph encoder design, and policy network design.

[0078] Example 1:

[0079] This embodiment provides a graph model-based method for the whole-body control of a legged robot, such as... Figure 1 , Figure 2 As shown, it includes:

[0080] Step S1: Establish a node set based on the robot's sensing units and actuators, establish an edge set based on the physical structure connection relationship of the robot, obtain the feature vector of the node by tensor concatenation of multi-source sensing data, form the feature matrix of the node by the feature vector of the node, and construct a graph model with the node set, edge set and feature matrix of the node.

[0081] (1) The establishment of a node set using the robot's sensing unit and actuator includes:

[0082] Step S1.1.1: Use the robot's inertial measurement unit as the body node, the robot's leg joints as the motion node, and the robot's robotic arm joints as the operation node.

[0083] Step S1.1.2: Create a node set for the body node, motion node, and operation node.

[0084] simulation model of legged robot such as Figure 7 As shown, in order to achieve full-body control of the legged robot, this embodiment adopts a graphical model structure. Modeling the physical system of the robot, such as Figure 3 As shown. Among them, For a set of nodes, For edge set, is the feature matrix of the node.

[0085] Among them, node set It contains three types of entities: a) Ontology nodes Characterizing the fuselage system, integrating IMU sensor features; b) motion nodes Characterizing leg joints; c) Operation nodes Characterizes the joints of the robotic arm.

[0086] (2) The establishment of edge sets based on the physical structural connection relationships of the robot includes:

[0087] Step S1.2.1: Based on the physical connection structure between any two nodes in the node set, construct a topological adjacency matrix, including:

[0088] Based on the physical connection structure between any two nodes in the node set, a basic topological adjacency matrix is ​​constructed, and the calculation formula is as follows:

[0089] ;

[0090] Where A is the basic topological adjacency matrix. Given any element in the topological adjacency matrix based on n, where n is the number of nodes;

[0091] Based on the basic topological adjacency matrix, an identity matrix and weight coefficients are introduced to construct a topological adjacency matrix with added self-loops, calculated as follows:

[0092] ;

[0093] in, It is a topological adjacency matrix. These are the weighting coefficients of the identity matrix. It is an identity matrix.

[0094] Step S1.2.2: Calculate the feature cosine similarity between any two nodes in the node set;

[0095] In this embodiment, the edge set E reflects the connection relationships between nodes and is represented by an adjacency matrix. To characterize the structural correlation and feature similarity of the robot system, a dual-graph structure of "topological space graph - feature space graph" is constructed.

[0096] Topological adjacency matrix Characterizing only the physical connections between nodes is insufficient to capture relationships between nodes exhibiting "similar features." However, in whole-body robot control, nodes with similar features are often interconnected, affecting the overall coordination of the robot system. For example, although the left and right leg joints of a quadruped robot are not directly physically connected, their angle and angular velocity characteristics are highly similar when performing symmetrical gait. Therefore, a feature adjacency matrix is ​​introduced. This allows GNNs to capture collaborative information about the robot, enabling the entire robot system to have a certain degree of coordination.

[0097] Based on node feature similarity Nearest Neighbor Graph (KNN). Calculate the cosine similarity of node features to construct a similarity matrix. :

[0098] ;

[0099] in, Represents a node eigenvectors, Represents a node eigenvectors, Similarity matrix The elements in the array, where n is the number of nodes.

[0100] Step S1.2.3: Select the k nodes with the highest feature cosine similarity as neighbor nodes, set the neighbor nodes to 1 and the rest of the nodes to 0, and obtain the sparse feature adjacency matrix;

[0101] In this embodiment, the feature similarity is selected. Each node is set as a neighbor, and then... The rest are 0, forming a sparse characteristic adjacency matrix.

[0102] Step S1.2.4: Perform dual-graph fusion on the topological adjacency matrix and the sparse feature adjacency matrix to obtain the final adjacency matrix, and use the final adjacency matrix as the edge set.

[0103] In this embodiment, learnable weights are used. The adjacency matrices of the two graphs are adaptively fused to obtain the final adjacency matrix, calculated as follows:

[0104]

[0105] in, This is the final adjacency matrix. Let be the degree matrix of the topological graph. Let be the degree matrix of the feature map. It is a topological adjacency matrix. For the characteristic adjacency matrix, The weights for the fusion of two graphs. Through training, the contributions of structural information and feature similarity are balanced.

[0106] (3) The feature matrix of the node is composed of the feature vector of each node, and the feature vector is calculated as follows:

[0107] ;

[0108] in, Let v be the feature vector of the v-th node. The global point cloud features extracted by the point cloud network are denoted by [symbol]. This indicates a feature concatenation operation. The acceleration measured by the inertial measurement unit. Angular velocity measured by the inertial measurement unit. For the fuselage attitude quaternion, Let be the single joint angle of the v-th node. Let v be the angular velocity of the v-th node. The torque at the v-th node, For the ontology node, For the j-th moving node, This is the kth operation node.

[0109] In this embodiment, the feature matrix Each node is composed of a feature vector, and each node can acquire different perceptual data. The body node can acquire four types of data: acceleration, angular velocity, fuselage attitude quaternions, and point cloud features. Motion nodes and manipulation nodes can acquire three types of data: joint angles, angular velocities, and torque. For nodes... The features it can acquire are the features of the node itself. It can be expressed as the above calculation formula.

[0110] When performing graph convolution operations, each node not only has its own features, but also aggregates the features of other neighboring nodes, which changes the feature matrix H.

[0111] Step S2: Input the graph model into a permutation-invariant graph encoder for encoding to obtain the fused features;

[0112] In this embodiment, permutation-invariant graph neural networks (PNNs) are a type of neural network whose output does not change with the order of input elements. In processing graph data, especially graphs with a large number of nodes, they can simplify the number of network parameters and improve model performance. These networks, through specific structural design and feature aggregation operations, strip away the sequential information of the input data, extracting only content-related global features. For example, input elements are processed independently using modules with the same structure, and then the features are integrated using aggregation functions with permutation invariance (such as max pooling). Regardless of how the order of the input elements is adjusted, the network output remains unchanged, i.e., it remains constant for any permutation matrix. The graphical model output satisfies permutation invariance: .in, It is an adjacency matrix. Let P be the characteristic matrix. The permutation matrix P is used to exchange the order of elements. In each row and column of the matrix, there is exactly one element that is 1 and the rest are 0.

[0113] This invention employs a permutation-invariant graph neural network as the core encoder architecture, and through weight sharing and global pooling, makes the global features permutation-invariant.

[0114] The process of inputting the graph model into a permutation-invariant graph encoder to obtain fused features includes:

[0115] Step S2.1: Expand the feature vector of each node to construct an initial feature matrix, and use the initial feature matrix as the input of the graph encoder;

[0116] In this embodiment, the input layer of the graph neural network consists of the features of the nodes themselves. The initial feature matrix is ​​constructed by expanding the feature vectors of the 19 nodes. :

[0117] ;

[0118] in, The feature vector of the ontology node. This is the feature vector of the first moving node. The feature vector of the sixth operation node, the ontology node. The features include IMU acceleration (3D) and angular velocity (3D), body linear velocity (3D) and attitude quaternions (4D), and visual point cloud features (64D), totaling 77 dimensions. The features for the 12 leg joints and the 6-DOF arm joints include joint angles (1D), angular velocity (1D), and torque (1D), totaling 3 dimensions, with remaining positions padded with zeros. Therefore, the dimension of the feature matrix for the 19 nodes is... .

[0119] Step S2.2: Based on the shared weights of each layer, a three-level progressive feature extraction method is used to obtain the feature matrix of each layer of the graph network;

[0120] In this embodiment, the graph convolutional network adopts a three-level progressive feature extraction architecture, such as... Figure 4 As shown, global topology information is gradually fused by expanding the receptive field. layer The feature update formula is:

[0121] ;

[0122] in, For the first Feature matrix of layer nodes For the first Layer feature dimension For the shared weights of layer l, For the first Layer feature dimension The bias is set to the l-th layer. In this embodiment, each layer shares the same shared weight and corresponding bias, and both the shared weight and bias are learnable parameters that can be obtained through neural network training. This is the final adjacency matrix. for The activation function, where n is the number of nodes.

[0123] Step S2.3: Based on the feature matrix of the third-layer graph network, perform permutation-invariant global pooling to obtain the fused features;

[0124] In this embodiment, the max pooling operation, such as Figure 5 As shown, this is the core element that makes GNN permutation invariant; this operation is used to transform node-level features... Transform into global feature vector Global feature vectors It is a unique feature of the entire graph, independent of the number of nodes, and is derived by taking the maximum value of each column of the feature matrix.

[0125] ;

[0126] in, This is the feature matrix of the third layer graph network. yes The List, Let be the infinite norm of the vector. This is a feature of fusion.

[0127] Step S2.4: Concatenate the fused features at different time points to form a reinforcement learning state space, which is then used as the output of the graph encoder.

[0128] ;

[0129] in, To enhance the learning state space, The fusion features at time t are... The fusion features at time t-1, The fusion features are at time t-2.

[0130] In this embodiment, the entire graph model fully captures the dynamic coupling relationship and spatial correlation of the robot system, laying an information foundation for subsequent policy network control.

[0131] Step S3: Use the high-level planner in the policy network to decompose the task to be controlled into subtasks. In each subtask, determine the triggering condition based on the fusion features, and output the corresponding instruction based on the triggering condition.

[0132] Step S4: According to the corresponding instructions, the lower-level controller outputs the corresponding actions of the robot.

[0133] In this embodiment, the policy network adopts a two-layer reinforcement learning architecture, completing long-term tasks through task decomposition and progressive training. The high-level planner uses a teacher-student framework, processing user-inputted instructions for the complex tasks to be controlled. Decompose into subtask operation sequences Outputs low-level execution instructions, including target location. and capture signal The teacher-student framework is existing technology. Through teacher learning and student imitation learning, knowledge distillation is performed to simplify model parameters, which will not be elaborated upon in this embodiment. Subtask decomposition is also existing technology and will not be elaborated upon in this embodiment. The low-level controller uses the PPO algorithm to output joint angles based on subtask instructions. The mapping relationship between high and low levels is implemented through a multilayer perceptron (MLP).

[0134] ;

[0135] in, ( ) represents a multilayer perceptron. To reinforce the learning state space, z represents the sequence of subtask operations. For the target location, To capture signals.

[0136] In this embodiment, the state space is learned through reinforcement learning. Calculations yielded , The distance from the object to the robot is indicated by the prior art, and will not be described in detail in this embodiment.

[0137] The robot's perceived information is used to extract features and output the state. The strategy network consists of a high-level planner (high-level) and a low-level controller (low-level). The high-level planner determines the sub-tasks that need to be completed based on the robot's current state, and the low-level controller then executes these sub-tasks. Sub-tasks include searching, approaching, grasping, transferring, and releasing. By continuously determining and executing different sub-tasks, the complex task given by the user is ultimately completed. Figure 6 This demonstrates the process of high-level and low-level systems cooperating to complete a specific task. When the system does not detect the target object, the high-level system determines that it should enter the search phase, outputting the free space point coordinates in real time to guide the low-level control robot to explore and move. If the target enters a 3-meter radius, the high-level system determines that it should switch to the approach phase, updating the target point to the object's centroid. When the distance between the robot and the target decreases to 0.1 meters, the high-level system determines that it should enter the grasping phase and send a grasping command, with the low-level system executing the grasping operation. If the grasping is successful, it enters the transfer phase, maintaining the grasping state and sending the object to the target area. When the object's position coincides with the target area, a release command is sent to complete the operation. If the target leaves the detection range during the process, the system automatically reverts to the search phase for repositioning. If the grasping action fails or the object falls off during the transfer, it resets to the approach phase to correct the pose. Each time an operation times out, an emergency stop state is triggered, forcibly scanning the environment and restarting the subtask process to ensure that the system can still complete the task through state rollback under dynamic interference. As shown in Table 1. Represents a random target point. Represents the object's coordinates. This indicates the distance from the block to the robot.

[0138] Table 1 Task Breakdown Table:

[0139] ;

[0140] This embodiment proposes a graph model-based method for the whole-body control of a legged manipulator, which solves the problems of insufficient motion and manipulation coordination and unstable whole-body coordination performance through the following key technologies:

[0141] (1) Permutation-invariant graph structure modeling: Based on the physical connection characteristics of the robot, a unified graph model is constructed, and the quadruped body motion system, the robotic arm operating system and the sensing unit are mapped as graph nodes. The coupling relationship between the mechanisms is described by the adjacency matrix. Combined with the shared weight and global pooling mechanism, the whole body is modeled in a unified manner.

[0142] (2) Multimodal feature fusion: The design graph encoder aggregates visual, IMU and motor torque information, and gradually propagates the neighborhood features to the global through a three-layer network, and finally generates permutation-invariant global features through global pooling;

[0143] (3) Hierarchical decision architecture: Construct a two-layer policy network of high-level planning and low-level execution. The high-level planner parses the semantics of the task and decomposes it into specific sub-tasks. The low-level controller generates joint control signals through reinforcement learning algorithms.

[0144] Example 2:

[0145] This embodiment also provides a whole-body control system for a legged manipulator based on a graph model, such as... Figure 8 As shown, it includes:

[0146] The system comprises a graph model construction module, a graph encoding module, an instruction output module, and an action output module, which are connected sequentially.

[0147] The graph model construction module is used to establish a node set based on the robot's sensing units and actuators, establish an edge set based on the physical structure connection relationship of the robot, obtain the feature vector of the node by concatenating tensors of multi-source sensing data, form the feature matrix of the node by combining the feature vector of the node, and construct the graph model with the node set, edge set and feature matrix of the node.

[0148] The graph encoding module is used to input the graph model into the permutation-invariant graph encoder for encoding to obtain fused features;

[0149] The instruction output module is used to decompose the task to be controlled into subtasks using the high-level planner in the policy network. In each subtask, the triggering condition is determined based on the fusion characteristics, and the corresponding instruction is output based on the triggering condition.

[0150] The motion output module is used to output the corresponding actions of the robot using the low-level controller according to the corresponding instructions.

[0151] Example 3:

[0152] This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the aforementioned graph model-based whole-body control method for a legged manipulator.

[0153] The electronic device can be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements a graph model-based full-body control method for a legged manipulator robot as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0154] The processor is used to execute all or part of the steps in the graph model-based whole-body control method for a legged manipulator robot as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in an electronic device, as well as application-related data.

[0155] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the graph model-based whole-body control method for a legged robot described in the above embodiments.

[0156] Example 4:

[0157] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0158] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the graph model-based whole-body control method for a legged robot described in various embodiments of this application.

[0159] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, etc.), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the aforementioned graph model-based legged robot whole-body control method.

[0160] Example 5:

[0161] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned graph model-based method for controlling the whole body of a legged robot.

[0162] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0163] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0164] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of equivalent technology of this disclosure, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A method for whole-body control of a legged manipulator based on a graph model, characterized in that, include: A node set is established using the robot's sensing units and actuators, an edge set is established using the physical structural connections of the robot, and the feature vectors of the nodes are obtained by concatenating tensors of multi-source sensing data. The feature vectors of the nodes are then combined to form the feature matrix of the nodes, and a graph model is constructed using the node set, edge set, and feature matrix of the nodes. The graph model is input into a permutation-invariant graph encoder for encoding to obtain fused features; The high-level planner in the policy network is used to decompose the task to be controlled into subtasks. In each subtask, the triggering condition is determined based on the fusion characteristics, and the corresponding instruction is output according to the triggering condition. Based on the corresponding instructions, the lower-level controller outputs the corresponding actions of the robot. The process of inputting the graph model into a permutation-invariant graph encoder to obtain fused features includes: Expand the feature vector of each node to construct an initial feature matrix, and use the initial feature matrix as the input of the graph encoder; Based on the shared weights of each layer, a three-level progressive feature extraction method is used to obtain the feature matrix of each layer of the graph network. Based on the feature matrix of the third-layer graph network, permutation-invariant global pooling is performed to obtain the fused features; The fused features at different times are spliced ​​together to form a reinforcement learning state space, which is then used as the output of the graph encoder.

2. The whole-body control method for a legged manipulator based on a graph model according to claim 1, characterized in that, The establishment of a node set based on the robot's sensing units and actuators includes: The robot's inertial measurement unit is used as the body node, the robot's leg joints are used as motion nodes, and the robot's robotic arm joints are used as operation nodes. Create a node set for the body node, motion node, and operation node.

3. The whole-body control method for a legged manipulator based on a graph model according to claim 1, characterized in that, The establishment of edge sets based on the physical structural connections of the robot includes: Construct a topological adjacency matrix based on the physical connection structure between any two nodes in the node set; Calculate the feature cosine similarity between any two nodes in the node set. Select the k nodes with the highest feature cosine similarity as neighbor nodes, set the neighbor nodes to 1 and the rest nodes to 0, and obtain the sparse feature adjacency matrix. The topological adjacency matrix and the sparse feature adjacency matrix are fused to obtain the final adjacency matrix, which is then used as the edge set.

4. The whole-body control method for a legged manipulator based on a graph model according to claim 3, characterized in that, The construction of a topological adjacency matrix based on the physical connection structure between any two nodes in the node set includes: Based on the physical connection structure between any two nodes in the node set, a basic topological adjacency matrix is ​​constructed, and the calculation formula is as follows: ； Where A is the basic topological adjacency matrix. Given any element in the topological adjacency matrix based on n, where n is the number of nodes; Based on the basic topological adjacency matrix, an identity matrix and weight coefficients are introduced to construct a topological adjacency matrix with added self-loops, calculated as follows: ； in, It is a topological adjacency matrix. These are the weighting coefficients of the identity matrix. It is an identity matrix.

5. The whole-body control method for a legged manipulator based on a graph model according to claim 1, characterized in that, The feature matrix of a node is composed of the feature vector of each node, and the feature vector is calculated as follows: ； in, Let v be the feature vector of the v-th node. The global point cloud features extracted by the point cloud network are denoted by [symbol]. This indicates a feature concatenation operation. The acceleration measured by the inertial measurement unit. Angular velocity measured by the inertial measurement unit. For the fuselage attitude quaternion, Let be the single joint angle of the v-th node. Let v be the angular velocity of the v-th node. The torque at the v-th node, For the ontology node, For the j-th moving node, This is the kth operation node.

6. The whole-body control method for a legged manipulator based on a graph model according to claim 3, characterized in that, The topological adjacency matrix and the sparse feature adjacency matrix are fused to obtain the final adjacency matrix, calculated as follows: ； in, This is the final adjacency matrix. Let be the degree matrix of the topological graph. Let be the degree matrix of the feature map. It is a topological adjacency matrix. For the characteristic adjacency matrix, The weights for merging the two graphs.

7. The whole-body control method for a legged manipulator based on a graph model according to claim 1, characterized in that, Based on the shared weights of each layer, a three-level progressive feature extraction method is used to obtain the feature matrix of each layer of the graph network. The calculation formula is as follows: ； in, For the first Feature matrix of layer nodes For the first Layer feature dimension For the shared weights of layer l, For the first Layer feature dimension For the bias of the l-th layer, This is the final adjacency matrix. for The activation function, where n is the number of nodes.

8. The whole-body control method for a legged manipulator based on a graph model according to claim 1, characterized in that, The fused features are obtained by performing permutation-invariant global pooling based on the feature matrix of the third-layer graph network, as calculated by the following formula: ； in, This is the feature matrix of the third layer graph network. yes The List, Let be the infinite norm of the vector. This is a feature of fusion.

9. A graph-based whole-body control system for a legged manipulator is provided, implemented using the graph-based whole-body control method for a legged manipulator as described in any one of claims 1 to 8, characterized in that... include: The graph model construction module is used to establish a node set based on the robot's sensing units and actuators, establish an edge set based on the physical structure connection relationship of the robot, obtain the feature vector of the node by concatenating tensors of multi-source sensing data, form the feature matrix of the node by combining the feature vector of the node, and construct the graph model with the node set, edge set and feature matrix of the node. The graph encoding module is used to input the graph model into the permutation-invariant graph encoder for encoding to obtain fused features; The instruction output module is used to decompose the task to be controlled into subtasks using the high-level planner in the policy network. In each subtask, the triggering condition is determined based on the fusion characteristics, and the corresponding instruction is output based on the triggering condition. The motion output module is used to output the corresponding actions of the robot using the low-level controller according to the corresponding instructions. The process of inputting the graph model into a permutation-invariant graph encoder to obtain fused features includes: Expand the feature vector of each node to construct an initial feature matrix, and use the initial feature matrix as the input of the graph encoder; Based on the shared weights of each layer, a three-level progressive feature extraction method is used to obtain the feature matrix of each layer of the graph network. Based on the feature matrix of the third-layer graph network, permutation-invariant global pooling is performed to obtain the fused features; The fused features at different times are spliced ​​together to form a reinforcement learning state space, which is then used as the output of the graph encoder.

Images