Inverse kinematics solving method for redundant space robot based on graph neural network

Abstract

The invention discloses a redundant space robot inverse kinematics solving method based on a graph neural network, and relates to the technical field of robot motion control and artificial intelligence crossing. A distance geometric graph is constructed to represent the configuration of the mechanical arm, an inverse kinematics problem is converted into a completion problem of a partial graph, joint configuration is sampled, and data pairs of a complete graph and the partial graph are stored as a data set; training based on an isotropic graph neural network and a conditional variation auto-encoder, and training a model based on a loss function of an evidence lower bound; and in the reasoning process, constructing a partial graph, inputting the partial graph into the trained prior network, outputting hidden variables meeting Gaussian mixture distribution, extracting sampling points, and generating a reconstructed complete graph to obtain joint angle information. The mechanical arm structure is represented through the distance geometric diagram, the inverse kinematics problem is converted into the complementation problem of partial diagrams, probability distribution of a solution space is learned through a conditional variation auto-encoder frame, multi-solution generation of different mechanical arm configurations is supported, and the solving precision and efficiency are high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot motion control and artificial intelligence, and particularly relates to a redundant space robot inverse kinematics solving method based on a graph neural network. BACKGROUND

[0002] Currently, there are two methods for solving the inverse kinematics of a space robot, namely a traditional solving method and a learning-based method.

[0003] The traditional solving method includes a numerical optimization method (such as TRAC-IK), an analytical method and a geometric heuristic method. Among them: the numerical optimization method relies on local search and is prone to local optimization, and can only return a single solution each time, and needs to be initialized multiple times to improve the success rate; the analytical method is only suitable for mechanical arms with less than 6 degrees of freedom and cannot handle redundant degrees of freedom; the geometric heuristic method (such as FABRIK) has fast calculation speed but cannot guarantee accuracy and cannot model the solution space distribution.

[0004] The existing learning-based methods, such as IKFlow and IKNet, need to train a model for each type of robot, lack generalization ability, and most of the methods only output a single solution, which cannot cover the multi-solution space. In addition, the current graph representation method does not use the Euclidean equivariance (E(n)) of the robot structure, which limits the solution accuracy. SUMMARY

[0005] To solve the problems in the background art, the present application provides a redundant space robot inverse kinematics solving method based on a graph neural network, which converts the inverse kinematics problem into a partial graph completion problem by representing the mechanical arm structure with a distance geometry graph, learns the probability distribution of the solution space using a conditional variational autoencoder framework, supports multi-solution generation for different mechanical arm configurations, and has high solution accuracy and efficiency.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solution: a redundant space robot inverse kinematics solving method based on a graph neural network, comprising the following steps:

[0007] Step 1, generate a training data set:

[0008] Construct a distance geometry graph to represent the mechanical arm configuration of the redundant space robot, convert the inverse kinematics problem into a partial graph completion problem, for each mechanical arm configuration, import the urdf model to construct the distance geometry graph, and uniformly sample the joint configuration in the joint space , for each degree of freedom, each set of sampled joint configurations corresponds to a complete graph and a partial graph Each ( ) for a data and storing the data set;

[0009] Step 2, model training:

[0010] Based on the equivariant graph neural network EGNN and the conditional variational autoencoder, the training network includes an encoder GNNenc, a prior network GNNprior and a decoder GNNdec, wherein: the encoder GNNenc converts the complete graph Encoding into a latent variable Distribution, represented as ; the prior network GNNprior generates a prior distribution from the partial graph , represented as ; the decoder GNNdec obtains the reconstructed complete graph Based on the latent variable And the partial graph , represented as , the loss function based on the evidence lower bound ELBO is used to train the model;

[0011] Step 3, model inference:

[0012] In the inference process, the input is the end pose of the target And the urdf model of the robot arm, the partial graph Is constructed, which contains the distance constraints of the known base node And the joint node Of the end, the partial graph Is input into the trained prior network GNNprior, and the latent variable Satisfying the Gaussian mixture distribution is output, from which Group of sampling points , Each group of sampling points Corresponds to a potential solution, and the reconstructed complete graph Is generated to obtain the joint angle information.

[0013] Further, when constructing the distance geometry graph in step 1, the positions of the base and joints of each robot arm configuration are set as nodes, the base is a fixed point , represented as Each joint Take two points on the rotation axis , represented as , Located at the starting end of the joint axis, Located at the end of the joint axis, and satisfy the distance ; on this basis, the fixed distance between the joint nodes Determined by the robot arm configuration is defined as the structure edge​ , defined by the end pose Determine the end joint node To base node The distance to the end edge , define the joint nodes except the end To base node The distance to the joint edge ; Define the structure diagram for the robotic arm configuration , terminal constraint subgraph Reconciliation Diagram ,in:

[0014] Structure diagram The nodes are all joint nodes , structural edge For adjacent joint nodes A fixed distance between

[0015] Terminal constraint subgraph The nodes include the base node and the end joints , through the end edge Constrain its position, end constraint subgraph The end position Encoded as distance constraint, structure diagram and terminal constraint subgraph Joint component diagram ;

[0016] Diagram The node is the base node and all joint nodes , joint edge Supplementary Figures The missing distance in the structure diagram , terminal constraint subgraph Reconciliation Diagram Combined to form a complete graph Uniquely determine a joint configuration .

[0017] Furthermore, the joint configuration is sampled in step 1 During the process, the end pose is calculated by forward kinematics , ensuring that each end pose At least one feasible solution is corresponding, and grid space hashing is applied to merge similar poses to avoid data redundancy. The similar poses refer to two adjacent joint configurations. satisfy ,in, and Represents the configuration of two adjacent joints.

[0018] Furthermore, the training network in step 2, the encoder GNNenc input is the complete graph , containing node coordinates , category labels and including structural edges , end edge and joint edges All edge distances are passed to the EGNN layer for message passing and joint node updates, and the output is the hidden variable The distribution parameters of and the variance is ; Prior network GNNprior input is part of the graph , through the multi-head EGNN network, the separated MLP head outputs the mixed Gaussian parameters; the decoder GNNdec input is the hidden variable and part of the graph The splicing uses the same EGNN network as the encoder, and the output is the Gaussian distribution parameters of the joint node positions.

[0019] Furthermore, the loss function in step 2 is expressed as:

[0020]

[0021] Where, Represents the latent variable In distribution Under expectations, To adjust the coefficients, the training objectives include minimizing the pose error of the solution while reducing the encoder output and the prior network Divergence, constrained latent variables The distribution is close to the prior.

[0022] Furthermore, the conditional variational autoencoder is distributed from latent variables mid-sampling To calculate the gradient of ELBO, design reparameterization techniques and introduce auxiliary noise variables , rewrite the sampling as .

[0023] Furthermore, the sampling points are extracted in step 3 When setting the reference latent variable , based on the reference latent variable during the sampling process Perform nearest neighbor selection and calculate reference latent variables The Euclidean distance to the mean of each component is used to select the component with the closest distance and sample only from that component.

[0024] Furthermore, in step 3, the reconstructed complete graph Parallel screening is performed, or millisecond-level fine-tuning is performed as an initial value feeding into the TRAC-IK numerical optimizer.

[0025] Compared with the prior art, the beneficial effects of the present application are: the method of the present application is suitable for scenarios such as redundant space robot motion control, the structure of a redundant space robot is represented by a distance geometry graph, and the inverse kinematics problem is converted into a partial graph completion problem, the probability distribution of the solution space is learned using a conditional variational autoencoder framework, and diversified solutions are supported in parallel generation, compared with traditional numerical optimization methods: first, it has universality, a single model can adapt to mechanical arms of different degrees of freedom and geometric structures; second, it has high efficiency, GPU parallelization can realize millisecond-level multi-solution generation; in addition, it has scalability, and supports learning of joint constraints and other constraint conditions. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a training network structure diagram in the method of the present application;

[0027] Figure 2 is a reasoning network structure diagram in the method of the present application;

[0028] Figure 3 is a distance geometry graph of an SSRMS configuration mechanical arm in the embodiment. DETAILED DESCRIPTION

[0029] The technical solutions in the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0030] As shown in Figures 1-2 , a redundant space robot inverse kinematics solving method based on a graph neural network is a generative graph inverse kinematics framework, which combines a distance geometry graph representation and a conditional variational autoencoder to form an equivariant graph neural network, and converts the solving problem of redundant space robot inverse kinematics into a graph structure completion problem, that is, unknown distances are predicted from known distances to restore the complete configuration, and specifically includes the following steps:

[0031] Step 1: Generate a training data set

[0032] 1.1, Construct a distance geometry graph

[0033] The distance geometry graph structure includes nodes , edges and a weight function , wherein the weight function Defined by the node The node coordinates of the index The distance between them.

[0034] For each configuration of the redundant space robot's manipulator, the position of its base and each joint is set as a node. For the base, it is a fixed point , the base node is represented as ; For each joint , take two points on the rotation axis , Located at the starting end of the joint axis, Located at the end of the joint axis and meeting the distance , the joint nodes are represented as On this basis, define the joint nodes determined by the robot arm configuration The fixed distance between the structural edges , defined by the end pose Determine the end joint node To base node The distance to the end edge , define the joint nodes except the end To base node The distance to the joint edge , joint edge Corresponding to the joint configuration to be solved .

[0035] Define the structure diagram for a certain robotic arm configuration , terminal constraint subgraph Reconciliation Diagram ,in:

[0036] Structure diagram The nodes are all joint nodes , structural edge For adjacent joint nodes A fixed distance between is the connecting rod length), structural side Reflects the rigid structure of the manipulator, and its weight function does not change with the joint configuration change.

[0037] Terminal constraint subgraph The nodes include the base node and the end joints , through the end edge Constrain its position, end constraint subgraph The end position Encoded as distance constraint, structure diagram and terminal constraint subgraph Jointed component graph .

[0038] Solved graph The nodes of the base node and all joint nodes Joint edges Supplement component graph The missing distance (such as the distance between non-adjacent joint nodes) in the component graph is determined by the joint configuration .

[0039] Finally, the structure graph , the end constraint subgraph and the solved graph Jointly constitute a complete graph Uniquely determine a joint configuration .

[0040] 1.2, Joint configuration sampling and storing data sets

[0041] For each robot arm configuration, import the urdf model to build the distance geometry graph. Uniformly sample joint configurations in the joint space ( for degrees of freedom), ensuring that the joint configuration satisfies the joint constraints. Then, calculate the end pose by forward kinematics, ensuring that each end pose corresponds to at least one feasible solution. Apply grid space hashing to merge similar poses to avoid data redundancy, where similar poses refer to two adjacent joint configurations satisfy , where and represent two adjacent joint configurations. Each group of sampled joint configurations corresponds to a complete graph and a component graph , and each pair of is stored as a data.

[0042] Step 2: Model training

[0043] 2.1, Grid architecture

[0044] To maintain geometric consistency during robot motion, an equal-variant graph neural network (EGNN) is used, based on the principle of conditional variational autoencoder, combined with Figure 1 As shown, the training network consists of an encoder GNNenc, a prior network GNNprior and a decoder GNNdec.

[0045] The encoder GNNenc encodes the complete graph into hidden variables distribution, capturing the multi-solution property, is expressed as The input is a complete graph (including node coordinates , category labels and including structural edges , end edge and joint edges All edge distances), message passing and joint node updates are performed in the EGNN layer, and the output is the hidden variable The distribution parameters of and the variance is .

[0046] Prior network GNNprior from partial graph Generate a prior distribution, expressed as The input is a partial graph , through the multi-head EGNN network, the separated MLP head outputs the mixed Gaussian parameters. Since a single Gaussian distribution is difficult to capture the multi-peak solution space, a Gaussian mixture distribution is used to explicitly model the multi-solution property and generate a variety of latent variables. .

[0047] The decoder GNNdec is based on latent variables and part of the graph Get the reconstructed complete graph , expressed as . Input is a hidden variable and part of the graph The splicing uses the same EGNN network as the encoder, and the output is the Gaussian distribution parameters of the joint node positions.

[0048] 2.2 Training Architecture

[0049] The loss function is the Evidence Lower Bound (ELBO) objective, expressed as:

[0050]

[0051] Where, Represents the latent variable In distribution Under expectations, To adjust the coefficients, the training objectives include minimizing the pose error of the solution while reducing the encoder output and the prior network Divergence, constrained latent variables The distribution is close to the prior.

[0052] Since in the conditional variational autoencoder, it is necessary to distribute the latent variables mid-sampling To compute the gradient of ELBO, but because the sampling operation is not differentiable, direct sampling will cause the gradient to be unable to backpropagate, so a reparameterization trick is designed to introduce auxiliary noise variables , the sampling is rewritten as .

[0053] Step 3: Model inference

[0054] As shown in Figure 2 , the inference network takes the end pose of the target and the urdf model of the robot arm as input during inference.

[0055] 3.1, input partial graph and encode latent variable distribution

[0056] Through the input end pose of the target , first construct a partial graph by step 1.1, which contains the known base node and the distance constraints of the joint nodes at the end.

[0057] Input the partial graph into the trained prior network GNNprior, and output the latent variable satisfying the Gaussian mixture distribution.

[0058] 3.2, sampling from latent variable distribution

[0059] Draw groups of sampling points from the latent variable , Each group of sampling points corresponds to a potential solution. When sampling, a reference latent variable (such as the latent variable distribution sampled when solving the previous path point when performing continuous trajectory tracking) can be set, and during the sampling process, the nearest neighbor selection is based on the reference latent variable , the Euclidean distance between the reference latent variable and the mean of each component is calculated, the nearest component is selected, and only samples from this component are selected. The generated samples will be concentrated in the distribution area closest to the reference latent variable , rather than randomly selecting components, thereby achieving controllable sampling based on the reference latent variable .

[0060] 3.3, decode to generate a reconstructed complete graph

[0061] Input the latent variable and the node coordinates of the partial graph (the unknown node positions are initialized to 0) and the class label (To distinguish base nodes and joint nodes), update node coordinates by multi-layer EGNN message passing and weight functions . Output Gaussian likelihood , from the reconstructed complete graph The joint angle information can be obtained from the reconstructed complete graph

[0062] 3.4, precision improvement means

[0063] Due to the continuous sampling characteristics of the latent variable , the generated solution (reconstructed complete graph ) can cover different regions of the joint space, forming a continuous solution manifold for redundant manipulators. To improve practicality, the reconstructed complete graph can be screened in parallel, or fed as an initial value into a numerical optimizer such as TRAC-IK for millisecond-level fine-tuning.

[0064] Embodiments

[0065] Step 1: Generate training data set

[0066] This embodiment selects SSRMS configuration manipulator as the inverse kinematics solving object, and creates a distance geometry graph structure as shown in Figure 3 Within the range of joint limits ( ), the joint angle is sampled, and 512000 pairs of ) are collected.

[0067] Step 2: Model training

[0068] A training network composed of an encoder GNNenc, a prior network GNNprior, and a decoder GNNdec is built.

[0069] First, the encoder GNNenc inputs the complete graph (containing node coordinates , class labels , and all edge distances containing structure edges, end edges and joint edges), message passing and node updating are performed in the EGNN layer, and the complete graph is encoded into a latent variable distribution, and the distribution parameters (mean and variance ) of the latent variable are output.

[0070] Subsequently, the partial graph , through the multi-head EGNN network, the mixed Gaussian parameters are output in the separated MLP head.

[0071] Finally, the decoder GNNdec inputs the hidden variables and part of the graph splicing, using the message passing mechanism to update the node coordinates, from the hidden variables and part of the graph Get the reconstructed complete graph , predict the node position distribution parameters through multi-layer perceptron.

[0072] During training, the reconstruction loss (mean squared error between the predicted and true node positions) and the KL divergence (aligning the encoder with the prior distribution) are jointly optimized.

[0073] Step 3: Model Inference

[0074] During reasoning, the end target pose is first given, and the parameters of the SSRMS configuration manipulator are combined to generate a partial graph. , part of the graph There are two types of known information: (1) edges that describe the fixed geometric structure of the robot (such as the constant distance between adjacent joint nodes); (2) constraint edges that define the end pose (represented by the distance between the base node and the end joint node).

[0075] Prior network GNNprior will part of the graph Latent space distribution encoded as a multidimensional Gaussian mixture distribution , obtained by sampling Group latent variables Sampling points (This embodiment takes ), each group of sampling points represents a possible solution space region, Group sampling means that the same end pose can be generated simultaneously When sampling, a reference latent variable can be set (e.g., potential distribution sampling obtained when solving the last path point when performing continuous trajectory tracking), based on the reference latent variable Perform nearest neighbor selection and calculate reference latent variables The Euclidean distance from the mean of each component is used to select the component with the closest distance and sample only from that component, so that the generated samples are concentrated in the region with respect to the reference latent variable. The closest distribution area is selected instead of randomly selecting components, thus achieving the goal of Controllable sampling.

[0076] The decoder GNNdec converts the hidden variables With partial diagram Node coordinates of the graph Unknown node positions initialized to 0 and class labels To distinguish between base nodes and joint nodes, fuse, update the 3D coordinate predictions of all nodes in the graph iteratively through a message passing mechanism, and finally output the reconstructed complete graph .

[0077] Select the group with the smallest end pose error from the 32 groups of inverse solutions generated at the same time as the initial value of TRAC-IK to fine-tune the accuracy.

[0078] The output results before and after using TRAC-IK are shown in the following table:

[0079]

[0080] It is apparent to those skilled in the art that the present application is not limited to the details of the foregoing exemplary embodiments, and that the present application can be implemented in other embodiments without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in a descriptive sense only and not for purposes of limitation. The scope of the present application is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. No reference signs in the claims shall be construed as limiting the scope of the claims.

[0081] Furthermore, it should be understood that although the present specification is described in terms of embodiments, not every embodiment according to the present specification need necessarily include every component identified herein. The description herein of one or more embodiments, including the best mode, is intended to convey an appreciation of the principles disclosed herein to others skilled in the art and is not meant to limit the scope of the application to a single preferred embodiment. Thus, the scope of the present application should be determined by the appended claims and equivalents thereof.

Claims

1. A method for solving inverse kinematics of redundant space robots based on graph neural networks, characterized by: The following steps are involved: Step 1: Generate training dataset: Construct a distance geometry graph to represent the manipulator configuration of the redundant space robot, transform the inverse kinematics problem into a partial graph completion problem, import the urdf model to construct a distance geometry graph for each manipulator configuration, and uniformly sample the joint configuration in the joint space. , is the degree of freedom, the joint configuration of each group of samples Corresponding to a complete graph and a partial graph , each ( ) is a piece of data and stores the data set; Step 2: Model training: Based on the equivariant graph neural network EGNN and conditional variational autoencoder, the training network includes the encoder GNNenc, the prior network GNNprior and the decoder GNNdec, where the encoder GNNenc converts the complete graph into Encoded as latent variables Distribution, expressed as ; Prior network GNNprior from part of the graph Generate a prior distribution, expressed as ; The decoder GNNdec is based on latent variables and part of the graph Get the reconstructed complete graph , expressed as , the loss function training model is based on the evidence lower bound ELBO; Step 3: Model reasoning: During inference, the input is the target’s end pose And the urdf model of the robotic arm, build part of the graph , containing known base nodes and the end joints The distance constraint of the part graph Input the trained prior network GNNprior and output the latent variables that satisfy the Gaussian mixture distribution , from the latent variable Extract Group sampling points , , each set of sampling points Corresponding to a potential solution, generate a reconstructed complete graph To obtain joint angle information.

2. The method for solving inverse kinematics of a redundant space robot based on a graph neural network according to claim 1, characterized in that: When constructing the distance geometry diagram in step 1, the base and joint positions of each configuration of the robot arm are set as nodes, and the base is the fixed point. , expressed as , each joint Take two points on the rotation axis , expressed as , Located at the starting end of the joint axis, Located at the end of the joint axis and meeting the distance ; On this basis, define the joint nodes determined by the robot arm configuration The fixed distance between the structural edges , defined by the end pose Determine the end joint node To base node The distance to the end edge , define the joint nodes except the end To base node The distance to the joint edge ; Define the structure diagram for the robotic arm configuration , terminal constraint subgraph Reconciliation Diagram ,in: Structure diagram The nodes are all joint nodes , structural edge For adjacent joint nodes A fixed distance between Terminal constraint subgraph The nodes include the base node and the end joints , through the end edge Constrain its position, end constraint subgraph The end position Encoded as distance constraint, structure diagram and terminal constraint subgraph Joint component diagram ; Diagram The node is the base node and all joint nodes , joint edge Supplementary Figures The missing distance in the structure diagram , terminal constraint subgraph Reconciliation Diagram Combined to form a complete graph Uniquely determine a joint configuration .

3. The method for solving inverse kinematics of a redundant space robot based on a graph neural network according to claim 2, characterized in that: The sampling joint configuration in step 1 During the process, the end pose is calculated by forward kinematics , ensuring that each end pose At least one feasible solution is corresponding, and grid space hashing is applied to merge similar poses to avoid data redundancy. The similar poses refer to two adjacent joint configurations. satisfy ,in, and Represents the configuration of two adjacent joints.

4. The method for solving inverse kinematics of a redundant space robot based on a graph neural network according to claim 3, characterized in that: The training network in step 2, the encoder GNNenc input is the complete graph , containing node coordinates , category labels and including structural edges , end edge and joint edges All edge distances are passed to the EGNN layer for message passing and joint node updates, and the output is the hidden variable The distribution parameters of and the variance is ; Prior network GNNprior input is part of the graph , through the multi-head EGNN network, the separated MLP head outputs the mixed Gaussian parameters; the decoder GNNdec input is the hidden variable and part of the graph The splicing uses the same EGNN network as the encoder, and the output is the Gaussian distribution parameters of the joint node positions.

5. The method for solving inverse kinematics of a redundant space robot based on a graph neural network according to claim 4, characterized in that: The loss function in step 2 is expressed as: Where, Represents the latent variable In distribution Under expectations, To adjust the coefficients, the training objectives include minimizing the pose error of the solution while reducing the encoder output and the prior network Divergence, constrained latent variables The distribution is close to the prior.

6. The method for solving inverse kinematics of a redundant space robot based on a graph neural network according to claim 5, characterized in that: The conditional variational autoencoder is distributed from latent variables mid-sampling To calculate the gradient of ELBO, design reparameterization techniques and introduce auxiliary noise variables , rewrite the sampling as .

7. The method for solving inverse kinematics of a redundant space robot based on a graph neural network according to claim 1, characterized in that: Extract sampling points in step 3 When setting the reference latent variable , based on the reference latent variable during the sampling process Perform nearest neighbor selection and calculate reference latent variables The Euclidean distance to the mean of each component is used to select the component with the closest distance and sample only from that component.

8. The method for solving inverse kinematics of a redundant space robot based on a graph neural network according to claim 1, characterized in that: In step 3, the reconstructed complete graph Perform parallel screening or feed the TRAC-IK numerical optimizer as initial values ​​for millisecond-level fine-tuning.

Images