Method for controlling a robot device

Abstract

Method (500) for controlling a robot device (100), comprising: Providing (501) demonstrations of a robot capability, each demonstration showing a trajectory comprising a sequence of robot configurations, each robot configuration being described at least partially by a point on a first Riemannian manifold; Determine (502) a vector field on a second Riemannian manifold; Training (503) a diffeomorphism between the first Riemannian manifold and the second Riemannian manifold by searching for parameter values ​​of the diffeomorphism that minimize a loss, including for each demonstration and for at least one robot configuration of the trajectory shown by the demonstration, a difference between a configuration change to the next configuration of the trajectory and a configuration change given by the specific vector field via the diffeomorphism, where determining the configuration change given by the specific vector field for a configuration includes Mapping the point on the first Riemannian manifold that describes the configuration to a point on the second Riemannian manifold using the diffeomorphism; determining a change specified by the vector field at the point on the second Riemannian manifold; mapping the determined change to the tangent space of the point on the first Riemannian manifold using the differential of the inverse diffeomorphism, where the configuration change is given by the result of the mapping; and Control (504) of the robot device (100), for a configuration for which a control action is to be determined, according to a configuration change given by the specific vector field over the trained diffeomorphism for the configuration for which a control action is to be determined.

Description

State of the art

[0001] The present invention relates to methods for controlling a robot device.

[0002] In many applications, it is desirable for robots to be able to function autonomously in potentially dynamic and unstructured environments. To do this, they need to learn how to move and interact with their environment. To do this, robots may rely on a library of skills that can be used to perform simple movements or complex tasks such as a combination of several skills. One way to learn movement skills is through human examples, known as learning from demonstrations (LfD). This involves a (typically human) expert demonstrating one or more times a specific skill (e.g., a movement) that the robot is expected to imitate.

[0003] Learning from demonstrations enables robots to directly acquire human-like capabilities for performing a wide variety of tasks. Under the LfD framework, demonstrations are usually represented as state-action trajectories, for which a state might be, for example, the robot end-effector position, and an action the required Cartesian velocities. In the context of robot movement learning, approaches such as behavioral cloning offer the opportunity to learn a direct mapping (i.e., a function) from states to actions using demonstration data. However, these methods can suffer from data mismatch and extrapolation (generalization) difficulties, sometimes leading to unreliable movements when generated outside of demonstration data support.One way to overcome this is to leverage control theory, particularly stability certificates, which can improve the generalization capabilities of motion learning methods by introducing stability guarantees. Therefore, the problem of robot motion learning requires building a model that encapsulates the demonstrated motion dynamics and guarantees stability.

[0004] In these aspects, the usual assumption is that robot motion trajectories lie in Euclidean spaces. However, this assumption does not strictly hold when considering the motion of the robot end-effector due to the orientation representation. For example, there are various types of representations for orientation, such as quaternions in the 3-sphere manifold and rotation matrices in the manifold of the special orthogonal group (SO(3)).

[0005] Therefore, an object of the invention is to provide a stable training system for a robot device, particularly for learning from demonstrations, that can effectively handle the geometric constraints of the demonstration data in non-Euclidean spaces. This object is achieved according to the set of claims.

[0006] DE 10 2021 204 961 A1 relates to a method for controlling a robot device.

[0007] DE 10 2020 214 231 A1 relates to a method for controlling a robot device.

[0008] DE 10 2020 127 508 A1 concerns devices, systems and techniques that estimate the pose of an object while the object is manipulated by a robot limb.

[0009] DE 10 2019 216 229 A1 relates to a method for controlling a robot device.

[0010] EP 3 838 503 A1 relates to a method for controlling a robot.

[0011] WO 2022 / 171 573 A1 relates to a dynamic nonlinear system with multiple degrees of freedom, wherein the system has at least one potential element, wherein eigenmodes of the system are generated by the potential element. Disclosure of the invention

[0012] According to various embodiments, a method for controlling a robotic device is provided, including providing demonstrations for a robot capability, each demonstration showing a trajectory including a sequence of robot configurations, each robot configuration being described at least in part by a point on a first Riemannian manifold, determining a vector field on a second Riemannian manifold, and training a diffeomorphism between the first Riemannian manifold and the second Riemannian manifold by searching for parameter values ​​of the diffeomorphism that minimize a loss.The loss includes, for each demonstration and for at least one robot configuration of the trajectory shown by the demonstration, a difference between a configuration change to the next configuration of the trajectory and a configuration change given by the determined vector field via the diffeomorphism.Determining the configuration change given by the determined vector field for a configuration involves mapping the point on the first Riemannian manifold describing the configuration to a point on the second Riemannian manifold using the diffeomorphism, determining a change specified by the vector field at the point on the second Riemannian manifold, and mapping the determined change to the tangent space of the point on the first Riemannian manifold using the differential of the inverse diffeomorphism, where the configuration change is given by the result of the mapping.The method further includes controlling the robot device, for a configuration for which a control action is to be determined, according to a configuration change given by the determined vector field over the trained diffeomorphism for the configuration for which a control action is to be determined.

[0013] The method described above enables learning from demonstrations for dynamic motions from a Riemannian perspective, where position-velocity pairs (generally pairs of configuration (state) and a configuration change (action) that should be performed in the configuration) describe a vector field on a Riemannian manifold (corresponding to a state-action mapping function). The underlying Riemannian geometry is integrated during the learning process to ensure that the learned vector field (on the first Riemannian manifold) satisfies the underlying geometry of the demonstration data. For this purpose, a diffeomorphism is used which, if the given vector field (on the second Riemannian manifold) is stable, ensures stability (e.g.,in the sense of Lyapunov) of the learned control strategy (given by the learned vector field on the first Riemannian manifold).

[0014] This can be achieved by determining a (e.g., canonical) stable vector field according to the Lyapunov stability requirements on a predetermined space (here, the second Riemannian manifold, e.g., for orientations, but the same can also be done for the Euclidean space for positions, where the situation is much simpler). Then, a vector field on the first Riemannian manifold is provided by learning a diffeomorphism between the desired (observed) demonstration data and the second Riemannian manifold (representing a stable system by means of the (pre)determined vector field). The learned vector field can be used to reproduce stable dynamics in the controller that mimic the demonstration trajectories.

[0015] The above approach can be applied to a wide variety of applications where data resides on a Riemannian manifold. It guarantees global asymptotic stability of the robotic device along with high accuracy of motion reproduction.

[0016] Various examples are given below.

[0017] Example 1 is a method for controlling a robot device as described above.

[0018] Example 2 is the procedure of Example 1, including determining that the vector field is a stable vector field.

[0019] Thus, it can be achieved that the control actions performed according to the configuration changes given by the particular vector field via the diffeomorphism lead to a stable and efficient control of the robot device.

[0020] Example 3 is the method of Example 1 or 2, including training the diffeomorphism by training a neural network, where the diffeomorphism maps a point to the solution of a neural ordinary differential equation, where the value of the derivative at a point is given by the output of the neural network when supplied with the point.

[0021] Thus, the diffeomorphism can be efficiently trained by training a neural network, since efficient training methods are available for neural networks. Furthermore, a neural ordinary differential equation can be efficiently solved in a forward iteration.

[0022] Example 4 is the method of any of Examples 1 to 3, including determining the vector field on the second Riemannian manifold in dependence on the output of another neural network and training the another neural network together with the diffeomorphism by searching for parameter values ​​of the diffeomorphism and the another neural network to minimize the loss.

[0023] Thus, additional degrees of freedom are introduced in the form of the additional neural network, allowing for a better adaptation to the demonstrations. The additional neural network receives, for example, points on the second Riemannian manifold as input.

[0024] Example 5 is the method of any of Examples 1 to 4, wherein each robot configuration includes an orientation of a component of the robot device described by the respective point on the first Riemannian manifold.

[0025] Orientations can be described more efficiently using (general) Riemannian manifolds instead of Euclidean spaces.

[0026] Example 6 is a robot device controller configured to perform the method of any one of Examples 1 to 5.

[0027] Example 7 is a computer program including instructions that, when executed by a processor, cause the processor to perform a method according to any one of Examples 1 to 5.

[0028] Example 8 is a computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method according to any one of Examples 1 to 5.

[0029] In the drawings, like reference characters generally designate the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being generally placed upon illustrating the principles of the invention. In the following description, various aspects are described with reference to the following drawings, in which: Fig. 1 shows a robot. Fig. 2 shows an illustration of a sphere manifold S2, whose points can, for example, each represent a possible orientation of a robot end effector. Fig. 3 illustrates a diffeomorphism between a first Riemannian manifold and a second Riemannian manifold used to derive robot control actions, according to various embodiments. Fig. Figure 4 illustrates a numerical integration on a manifold using a projection method. Fig. 5 shows a flowchart illustrating a method for controlling a robotic device, according to one embodiment.

[0030] The following detailed description refers to the accompanying drawings, which illustrate specific details and aspects of this disclosure in which the invention may be practiced. Other aspects may be utilized, and structural, logical, and electrical changes may be made without departing from the scope of the invention. The various aspects of this disclosure are not necessarily mutually exclusive, as some aspects of this disclosure may be combined with one or more other aspects of this disclosure to form new aspects.

[0031] Various examples are described in more detail below.

[0032] Fig. 1 shows a robot 100.

[0033] The robot 100 includes a robot arm 101, for example, an industrial robot arm for handling or assembling a workpiece (or one or more other objects). The robot arm 101 includes manipulators 102, 103, 104 and a base (or support) 105 with which the manipulators 102, 103, 104 are supported. The term "manipulator" refers to the movable members of the robot arm 101, the actuation of which enables physical interaction with the environment, e.g., to perform a task. For control, the robot 100 includes a (robot) controller 106 configured to implement the interaction with the environment according to a control program. The last link 104 (furthest from the support 105) of the manipulators 102, 103, 104 is also referred to as the end effector 104 and may include one or more tools such as a welding torch, a gripping instrument, a painting device or the like.

[0034] The other manipulators 102, 103 (closer to the support 105) can form a positioning device, so that, together with the end effector 104, the robot arm 101 is provided with the end effector 104 at its end. The robot arm 101 is a mechanical arm that can provide functions similar to a human arm (possibly with a tool at its end).

[0035] The robot arm 101 may include joint elements 107, 108, 109 that connect the manipulators 102, 103, 104 to each other and to the support 105. A joint element 107, 108, 109 may have one or more joints, each of which may provide a rotatable movement (i.e., rotational movement) and / or a translational movement (i.e., displacement) for associated manipulators relative to each other. The movement of the manipulators 102, 103, 104 may be initiated by means of actuators controlled by the controller 106.

[0036] The term "actuator" can be understood as a component configured to influence a mechanism or process in response to a control. The actuator can implement instructions generated by the controller 106 (the so-called activation) into mechanical movements. The actuator, e.g., an electromechanical converter, can be configured to convert electrical energy into mechanical energy in response to a control.

[0037] The term "controller" can be understood as any type of logic-implementing entity, which may, for example, include a circuit and / or a processor capable of executing software stored in a storage medium, firmware, or a combination thereof, and which can generate instructions, e.g., to an actuator in the present example. The controller can be configured, for example, through program code (e.g., software) to control the operation of a system, a robot in the present example.

[0038] In the present example, the controller 106 includes one or more processors 110 and a memory 111 that stores code and data based on which the processor 110 controls the robot arm 101. According to various embodiments, the controller 106 controls the robot arm 101 based on a machine learning model 112 stored in the memory 111.

[0039] The machine learning model 112 is trained by learning from demonstrations, e.g., provided by a user manually moving the robot arm 101. Each demonstration is a sequence of robot configurations. According to various embodiments, each configuration is given at least in part by a point on a Riemannian manifold. For example, a certain demonstration trajectory τ={yt}t=1T from data points yt∈ℝ3×S3, ie pairs of points, where the first point (of ℝ 3 ) for example, specifies the end effector position and the second point is a quaternion (of S3 ) and specifies the orientation of the end effector. Quaternions have advantageous properties for robot control. However, since quaternions (which are used for robot control) satisfy a unit norm constraint, they do not form a vector space. However, they lie on the Riemannian manifold S3. Another example of a Riemannian manifold whose elements (i.e., points) can be used for robot control is the special orthogonal group (SO(3)).

[0040] A Riemannian manifold M is an m-dimensional topological space for which every point belongs to a Euclidean space ℝ m and which has a globally defined differential structure. For each point x∈M there exists a tangent space TxM, which is a vector space consisting of the tangent vectors of all possible smooth curves passing through x. A Riemannian manifold is provided with a smoothly varying positive definite inner product called a Riemannian metric, which allows curve lengths to be expressed in M These curves, called geodesics, are the generalization of straight lines in Euclidean space to Riemannian manifolds, since they represent the curves of minimum length between two points in M.

[0041] Fig. 2 shows an illustration of a sphere manifold S2, whose points can, for example, each represent a possible orientation of a robot end effector.

[0042] Two points x and y are specified on the sphere, which can be used by the controller 106 to represent two different orientations of the robot end effector 104.

[0043] The shortest distance between the two points in the surrounding space would be a straight line 201, while the shortest path on the manifold is a geodesic 202.

[0044] To use the Euclidean tangent spaces, mappings can be made back and forth between the tangent spaces TxM which are called the exponential mapping and the logarithmic mapping respectively.

[0045] The exponential mapping Expx:TxM→M maps a point u in the tangent space of x to a point y on the manifold such that it lies on the geodesic starting at x in the direction u such that the geodesic distance between x and y is equal to the norm of the distance between x and u. The inverse operation is the logarithm map Logx:M→TxM called, ie u=Logx(y).

[0046] Another useful operation over manifolds is parallel transport Γx→y:TxM→TyM, which moves elements between tangent spaces so that the inner product between two elements in the tangent space remains constant.

[0047] For example in Fig. 2, Γ(v1), Γ(v2) are the parallel transported vectors v1 and v2 of TxM to TyM (the index x→y is omitted for simplicity).

[0048] The following are the expressions for the Riemannian distance, exponential and logarithmic mappings and parallel transport operation for the sphere manifold Sm. Operation Formel dM(x,y) arccos(x T and) Exp x (u) xcos(‖u‖)+u¯sin(‖u‖) with u¯=u‖u‖ Log x (and) dM(x,y)y−xτyx‖y−xτyx‖ C x→y (v) (−xsin(‖u‖)u¯τ+u¯cos(‖u‖)u¯τ+(I−u¯u¯τ))v mit u¯=u‖u‖ und u=Logx(y)

[0049] The task of learning from demonstrations can be viewed as learning a vector field in the respective space of configurations of the robot. In the following, the space of configurations is a Riemannian manifold. However, it should be noted that each configuration may have further components, in particular a component in Euclidean space (such as ℝ 3 in the example above), which is omitted below for simplicity (and which is easier to treat than the more general case of the Riemannian manifold).

[0050] Thus, an approach for learning a vector field on a Riemannian manifold while guaranteeing global asymptotic stability is described below. According to various embodiments, this is achieved by exploiting a diffeomorphism that transforms a target vector field (i.e., the one to be learned) on the Riemannian manifold into a simple canonical (i.e., predetermined) stable vector field on a latent manifold. The diffeomorphism is implemented by a (first) neural network, as described in more detail below.

[0051] Fig. 3 illustrates a diffeomorphism Φ θbetween a first Riemannian manifold 301 (the one on which a vector field is to be learned, i.e., whose points (at least partially) specify robot configurations) and a second Riemannian manifold 302 (i.e., the one referred to above as the Riemannian manifold). On the first Riemannian manifold 301, a target vector field (represented here by the S-form) should be provided (i.e., learned from the demonstrations). On the second Riemannian manifold 302, a vector field is predefined (represented by the “meridians”). It should be noted that the Riemannian manifolds 301, 302 are the same in a topological sense, i.e., there exists a homeomorphism between the two manifolds.

[0052] For example, a C1-diffeomorphism is learned using continuous normalization flows on manifolds. Before learning the diffeomorphism, the target vector field 302 is selected on the latent manifold 302. As expected, one of the conditions that must be met when selecting a latent manifold is that the latent manifold 302 and the original manifold (i.e., the first manifold 301) must be diffeomorphic. This target vector field 302 is predefined to behave as a geodesic vector field, with the path integral on this vector field producing geodesics on the latent manifold.

[0053] To construct a stable target vector field 302, the logarithm mapping can be used to make the vector field follow the geodesics. In particular, considering an attractor point y0∈N the tangential vector ẏ at the location y∈N be calculated as y˙=g(y)=kγ(y)Logy(y0)‖Logy(y0)‖2.

[0054] According to various embodiments, the factor k γ constructed and trained using a (second) neural network with parameter y, using the demonstration data together with the diffeomorphism Φ θ to learn. It should be noted that the factor k γ only needs to be greater than zero at all locations y ≠ y0 for the vector field to be stable on the manifold. According to various embodiments, the exponential function is used, whose exponent is modeled by the second neural network. ψγ:ℝn⊃N→ℝ, i.e. kγ(y)=eψγ(y)+ε, where ε is a very small constant (e.g. the machine precision).

[0055] The first manifold 301 shall be denoted by M and the target vector field shall be V∈TM Learning this vector field means enabling the evaluation of the velocity vx at the location x∈M. It should be noted that a vector field can be defined as a mapping between the manifold and its tangent space v:M→TM can be viewed.

[0056] According to various embodiments, instead of a direct parameterization of the target vector field, the diffeomorphic mapping Φθ:M→N, which is to be applied to project it onto the latent manifold 302, denoted by , learned.

[0057] Learning the target vector field thus involves learning the diffeomorphism (i.e., adjusting its parameters θ, which are, for example, weights of a neural network for implementing the diffeomorphism, as described in more detail below) so that it projects the target vector field onto the predefined (globally stable) vector field on the second manifold 302. The second manifold is, for example, defined by a geodesic vector field gγ:N→TN given. Due to the fact that a diffeomorphism preserves topological properties, the target vector field learned in this way can be expected to remain globally stable as long as the geodesic vector field on the latent manifold is stable.

[0058] After learning the diffeomorphism and its inverse, the respective controller (e.g., controller 106), in order to determine a control action for a current robot configuration, can determine the action for a configuration (i.e., the configuration change such as speed) defined by the point x∈M is specified by • Mapping the point x onto the latent manifold 302 using the learned diffeomorphism Φθ:M→N. • Evaluate the geodesic vector field for the particular change (e.g., velocity) for the latent variable y (i.e., the point on the second Riemannian manifold to which x has been mapped using the diffeomorphism) • Using the pullback operator DyΦθ*, to transfer the certain change back to the tangent space of the first manifold 301 at the location x∈M The pullback operator can be viewed as the inverse mapping of the differential of the diffeomorphism, i.e. the inverse of DxΦθ:TΦθ(x)N→TxN, which is obtained by calculating the derivatives of Φ θ and then deriving the inverse or calculating the derivatives of the inverse diffeomorphism.

[0059] As in Fig. 3, this procedure can be combined into a single equation as follows x˙=(DyΦθ*∘gγ∘Φθ)(x)=DΦθ*(y˙)

[0060] As mentioned above, continuous normalization flows on manifolds can be used to construct the (trainable) diffeomorphism.

[0061] In particular, according to various embodiments, a residual flow is used to construct the diffeomorphism. A residual flow can be represented as the composition of residual connections of the following form zk+1=zk+ƒθ(zk) where f θ a feedforward neural network and z k represents a k-th hidden state.

[0062] Iterating through equation (2) (from a start index s to an end index e) results in calculating the transformation of a point z S = x to a point y = z e . Furthermore, by limiting the Lipschitz constant of f θ to less than 1, ie Lip(f θ ) < 1, the invertibility of the residual blocks is sufficiently ensured.

[0063] The residual flow can be due to the fact that the neural network f θcalled a neural ODE, can be regarded as an Euler discretization of a continuous ordinary differential equation (ODE), ddtz(t)=ƒθ(z(t),t)

[0064] The diffeomorphic mapping given by the residual flow (and thus given by the neural network and its weights) can be obtained by iteratively traversing the residual network f θ This means that calculating the diffeomorphism is equivalent to solving an initial value problem (or integral). To do this, the initial point x = z(t s ) ∈ ℝ n used to calculate y by y=Φθ(x)=x+∫τ=isteƒθ(z(τ),τ)dτ where the result of the calculation on the right-hand side is considered to be a point of the second manifold 302 by means of the homeomorphism between the two manifolds.

[0065] The inverse diffeomorphism Φθ−1 can be easily obtained by reversing the time interval for integration, ie x=Φθ−1(y)=y+∫τ=tetsƒθ(z(τ),τ)dτ (e.g. according to the machine learning model 112)

[0066] The above describes the construction of a diffeomorphism under the Euclidean setting. To account for the underlying geometry of data (i.e., configurations represented at least partially by points on a Riemannian manifold), the continuous normalization flow is extended to a manifold setting, i.e., where x,zk∈M and ƒθ:M×ℝ→TM. This leads to the formulation of neural manifold ODEs. It should be noted that neural ODEs are merely a special case of neural manifold ODEs when the manifold is Euclidean. Since Euclidean space is merely a special case of manifolds, all references to equation (3) below are intended in the context of neural manifold ODEs.

[0067] In the general manifold case, the diffeomorphism can also be computed using equation (4) by performing the integration over the manifold instead of in Euclidean space.

[0068] Integration on manifolds can be performed using projection methods and integrators based on local coordinates.

[0069] Fig. Figure 4 illustrates a numerical integration on a manifold using a projection method.

[0070] With an ODE given by equation (3) on a manifold , where ƒθ(⋅,t)∈Tz(t)M for all z(t)∈M, The projection method calculates a one-step integral with an arbitrary numerical integrator in Euclidean space and then projects the value onto the manifold to obtain the integral results on the manifold, as in Fig. 4. For example, an Euler solver with step size dt is used, which takes a single step from the initial point z a The next state z̃1 is then calculated as z̃1 = z0 + f θ (z0, 0)dt. This is continued until the end point z̃ e which is projected back onto the manifold.

[0071] Integration methods on manifolds based on local coordinates consider a local representation of the manifold by a coordinate map, which allows a formulation of an equivalent ODE in Euclidean space that can be solved and, using the coordinate map with inverse map, yields a solution on the manifold.

[0072] To implement a manifold ODE in a deep learning framework (for training the neural network f θ), the gradient of the loss with respect to the neural network weights must be efficiently computed. This can be achieved using adjoint sensitivity methods, which allow treating the ODE solver as a black box and compute gradients backward in time by solving a second augmented adjoint ODE, rather than directly differentiating them with ODE solvers. This approach is memory-efficient, especially for adaptive ODE solvers that adjust the step size on the fly.

[0073] It should be noted that using a neural network to model the dynamics of a manifold ODE imposes some constraints on the neural network. Due to this, according to various embodiments, a projection operator is added to the last layer of a "freeform" neural network (i.e., a neural network without these constraints) to project its output onto the tangent space at the input location. Let the robot configuration be specified (at least partially) by points on a manifold M of dimension d, which is embedded in a surrounding n-dimensional Euclidean space. Given a freeform neural network ψ θ : ℝ n × ℝ → ℝ n is the figure ƒθ:ℝn×ℝ→TM then given by f θ (z(t),t) = proju ∘ ψ θ (z(t),t)

[0074] Input data to the neural network is assumed to be on manifold 301. The neural network can be, for example, a fully connected neural network (FCNN) with tanh activation function.

[0075] As described above, after obtaining the configuration change given for the point x by the vector field on the second Riemannian manifold via the diffeomorphism, the tangent vector ẏ for the latent variable y (i.e., the point to which the diffeomorphism maps x) is mapped back to the tangent space at the location x∈M by the pullback operator DyΦθ* shown.

[0076] This operator can be viewed as the inverse map of the differentials of the diffeomorphism, i.e., the differentials that map ẋ to ẏ: DxΦθ(x)x˙=y˙

[0077] In the Euclidean case, ẋ can be calculated directly by multiplying ẏ by the inverse of D x Φ θ However, in a manifold environment, directly computing the inverse is difficult, especially for a d-dimensional submanifold in an n-dimensional Euclidean space ℝ n This is due to an unavoidable loss of rank in the matrix representation for D x Φ θ (x).

[0078] For example, the tangent space of a 2-sphere located in ℝ 3 also has the same dimension as the manifold itself, which implies that the operator D x Φ θ (x) should be a mapping between two 2-dimensional spaces. However, since the 2-sphere and its tangent space are embedded in the 3-dimensional surrounding Euclidean space, the matrix representation for D x Φ θ(x) a 3-by-3 matrix. A 3-by-3 matrix for mapping between 2-dimensional spaces must be rank-deficient and thus cannot be directly inverted. It is also typically not helpful to compute the generalized inverse of this matrix, since this is equivalent to computing a minimum-norm solution for ẋ, which does not satisfy the properties of a tangent vector. According to various embodiments, this is addressed by introducing a constraint that allows imposing ẋ on the tangent space. In the case of a sphere manifold, the tangent vector ẋ must be perpendicular to the position vector x, which can be formulated as xTx˙=0

[0079] Combining these two equations (6) and (7) yields a unique solution. A solution can be found by solving the linearly constrained problem either by creating a new system of linear equations or by applying a Lagrange multiplier. Below, we will show how to augment the matrix to generate a new linear equation and calculate the pseudo-inverse. Since the augmented matrix has full rank, the pseudo-inverse produces the same result as the direct inverse. (DxΦθ(x)xT)x˙=(y˙0) (DxΦθ(x)xT)T(DxΦθ(x)xT)x˙=(DxΦθ(x)xT)T(y˙0) (DxΦθ(x)TDxΦθ(x)+xxT)x˙=DxΦθ(x)Ty˙

[0080] This leads to the unique solution and the expression of the pullback operator D x Φ θ , formulated as x˙=(DxΦθ(x)TDxΦθ(x)+xxT)−1DxΦθ(x)Ty˙ DyΦθ*=(DxΦθ(x)TDxΦθ(x)+xxT)−1DxΦθ(x)T

[0081] Given the above derivation, the pullback operator DyΦθ* which is specific to spheres, since the restriction defined in equation (7) is only valid for sphere manifolds.

[0082] While in the sphere-manifold setting, the constraint is linear and easy to compute or satisfy, in other manifolds it can be nonlinear and complex. Therefore, introducing a nonlinear constraint into equation (6) requires an iterative procedure to solve a nonlinearly constrained optimization problem, which is computationally intensive.

[0083] Therefore, according to various embodiments, in order to generalize the computation for arbitrary manifolds, a way to determine the pullback operator DyΦθ* based on modified adjoint methods. As mentioned above, the pullback operator can also be written as a differential of an inverse diffeomorphism Dy(Φθ−1) Thus, one approach to determining the pullback operator is to first calculate the inverse diffeomorphism and then obtain its derivatives. Indeed, the procedure for calculating the diffeomorphism and the inverse diffeomorphism are identical for our manifold ODE, and only the time interval needs to be inverted, i.e., instead of integrating t s night e To obtain the diffeomorphism, the integration of t e night s to obtain the inverse diffeomorphism.

[0084] Furthermore, an adjoint method can be used to calculate the differential of the inverse diffeomorphism.

[0085] For this, A*(t) is defined as an adjoint variable as D z(t) Φ -1 defined. Algorithm 1 shows that the differential of the inverse diffeomorphism is given with a single pass of the ODE solver on the manifold, while the result for the diffeomorphism is obtained without any additional effort. Therefore, no forward computation is required before proceeding with the backpropagation.

[0086] The function ManifoldODESolve solves the manifold ODE (e.g., using a dynamic charting method). Here, the main challenge lies in calculating the differential of the vector fields on the manifold D z(t) fθ(z(t), t), whereas only partial derivatives are required in the Euclidean situation. Directly calculating the differential of vector fields can be avoided by several techniques, e.g., as described below.

[0087] Using an exponential map φi−1=Expzi,φi=Logzi, the inverse diffeomorphism Φθ−1:y=zk↦z0=x be written as Φθ−1=Logz0−1∘ƒ^0−1∘Expz0−1∘…∘Logzk−1−1∘ƒ^k−1−1∘Expzk−1−1=Expz0∘ƒ^0−1∘Logz0∘…∘Expzk−1∘ƒ^k−1−1∘Logzk−1.

[0088] Da f̂ j : y i (t s ) ↦ y i (t e ) and ƒ^i−1:yi(te)↦yi(ts) corresponding to solving the equivalent ODE forward and backward in time in Euclidean space, respectively, the differential of these mappings collapses to classical partial derivatives. This allows the effective use of Algorithm 1 in Euclidean space to find f̂ i and its partial derivatives in a single call to a classical ODE solver. In summary, traversing a single component and calculating its differential can be written as zi=(Expzi∘ƒ^i−1∘Logzi)zi+1Dzi−1zi=Dyi(ts)Expzi∘Dyi(te)ƒ^i−1∘Dzi−1Logzi, where an exponential and logarithmic map, as well as their derivatives, have analytic expressions. Iteratively solving equation (8) yields the results of the diffeomorphism and the pullback operator.

[0089] In summary, according to various embodiments, a method as in Fig. 5 is provided as an illustration.

[0090] Fig. 5 shows a flowchart 500 illustrating a method for controlling a robotic device.

[0091] At 501, demonstrations (e.g., by a human user) of a robot capability are provided, each demonstration showing a trajectory including a sequence of robot configurations, each robot configuration being described at least in part by a point on a first Riemannian manifold.

[0092] At 502, a vector field is (pre)defined on a second Riemannian manifold (homeomorphic to the first Riemannian manifold, so that it can also be regarded as a second “instance” of the Riemannian manifold).

[0093] At 503, a diffeomorphism between the first Riemannian manifold and the second Riemannian manifold is trained by searching for parameter values ​​of the diffeomorphism that minimize a loss including, for each demonstration and for at least one robot configuration of the trajectory shown by the demonstration, a difference between a configuration change to the next configuration of the trajectory and a configuration change given by the determined vector field over the diffeomorphism.

[0094] Here, determining the configuration change given by the particular vector field for a configuration involves • Mapping the point on the first Riemannian manifold that describes the configuration to a point on the second Riemannian manifold using the diffeomorphism; • Determining a change specified by the vector field at the point on the second Riemannian manifold; and • Mapping the determined change to the tangent space of the point on the first Riemannian manifold using the differential of the inverse diffeomorphism. The result of this mapping is the configuration change given by the determined vector field for a configuration via the diffeomorphism.

[0095] At 504, the robot device is controlled for a configuration for which a control action is to be determined according to a configuration change given by the determined vector field over the trained diffeomorphism for the configuration for which a control action is to be determined (where the configuration change can be determined according to the above three bullet points).

[0096] Controlling the robot device according to a configuration change can be understood as controlling the robot device to perform the configuration change. For example, the configuration change specifies a speed in a certain direction, and the robot device is controlled to move (or rotate) at the speed in that direction.

[0097] Riemannian manifolds, for example, are spaces of points that specify an orientation of the robot device, e.g., the orientation of an end effector.

[0098] The approach of Fig. 5 enables the learning of a control strategy (corresponding to a vector field on the first Riemannian manifold) via human demonstrations, where path integrals (geodesics) are natural motion abilities. The vector field is learned on the first Riemannian manifold by learning a diffeomorphism that maps the desired (observed) demonstration data to the (pre)determined (stable) vector field on the second Riemannian manifold. Therefore, the global stability of the learned vector field can be guaranteed by the diffeomorphism applied to the (pre)determined vector field.

[0099] The procedure of Fig. 5 may be performed by one or more computers including one or more data processing units. The term “data processing unit” may be understood as any type of entity that enables the processing of data or signals. For example, the data or signals may be handled according to at least one (i.e., one or more than one) specific function performed by the data processing unit. A data processing unit may include or be formed from an analog circuit, a digital circuit, a mixed-signal circuit, a logic circuit, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), an integrated circuit, or any combination thereof.Any other way of implementing the respective functions may also be understood as a data processing unit or logic circuit arrangement. It is understood that one or more of the method steps described in detail herein may be executed (e.g., implemented) by a data processing unit through one or more specific functions performed by the data processing unit.

[0100] Various embodiments may receive and use image data from various visual sensors (cameras) such as, for example, video, radar, LiDAR, ultrasound, thermal imaging, sonar, etc., to obtain the demonstration data.

[0101] The approach of Fig.5 can be used to calculate a control signal for controlling a technical system, such as a computer-controlled machine such as a robot, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system. According to various embodiments, a strategy for controlling the technical system can be learned, and then the technical system can be operated accordingly.

[0102] According to one embodiment, the method is computer-implemented.

Claims

[1] A method (500) for controlling a robot device (100), comprising: Providing (501) demonstrations for a robot capability, each demonstration showing a trajectory comprising a sequence of robot configurations, each robot configuration being described at least in part by a point on a first Riemannian manifold; Determining (502) a vector field on a second Riemannian manifold; Training (503) a diffeomorphism between the first Riemannian manifold and the second Riemannian manifold by searching for parameter values of the diffeomorphism that minimize a loss including for each demonstration and for at least one robot configuration of the trajectory shown by the demonstration, a difference between a configuration change to the next configuration of the trajectory and a configuration change given by the determined vector field via the diffeomorphism, wherein determining the configuration change given by the determined vector field for a configuration comprises Mapping the point on the first Riemannian manifold describing the configuration to a point on the second Riemannian manifold using the diffeomorphism; determining a change specified by the vector field at the point on the second Riemannian manifold; mapping the determined change to the tangent space of the point on the first Riemannian manifold using the differential of the inverse diffeomorphism, where the configuration change is given by the result of the mapping; and Controlling (504) the robot device (100), for a configuration for which a control action is to be determined, according to a configuration change given by the determined vector field over the trained diffeomorphism for the configuration for which a control action is to be determined. [2] The method of claim 1, comprising determining that the vector field is a stable vector field. [3] A method according to claim 1 or 2, comprising training the diffeomorphism by training a neural network, the diffeomorphism mapping a point to the solution of a neural ordinary differential equation, the value of the derivative at a point being given by the output of the neural network when supplied with the point. [4] A method according to any one of claims 1 to 3, comprising determining the vector field on the second Riemannian manifold in dependence on the output of a further neural network and training the further neural network together with the diffeomorphism by searching for parameter values of the diffeomorphism and the further neural network to minimize the loss. [5] A method according to any one of claims 1 to 4, wherein each robot configuration comprises an orientation of a component of the robot device described by the respective point on the first Riemannian manifold. [6] Robot device controller adapted to carry out the method according to any one of claims 1 to 5. [7] A computer program comprising instructions which, when executed by a processor, cause the processor to perform a method according to any one of claims 1 to 5. [8] A computer-readable medium storing instructions which, when executed by a processor, cause the processor to perform a method according to any one of claims 1 to 5.

Images