Methods for learning neural network parameters, generating exoskeleton trajectories, and moving exoskeletons

Abstract

The present invention provides a method for learning parameters of a neural network for generating trajectories of an exoskeleton (1), the method comprising: (a) performing, by a data processing means (11a) of a first server (10a), the steps of: (a) learning parameters of a first neural network suitable for generating periodic basic trajectories of the exoskeleton (1) according to a first database for learning periodic trajectories for a set of possible gaits of the exoskeleton (1), each of the parameters being for a given gait of the exoskeleton defined by n sets of gait parameters; and (b) using parameters from the first neural network, learning parameters of a second neural network suitable for generating periodic basic trajectories of the exoskeleton (1) and transitions from one periodic basic trajectory of the exoskeleton (1) to another periodic basic trajectory of the exoskeleton (1) according to a second learning database of periodic basic trajectories and transitions for a set of possible gaits of the exoskeleton (1).

Description

[Technical field]

[0001] The present invention relates to the field of robots of the exoskeleton type, more particularly to a method for learning parameters of a neural network, a method for using a neural network to generate a trajectory of an exoskeleton, and a method for moving an exoskeleton. [Background technology]

[0002] Recently, for people with substantial mobility problems such as paraplegia, walking aids called exoskeletons have appeared, which are external robotic devices "worn" by an operator (human user) thanks to a system of fasteners that link the movements of the exoskeleton to the operator's own movements. Exoskeletons for the lower limbs generally have several joints at least at the knees and hips to reproduce the movements of walking. Actuators allow these joints to be moved, and therefore the operator to move. An interface system allows the operator to give instructions to the exoskeleton, and a control system converts these instructions into commands for the actuators. The devices are generally equipped with sensors.

[0003] These exoskeletons represent an advancement over wheelchairs, as they allow the operator to stand and walk. Exoskeletons are no longer limited by wheels and can theoretically operate in most environments that are not flat. Wheels, unlike legs, cannot traverse substantial obstacles such as steps, stairs, or obstacles of too great height.

[0004] However, none of these exoskeletons are capable of performing autonomous human walking (i.e., stable, viable, and unassisted artificial walking) over a wide range of terrains in their use.

[0005] In most cases, these limitations are due to the device's inability to manage balance or gait direction by itself, two tasks that are usually transferred to the operator, who performs them with the aid of crutches, as proposed for example in US Pat. No. 5,399,410 or US Pat. No. 5,499,420.

[0006] Patent document 3 describes the only exoskeleton that can be used without external assistance for people who are unable to ensure their own stability. The control principle described in paragraph 0122 of patent document 3 clearly explains the need to transfer the center of pressure (the physical point corresponding to the result of the reaction forces acting on the system by the ground) of one part of the support polygon (the convex envelope of the contact points with the ground) to another part of the support polygon.

[0007] This limitation results in very slow walking (a few meters per minute, compared to a typical walking speed of over 2 km / h) with short steps (less than 30 cm, compared to a typical stride length of 50-80 cm), during which the supporting foot is always in flat contact with the ground. This limits the type of environment that can be accessed, since uneven terrain is practically excluded. Similarly, even slight obstacles such as small pebbles or small objects pose a risk of losing balance and ultimately causing the system to fall if the system places a foot on them at any point.

[0008] In contrast, "natural" human gait is characterized by a series of phases in which the foot can be flat on the ground, in the air, or rolling on the ground. The ability to roll the foot is essential to walking because it allows for larger strides and stability on a wide variety of terrains.

[0009] However, the so-called first generation exoskeletons mentioned above do not have actuating feet or maintain supporting feet on the ground.

[0010] For a bipedal humanoid robot or robotic device, performing this rolling is indeed complicated: when the center of pressure reaches the limit of the support polygon, the system starts to rotate around this point, so that it is no longer in static equilibrium.

[0011] In walking, foot rolling causes partial loss of contact with the ground in the supporting leg, which has several consequences. -Support polygons (maintaining faces) are potentially reduced to a point where it becomes difficult or even impossible to maintain a center of pressure within the support polygons. -The system is under-actuated, i.e. it can no longer act in all of its degrees of freedom. Then, any movement is no longer possible.

[0012] In such circumstances, the traditional form of flat-footed walking as described for example in Non-Patent Document 1 or the principle described in Patent Document 3 no longer works.

[0013] A natural idea is how to talk about the process of "falling", where one swings a leg forward and places the second foot on the ground to return to the support polygon and balance, while the system is free to rotate about the supporting foot, and this is called dynamic walking, since the body takes a series of unstable postures only as transitions (if the person were "stopped" in place, they would fall).

[0014] In this dynamic walking approach, it is complicated to place the swinging leg in a position to quickly re-establish balance, at least for a short time. Indeed, if this foot were made to follow a parameterized trajectory in precalculated time, there would be a risk that the uncontrollable behavior of the underactuated system would cause the foot to hit the ground too early or too late, even when subjected to a slight disturbance (it is impossible to correct a trajectory that deviates slightly from the planned trajectory). This could cause discomfort to the operator, leading to him becoming unbalanced or even causing him to fall on terrain, including simple ones.

[0015] For this reason, all first generation exoskeletons (and many humanoid robots) try to avoid this situation by keeping the supporting foot flat, resulting in the aforementioned problems of limited walking speed, limited stride length, type of acceptable terrain, and general walking stability.

[0016] As a result, a new walking paradigm for exoskeletons was proposed in Patent Document 4, combining the principles of "virtual constraints" and "hybrid zero dynamics" (HZD) to enable fast and natural walking without the risk of falls or imbalance, even on difficult and unexpected terrain.

[0017] Traditionally, the trajectory, the change in each degree of freedom, is expressed as a function of time. The "dynamics" of a system is the function

[0018]

number

[0019]

number

[0020]

number

[0021] In contrast, HZD is the dynamics of the unactuated degrees of freedom. This dynamics is called "zero" because it corresponds to a degree of freedom where no command can or does not want to act, i.e. the command has a value of 0. And it is called "hybrid" because the impact of the foot on the ground imposes a discontinuous instantaneous phase that intersects with a continuous phase.

[0022] In the so-called "virtual constraint" method, the principle is to define, for a selection of the degrees of freedom of movement, trajectories parameterized by changing parameters, which depend directly on the configuration and not on the time, and which are called phase variables. An example of such a phase variable is the angle between the heel-hip axis and the vertical direction, which constitutes the non-actuated degrees of freedom mentioned above.

[0023] The phase variable allows to define the "progression" of the steps. More precisely, at each step the phase variable successively switches from an initial value to a final value before the initial value is reassigned, which is the start of the next step. To make it easier, the values ​​of the phase parameter can be normalized between 0 and 1.

[0024] Each value of the varying parameter corresponds to a value of an actuation degree of freedom that the system must obey, and these relationships (one for each actuation degree of freedom that is desired to be controlled in this way) are called virtual constraints.

[0025] If the system strictly follows this trajectory for the degrees of freedom in which it is possible and desirable to operate, i.e., if virtual constraints are compiled for these degrees of freedom, then the evolution of the system will be determined entirely by these non-actuated degrees of freedom, which follow their own dynamics, the HZD.

[0026] Therefore, a good choice of virtual constraints can make this dynamics contain attractive periodic "orbits", i.e. stable orbits to which the system is naturally attracted.

[0027] This HZD method gives great satisfaction, but the trajectories are difficult to generate (as is the case with the "flat foot" method). In fact, a given trajectory must be stable, related to the "characteristics" of the gait, such as stride length, step frequency and inclination of the bust (step height when negotiating stairs, instantaneous rotation angle in bending movements, position of the center of the bust and also lateral deviation in the framework of rehabilitation activities). If the characteristics of the gait change, a new trajectory is necessary. [Prior art documents] [Patent documents]

[0028] [Patent Document 1] U.S. Patent No. 7,153,242 [Patent Document 2] US Patent Application Publication No. 2016 / 038371 [Patent Document 3] European Patent No. 2231096 [Patent Document 4] International Publication No. 2018 / 130784 [Non-patent literature]

[0029] [Non-Patent Document 1] Kajita S., KF (2003), Biped Walkingpattern generation by using preview control of Zero-Moment Point, ICRA, (pp.1620-1626) Summary of the Invention [Problem to be solved by the invention]

[0030] Today, no solutions exist that allow generating trajectories in real time; they must be pre-calculated and pre-tested, and there are several reasons for this. -Current trajectory generation algorithms are based on methods for optimizing nonlinear nonconvex problems under constraints. This kind of algorithm does not guarantee convergence. Therefore, it is not possible to be sure that a trajectory for the desired walking characteristics can be obtained even if we are limited to a mathematically compact search space. Solving such a nonlinear, non-convex problem under the constraints is complex and requires significant computational power: a computation time of about 90 s per trajectory per physical core on a high-end computer is required, but to consider real-time operation this time should be about 100 μs. Currently, calculations are performed on a remote server and then the results are transferred over the Internet, which requires the user of the exoskeleton to have a permanent and stable connection, which represents a major inconvenience for use in everyday life.

[0031] This allows the exoskeleton operator to perform the task without the slightest risk. It would be desirable to have a new solution for generating trajectories that are ultimately executable in real time. [Means for solving the problem]

[0032] Therefore, according to a first aspect, the invention relates to a method of learning parameters of a neural network to generate a trajectory of an exoskeleton, said method comprising the steps of: (a) learning parameters of a first neural network suitable for generating periodic basic trajectories of the exoskeleton according to a first database for learning periodic trajectories for a set of possible gaits of the exoskeleton, each of the parameters being for a given gait of the exoskeleton defined by a set of n gait parameters; (b) using parameters from the first neural network to learn parameters of a second neural network suitable for generating periodic basic trajectories of the exoskeleton and transitions from one periodic basic trajectory of the exoskeleton to another periodic basic trajectory of the exoskeleton according to a second learning database of periodic basic trajectories and transitions for a set of possible gaits of the exoskeleton; This includes carrying out the following:

[0033] According to another advantageous but non-limiting feature, step (a) includes constructing the first training database of periodic trajectories for a set of possible gaits of the exoskeleton by using an optimization algorithm.

[0034] The set of possible gaits for the exoskeleton is selected such that the space in which the n sets of gait parameters have values ​​is uniformly covered.

[0035] The step (a) further comprises verifying a criterion representative of the accuracy of the prediction of the first neural network, and if the criterion is not verified, the step (a) is repeated.

[0036] The step (b) includes using the first training database to construct the second database for learning periodic trajectories and transitions for a set of possible gaits for the exoskeleton.

[0037] The second database includes all transitions from a periodic basic orbit of the exoskeleton of the first training database to another periodic basic orbit of the exoskeleton of the first training database.

[0038] Each transition from a periodic basic orbit of the exoskeleton, called an initial periodic basic orbit, to another periodic basic orbit of the exoskeleton, called a final periodic basic orbit, is defined as a series of periodic basic orbits that successively include the initial periodic basic orbit, at least one intermediate periodic basic orbit and the final periodic basic orbit, and constructing the second database includes determining the at least one intermediate periodic basic orbit for each pair of initial and final periodic basic orbits of the first learning database.

[0039] Each intermediate periodic fundamental orbit between the initial periodic fundamental orbit and the final periodic fundamental orbit is a linear blend of said initial periodic fundamental orbit and the final periodic fundamental orbit.

[0040] For each pair of the initial and final periodic orbits, determining the at least one intermediate periodic basic orbit uses a so-called shortest path algorithm from the initial periodic basic orbit to the final periodic orbit in a graph of periodic basic orbits, such that the cost of passing from one periodic basic orbit to another periodic basic orbit represents a discrepancy in the dynamics of the exoskeleton.

[0041] The step (b) further comprises verifying a criterion representative of the accuracy of the predictions of the second neural network, and if the criterion is not verified, the step (b) is repeated.

[0042] According to a second aspect, the present invention relates to a method for generating a trajectory of an exoskeleton, the method comprising the steps of: (c) storing in a memory of the second server parameters of a second neural network trained using the method of training parameters of a neural network for generating a trajectory of an exoskeleton according to the first aspect; (d) generating a trajectory of the exoskeleton by a data processing means of the second server using the second neural network; Includes.

[0043] According to another advantageous, non-limiting feature, the exoskeleton is adapted to receive a human operator, and step (d) includes identifying a set of n-sets of gait parameters for the exoskeleton desired by the operator, and a trajectory of the exoskeleton is generated according to the set of n-sets of gait parameters.

[0044] The generated trajectory of the exoskeleton includes a new periodic basic trajectory and a transition to the new periodic basic trajectory for each of the set of n gait parameters.

[0045] According to a third aspect, the present invention relates to a method for operating an exoskeleton comprising a plurality of degrees of freedom, at least one of which is actuated by an actuator controlled by a data processing means, the method comprising a step (e) of executing, by the data processing means of the exoskeleton, a trajectory of the exoskeleton generated by the method for generating an exoskeleton trajectory according to the second aspect, such that the exoskeleton is able to walk.

[0046] According to a fourth aspect, the present invention relates to a system comprising a first server, a second server and an exoskeleton, each comprising data processing means configured to implement the method for learning parameters of a neural network to generate a trajectory of the exoskeleton according to the first aspect, the method for generating a trajectory of the exoskeleton according to the second aspect and / or the method for moving the exoskeleton according to the third aspect.

[0047] According to fifth and sixth aspects, the present invention relates to a computer program product comprising code instructions for performing the method of training neural network parameters for generating an exoskeleton trajectory according to the first aspect, the method for generating an exoskeleton trajectory according to the second aspect and / or the method for moving an exoskeleton according to the third aspect, and to a storage means readable by an IT device on which is stored a computer program product comprising code instructions for performing the method of training neural network parameters for generating an exoskeleton trajectory according to the first aspect, the method for generating an exoskeleton trajectory according to the second aspect and / or the method for moving an exoskeleton according to the third aspect. [Brief description of the drawings]

[0048] Other characteristics and advantages of the invention will become apparent upon reading the following description of the preferred embodiments, which is given with reference to the accompanying drawings. [Figure 1] FIG. 1 is a diagram of an architecture for implementing the method according to the invention. [Diagram 2] FIG. 2 is a diagram of an exoskeleton used by the method of the present invention. [Diagram 3] FIG. 3 is a diagram of a neural network used in the method according to the invention. [Figure 4] FIG. 4 shows a diagram of a preferred embodiment of the method according to the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0049] architecture According to three complementary aspects of the invention: a method for learning the parameters of a neural network, in particular of the FNN (Feedforward Neural Network) type, to generate a trajectory of the exoskeleton 1; - a method for generating a trajectory of an exoskeleton 1 (advantageously using a neural network trained in the manner described above); - a method for moving the exoskeleton 1 (by applying a trajectory generated by the method according to the second aspect), is proposed.

[0050] These three types of methods are implemented by a first server 10a and / or a second server 10b in the architecture shown in Fig. 1. The first server 10a is a learning server (implementing the first method) and the second server 10b is a trajectory generating server (implementing the second method). It is entirely possible for these two servers to be integrated, but in practice the first server 10a is a remote server while the second server 10b can be integrated into the exoskeleton 1 for real-time operation.

[0051] Typically, each of these servers 10a, 10b is an IT device connected to a wide area network 20, such as the Internet network, for exchanging data, but in practice, when the neural network is trained and embedded in the second server 10b, the communication can be interrupted at least intermittently. Each server includes data processing means 11a, 11b of the processor type (in particular, the data processing means 11a of the first server has high computing power, since training is longer and more complicated than the simple use of the trained neural network) and, if applicable, data storage means 12a, 12b, such as a computer memory, for example a hard drive. Furthermore, the training database is stored in the memory 12a of the first server 10b.

[0052] According to a preferred embodiment combining the three methods of the present invention, the system comprises a first server 10a connected via a network 20 to an exoskeleton 1 in which a second device 10b is embedded. - the first server 10a, where applicable, uses its computing power to construct two learning databases, then implements a method for learning parameters of a second neural network using these learning databases, and transmits the learned parameters from the second neural network to the second server 1b. The second server 10b implements a method for generating a trajectory of the exoskeleton 1 by a neural network using the parameters read from the first server 10a. -Exoskeleton 1 operates by directly applying trajectories generated on the fly.

[0053] It can be seen that there can be multiple exoskeletons 1, each with its own built-in second server 10b (which, due to limited power and space, generates trajectories only for its own dedicated exoskeleton 1), or there can be multiple exoskeletons 1, each connected to a more powerful second server 10b which can optionally be integrated with the first server 10a (and has the ability to generate trajectories on the fly for all exoskeletons 1).

[0054] With reference to Fig. 2, the term "exoskeleton 1" refers to an actuated and controlled articulated mechanical system of the bipedal robotic device type, with two legs, each lower limb of which more precisely accommodates a human operator attached (notably by a strap) to a leg of the exoskeleton 1. It can therefore be more or less a humanoid robot. Here, "walking" refers to moving the robotic device 1 in a broad sense, providing alternating support to the legs in a standing position (on flat ground, on an inclined ramp, on stairs, etc.) so as to obtain not only displacements but also movements from a sitting position to a standing position (also called "standing up") or vice versa.

[0055] The exoskeleton 1 has multiple degrees of freedom, i.e. deformable joints (typically via rotation), i.e. it is movable relative to each other, and it is either "actuated" or "unactuated".

[0056] An actuated degree of freedom denotes a joint with an actuator controlled by the data processing means 11c, i.e. this degree of freedom is controlled and can be acted upon. Conversely, a non-actuated degree of freedom is a joint without an actuator, i.e. this degree of freedom follows its own dynamics and the data processing means 11c does not exercise any direct control over it (but a priori indirect control via other actuated degrees of freedom). In the example of Fig. 1, there is a point contact between the heel and the ground, so the exoskeleton 1 is free to rotate with respect to this contact point. The angle between the heel-hip axis and the vertical therefore constitutes a non-actuated degree of freedom.

[0057] Exoskeletons of the present invention, as previously discussed, naturally include at least one actuated degree of freedom, and preferably multiple actuated degrees of freedom, and at least one degree of freedom that is preferably not actuated (i.e., is said to be "underactuated"). The number of degrees of freedom that are not actuated is referred to as underactuated degrees.

[0058] The data processing means 11c represents IT equipment (usually an external processor in case the exoskeleton 1 is "remote controlled", but preferably integrated in the exoskeleton 1) and, if applicable, integrated with the data processing means 11b of the second server 10b when integrated (as shown in Fig. 2), suitable for processing instructions and generating commands for the various actuators. The latter can be electrical, hydraulic, etc.

[0059] The exoskeleton 1 may further include a memory 12 (in particular the memory of the second server 10b, i.e. if the server 10b is not integrated the means 12 may be external) and various sensors such as means for detecting foot impacts on the ground 13, inertial measurement means 14, an operator's sensor vest 15, etc.

[0060] This application is not limited to any exoskeleton 1 architecture, examples being as described in WO 2015 / 140352 and WO 2015 / 140353.

[0061] Therefore, preferably, and according to these applications, the exoskeleton 1 includes a foot structure for each leg that includes a support plane against which the person wearing the exoskeleton rests when the foot is flat.

[0062] The support plane includes a forward platform and a rear platform, and the foot peg joint connects the forward platform to the rear platform by providing a degree of freedom that is not actuated. Such an architecture is particularly suited for "HZD" type tracks.

[0063] However, a person skilled in the art would know how to adapt the method to any other mechanical architecture.

[0064] Principles of the invention Over the past few years, there has been growing interest in artificial intelligence techniques in the field of real-time control.

[0065] Unfortunately, the current process for generating trajectories is largely incompatible with learning-by-reinforcement methods due to the lack of analytical gradients required for backpropagation of errors. Secondly, more complex methods such as Q-learning and policy gradients are resource intensive and require more experimental data, which are expensive to acquire. Additionally, these methods cannot be trained on real exoskeletons1, and therefore require simulations.

[0066] Therefore, certain studies in robotics make use of neural networks to perform planning tasks, but their use is limited to nonlinear regression, most of the time aimed at compressing databases of trajectories and thus making them easier to embed in real devices. This approach limits neural networks to the status of a "black box" tool in the framework of research focused on challenges arising from the field of traditional robotics rather than machine learning, offering few research perspectives. Moreover, the performance of these studies is tempered because trajectories are difficult data to model, the latter having a lot of irregularity and chaotic nature with respect to the input parameters.

[0067] The method elegantly overcomes these difficulties by successively training two neural networks, called the first and second neural networks, the second of which is the "output" network that allows to effectively generate the exoskeleton's trajectory.

[0068] As explained, the "trajectory" of an exoskeleton refers to the change in each degree of freedom (especially the actuation degrees of freedom) expressed as a function of a time or phase variable.

[0069] The idea is to define a "complex" orbit as a set of periodic orbits, called "elementary", that intersect with transitions. The term periodic trajectory means any trajectory that is applied (repeatedly, if applicable) over the duration of a step in such a way that it starts from an initial state of the exoskeleton 1 (at the moment of foot contact) at the beginning of the step, and returns to the same state at the beginning of the next step (as explained, this encompasses any flat walking, but also walking on a ramp, going up and down stairs, etc.). Also, periodic trajectories are said to form a "limited period", so that said periodic trajectories are applicable over any number of steps in a stable manner.

[0070] That is, each basic trajectory is associated to a given gait of the exoskeleton 1 (defined by an n set of gait parameters) and allows to maintain this gait in a stable and feasible way (i.e., as seen, respecting all the constraints of the optimization problem and minimizing the cost function as far as possible). As mentioned before, the gait parameters correspond to gait "characteristics" such as step length, step frequency and bust inclination, but also step height when negotiating stairs, instantaneous turning angle in case of bending movements, morphological characteristics of the operator (a subgroup of gait parameters called patient parameters), e.g. size, weight, femur or tibia length, position of the center of mass (value of forward shift) and lateral deviation of the bust in the framework of rehabilitation activities.

[0071] The gait "constraints" mentioned above can vary and depend on the type of gait desired, e.g., "flat-footed" gait or "HZD." The method is not limited to any type of desired gait.

[0072] A transition corresponds to a change in gait, i.e. a change in the value of said gait parameters (e.g. an increase in stride length). Knowing the initial set of gait parameters and the final set of gait parameters, and thus the initial periodic trajectory (associated with the initial set of gait parameters) and the final periodic trajectory (associated with the final set of gait parameters), said transition is a fragment of a trajectory that allows moving from the initial periodic trajectory to the final trajectory. Note that there can be "initial" or "final" transitions, e.g. standing up or sitting down movements, but we will focus on "intermediate" transitions between periodic trajectories, in that the initial transitions or the final transitions can be calculated in advance.

[0073] As can be seen, such a transition can be seen as a path in the space of periodic orbits. That is, between the initial and final periodic orbits, one or more intermediate trajectories can be planned, which in particular correspond to a "mix" between these two orbits, i.e. have intermediate walking parameters between these initial and final orbits. In practice, a set of possible intermediate periodic basic orbits (in particular all trajectories by linearly changing the walking parameters from one of the initial and final orbits to the other) can be available in advance, and the determination of the transition orbit consists in selecting a suitable succession of intermediate orbits (advantageously parameterized by a function called "progression" as shown below). The number of these intermediate orbits can be set, and it is advantageous that they are uniformly distributed, i.e. all possible intermediate orbits are consecutive and form a linear transition.

[0074] That is, instead of going from an initial to a final orbit instantly, this proceeds in small increments over successive cycles. If the transition is slow enough, the effect of the gradual change in orbit can be neglected; the state of the system effectively belongs to a finite cycle at each step.

[0075] The first neural network is capable of predicting gait, i.e. generating periodic elementary trajectories of the exoskeleton 1 for a given gait of the exoskeleton 1. That is, the first neural network takes n sets of gait parameters as input and generates corresponding periodic trajectories as output. The advantage of this is that this approach has a limited number of dimensions, thus staying within the bounds of current systems.

[0076] The second neural network is able to generate the same periodic gaits obtained by the first neural network, but is also able to predict physically allowed transitions between periodic gait cycles, i.e. generate both periodic basic trajectories and transitions, i.e. ultimately generate arbitrary trajectories, thus generalizing and replacing the first neural network.

[0077] Each of the first and second networks is preferably a network of the FNN (Feedforward Neural Network) type, i.e. information flows only in one direction (forward) from the input nodes through the hidden layers (if applicable) to the output nodes without recurrent connections, because such networks have good scalability. In particular, FNNs with two or three hidden layers and 200-300 neurons per layer can be adopted. Alternatively, Convolutional Neural Networks (CNN), LSTM (Long Short-term Memory) or GRU (Gated Recurrent Other types of neural networks, such as recurrent networks of the (Unit) type, can also be used.

[0078] It is desirable that the first and / or second neural network have the property of being continuously derivable with respect to the walking parameters. For this purpose, for example, a network using an S-shaped activation function that is continuously derivable (particularly as opposed to ReLU) is preferable.

[0079] According to a particularly preferred embodiment shown in Fig. 3, each of the first and second networks is advantageously an FNN with a final block called deconvolution (or transposed convolution) with alternating convolution and upsampling layers. More precisely, this architecture forms a structure called an autoencoder (or "encoder-decoder"), where the main FNN block plays the role of an encoder by generating a feature map of small dimensions called the initial coding sequence, and the deconvolution block plays the role of a decoder by reconstructing an output sequence that defines the predicted trajectory from the initial coding sequence. This architecture does not need to learn this zero behavior, as is the case with conventional FNNs, since it has fewer parameters to learn, since its structure naturally predicts time sequences that are correlated with each other.

[0080] How to learn According to a first aspect, a learning method is proposed which is implemented by the data processing means 11a of the first server 1a.

[0081] As illustrated in FIG. 4, the method begins with step (a) of learning parameters of a first neural network capable of generating periodic basic trajectories of the exoskeleton 1 for a given gait of the exoskeleton 1 defined by an n-set of gait parameters, according to a first database for learning periodic trajectories for a set of possible gaits of the exoskeleton 1.

[0082] Preferably, this step (a) includes constructing said first learning database, ie generating said periodic orbits which constitute it.

[0083] The idea is to cover the set of possible gaits for the exoskeleton 1, i.e., to have a large number of n-tuple sets of gait parameters, preferably as well-distributed related trajectories.

[0084] As explained, optimization tools for this are known, and in particular it is possible to generate a first database of trajectories by optimization over a compact space of a set of permissible researches consisting of various walking parameters. For example, in the case of HZD trajectories, the problem of generating trajectories is formulated in the form of an optimal control problem that can be preferably solved by a so-called direct collocation algorithm. See Omar Harib et al., "Feedback Control of an Exoskeleton For more information, please refer to "For Paraplegics for Towards Robustly Stable Hands-Free Dynamic Walking".

[0085] Advantageously, the samples are generated randomly according to a uniform, discretized law over a finite set with the same number of possible values ​​for each parameter, so as to guarantee uniform coverage of the complete space even for a small number of samples and to include its boundaries exactly. Mathematically, this means that for any given set of n parameters, there is a "closed" set of samples.

[0086] Note that this step of constructing the first learning base is long, but it can be done far upstream and the first base can be completed piecemeal. It is also possible to start from a pre-constructed base.

[0087] The training of the first network itself can proceed in a conventional manner, in particular by supervised learning, and finally, a criterion representative of the predictive accuracy of the neural network can be tested to verify the learning.

[0088] More precisely, the prediction error must be significantly smaller on average than the maximum error of the reference monitoring of the actual exoskeleton during the step, i.e. smaller than a given fraction (in particular 4%) of the predictions that have a maximum error of prediction (at infinity norm, i.e. taken over all joints and all moments as a whole) larger than a threshold such as 0.01 rad.

[0089] If this is not the case, the trajectory generation algorithm itself can be modified in a way that encourages it to generate trajectories similar to the reference (which in this case corresponds to the prediction of the neural network).

[0090] A new first database is then preferably generated by following this methodology, and the first neural network is retrained on this new basis (i.e. starting from the weights obtained at the end of the first occurrence of step (a)) without reinitializing it at the new occurrence of step (a), but rather incrementally with respect to its previous state. If the accuracy criterion of the predictions is still not verified, the process is repeated.

[0091] At the end of step (a), a first neural network is available that is able to predict periodic orbits. These periodic orbits are guaranteed to be feasible and stable, i.e., they satisfy all the constraints of the optimization problem arising from the trajectory optimization algorithm and minimize as much as possible the cost function of the original problem. As explained, extending the proposed approach to the generation of transitions is not possible because the number of dimensions to be explored would be too large.

[0092] It is understood that there exists a large number (even infinite) of feasible trajectories for a given task (i.e. satisfying the constraints). The role of the optimization process is to calculate the one that minimizes the cost selected among the latter (actually the energy). However, all of these solutions are difficult to model by neural networks, and therefore we wish to limit the optimization problem to only those trajectories that can be fully modeled. Specifically, it is a problem of consensus between all the trajectories of the database, i.e. the latter can be modeled as a set and must agree with each other to guarantee accurate and reliable predictions (i.e. for the training samples, but also for the test samples, feasible / stable with the exoskeleton 1). Since solving this problem directly is not possible due to its excessive complexity, iterative methods are useful here.

[0093] Therefore, as explained, the method includes a step (b) of learning, using parameters from the first neural network, parameters of a second neural network suitable for generating periodic basic trajectories of the exoskeleton 1 and transitions from one periodic basic trajectory of the exoskeleton 1 to another periodic basic trajectory of the exoskeleton 1 according to a second learning database of periodic basic trajectories and transitions for a set of possible gaits of the exoskeleton 1.

[0094] Preferably, the second neural network and the first neural network have the same architecture as described (e.g., the architecture of FIG. 3), but the second neural network includes a larger number of parameters (e.g., by having more parameters per layer in the case of an FNN or by having more channels in the case of a convolutional network, for example) since additional parameters can be learned during step (b). Optionally, the common parameters can be re-learned in step (b) by starting from the values ​​obtained in step (a). Alternatively, the second neural network can correspond to the first neural network, with an additional layer relative to the first network, whose parameters are learned during step (b).

[0095] As in step (a), it will be appreciated that step (b) can involve the construction of said second training database, i.e. generating the periodic orbits and transitions which constitute it, advantageously from said first database. More specifically, all transitions are constructed from so-called initial periodic orbits of the first training database to other so-called final periodic orbits of the first training database. Again, the second training database can be constructed further upstream, completed piece by piece and / or pre-constructed.

[0096] As explained, the transitions can be considered as paths from the first orbit to the last orbit in the space of periodic orbits, and the second base can include a transition from one to the other for each pair of initial and final periodic basic orbits of the first base (i.e., for each pair of n walking parameters specifying the initial orbit and the final orbit of the other). Advantageously, all possible transitions are considered to be equally probable, regardless of the amplitude of the transition.

[0097] In fact, we can propose a heuristic method based solely on periodic gait prediction. The approach is as follows.

[0098] 1) Generate a lattice of periodic orbits ranging from an initial periodic orbit to a final orbit (with as many intermediate orbits as possible) by linearly varying the walking parameters. These intermediate orbits correspond to a linear blend between the two initial and final orbits. As explained, the number of intermediate orbits can be set and they are advantageously uniformly distributed.

[0099] 2) A transition corresponds to any path that passes through this lattice from the top left corner (i.e. the initial cyclic orbit at the start) to the bottom right corner (the final orbit at the end). The transition that is considered optimal among the infinite possible options can be calculated using so-called shortest path algorithms under constraints. By considering the grid as an undirected graph, i.e. the vertices of the graph are cyclic elementary orbits (initial, final, intermediate), the cost associated with each transition (movement of an edge, i.e. a change in trajectory) from one vertex to another corresponds to a "discrepancy" in the dynamics due to the transition from one cyclic walk to another, and the objective is to minimize the "conflict" of transitions over the entire trajectory. More precisely, it is desirable to avoid changes in the trajectory that may have an excessively large effect during the transition period with the risk of introducing imbalances. This discrepancy corresponds mathematically to a violation of the dynamics equations along the transition trajectory. For example, the residual error between the numerical derivatives calculated from the transition trajectory and the expected terms according to the dynamics equations can be used. In fact, as explained, the movement in the lattice can be parameterized by a function called "progression" that defines the gradual transition between the initial and final periodic orbits, and this function can be parameterized, for example, by a polynomial whose coefficients are optimized to minimize discrepancies and respect edge conditions as a result of said shortest path algorithm.

[0100] Training of the second network itself can proceed by minimizing the cost function, again checking at the end the criterion describing the predictive accuracy of the neural network to verify training.

[0101] At the end of step (b), a first neural network capable of predicting complex trajectories (including transitions) is available. The second neural network (and more specifically its learning parameters) can then be embedded in a second server 10b (preferably of the exoskeleton 1) in step (c).

[0102] How to generate a trajectory According to the second aspect, a method for generating a trajectory of a trajectory of an exoskeleton 1 is proposed, which is implemented by the data processing means 11b of the second server 1b. The method for generating a trajectory of a trajectory of an exoskeleton 1 follows the implementation of the method for training a second neural network according to the first aspect. More specifically, it starts with a step (c) of storing in the memory 12 of the second server 10b the parameters of the second neural network trained by the method for training parameters of a neural network for generating a trajectory of the exoskeleton 1.

[0103] As explained, the second neural network can be generated well in advance and built into many exoskeletons 1. It is preferably performed in real time.

[0104] The second method may be performed periodically over the operation period of the exoskeleton 1 and may, in particular, optionally comprise in step (d) of generating a trajectory of the exoskeleton 1 by the data processing means 11b of the second server 10b by using the second neural network, repeatedly calling the second neural network in such a way as to obtain periodic trajectories and / or transitions.

[0105] More specifically, a sequence of n-sets of walking parameters is assumed that are incrementally obtained (e.g., due to new commands from the exoskeleton's operator). For each new n-set of parameters, the second neural network determines a new periodic orbit and a transition towards this new periodic orbit.

[0106] To that end, the method of generating a trajectory advantageously involves determining (and, if applicable, regularly repeating) n sets of gait parameters for the exoskeleton 1.

[0107] Indeed, if exoskeleton 1 is an exoskeleton that accepts a human operator, it is the pose (and optionally button presses) of the human operator that determines the parameters (unlike in the case of a normal robot, which can directly receive start-up requests, including setpoints for walking speed and / or direction).

[0108] To that end, the operator may be provided with a sensor vest 15 making it possible to detect the configuration of their bust (the orientation of the latter) as explained. The direction in which the operator turns their bust is the direction in which they want to walk, and the speed is given by the strength with which they bring their bust forward (the extent to which they lie down). The start-up request corresponds to the pressing of a button (or a specific posture) by the operator to communicate the intention to start walking and to command the data processing means to determine said parameters. Some parameters, such as the instantaneous angle of rotation or the step height in case of negotiating stairs, may be predetermined or obtained by other sensors 13, 14.

[0109] According to a third aspect, a method of operating an exoskeleton 1 is proposed, comprising implementing the method according to the second aspect (steps (a), (b), (c), (d)) for generating a trajectory of the exoskeleton, and then (in a step referred to as (e)), executing said trajectory so that the exoskeleton 1 walks.

[0110] Steps (d) and (e) can be repeated in such a way that the trajectory of the exoskeleton 1 is constantly corrected in real-time.

[0111] Equipment and Systems According to a fourth aspect, the invention relates to a system for the implementation of the method according to the first and second and / or third aspects.

[0112] As described, the system includes a first server 10a, a second server 10b and a possibly integrated exoskeleton 1.

[0113] The first server 10a comprises data processing means 11a for the implementation of the method according to the first aspect.

[0114] The second server 10b generally comprises data processing means 11b for implementing the method according to the second embodiment and data storage means 12 for storing the second neural network (or its learning parameters).

[0115] The exoskeleton 1 comprises data processing means 11c configured for the implementation of the method according to the third aspect and, optionally, data storage means 12 (in particular the thing of the second server 10b), inertial measurement means 14 (inertial measurement means), means for detecting the impact of the foot on the ground 13 (contact sensors or any pressure sensors) and / or a sensor vest 15.

[0116] It has multiple degrees of freedom, at least one of which is actuated by an actuator controlled by the data processing means 11c within the framework of implementing the method according to the third aspect.

[0117] Computer Program Products According to a fifth and sixth aspect, the invention relates to a computer program product comprising code instructions for the execution (on a processing means 11a, 11b, 11c) of the method according to the first aspect for learning parameters of a neural network, the method according to the second aspect for generating a trajectory of an exoskeleton 1 and / or the method according to the third aspect for moving an exoskeleton 1 and storage means readable by an IT device in which this computer program product is found.

Claims

1. 1. A method of learning parameters of a neural network for generating a trajectory of an exoskeleton, the method comprising: (a) learning parameters of a first neural network suitable for generating periodic basic trajectories of the exoskeleton according to a first training database for learning periodic trajectories for a set of possible gaits of the exoskeleton, each of the periodic basic trajectories being for a given gait of the exoskeleton defined by n sets of gait parameters; (b) using parameters from the first neural network, learning parameters of a second neural network suitable for generating periodic basic trajectories of the exoskeleton and transitions from one periodic basic trajectory of the exoskeleton to another periodic basic trajectory of the exoskeleton according to a second learning database of periodic basic trajectories and transitions for a set of possible gaits of the exoskeleton; The method of claim 1,

2. 2. The method of claim 1 , wherein step (a) comprises constructing the first training database of periodic trajectories for a set of possible gaits of the exoskeleton by using an optimization algorithm.

3. The method of claim 2 , wherein a set of possible gaits for the exoskeleton is selected such that a space in which the n-set of gait parameters have values ​​is uniformly covered.

4. 4. The method of claim 2 or 3, wherein step (a) further comprises verifying a criterion representative of the accuracy of the predictions of the first neural network, and if the criterion is not verified, step (a) is repeated.

5. 5. The method of claim 1, wherein step (b) comprises using the first training database to construct the second training database for learning periodic trajectories and transitions for a set of possible gaits for the exoskeleton.

6. 6. The method of claim 5, wherein the second training database includes all transitions from a periodic basic orbit of the exoskeleton of the first training database to another periodic basic orbit of the exoskeleton of the first training database.

7. 7. The method of claim 6, wherein each transition from a periodic basic orbit of the exoskeleton, called an initial periodic basic orbit, to another periodic basic orbit of the exoskeleton, called a final periodic basic orbit, is defined as a series of periodic basic orbits that successively include the initial periodic basic orbit, at least one intermediate periodic basic orbit and the final periodic basic orbit, and constructing the second learning database includes determining the at least one intermediate periodic basic orbit for each pair of initial and final periodic basic orbits of the first learning database.

8. The method of claim 7 , wherein each intermediate periodic fundamental orbit between the initial periodic fundamental orbit and the final periodic fundamental orbit is a linear blend of the initial periodic fundamental orbit and the final periodic fundamental orbit.

9. 9. The method of claim 7 or 8, wherein for each pair of the initial and final periodic orbits, determining the at least one intermediate periodic basic orbit uses a so-called shortest path algorithm from the initial periodic basic orbit to the final periodic orbit in a graph of periodic basic orbits, such that the cost of passing from one periodic basic orbit to another periodic basic orbit represents a discrepancy in the dynamics of the exoskeleton.

10. 10. The method of claim 5, wherein step (b) further comprises verifying a criterion representative of the accuracy of the predictions of the second neural network, and wherein if the criterion is not verified, step (b) is repeated.

11. 1. A method for generating a trajectory for an exoskeleton, comprising: (c) storing in a memory of a second server parameters of a second neural network trained using the method for training parameters of a neural network for generating a trajectory of an exoskeleton according to any one of claims 1 to 10; and (d) generating a trajectory of the exoskeleton by a data processing means of the second server using the second neural network; A method comprising:

12. 12. The method of claim 11 , wherein the exoskeleton accepts a human operator, and wherein step (d) includes identifying a set of n-set gait parameters for the exoskeleton desired by the operator, and wherein a trajectory of the exoskeleton is generated in accordance with the set of n-set gait parameters.

13. 13. The method of claim 12, wherein the generated trajectory of the exoskeleton includes a new periodic base trajectory and a transition to the new periodic base trajectory for each of the set of n gait parameters.

14. 14. A method for moving an exoskeleton comprising a plurality of degrees of freedom, at least one of which is actuated by an actuator controlled by a data processing means, the method comprising the step (e) of executing, by the data processing means of the exoskeleton, a trajectory of the exoskeleton generated by the method for generating an exoskeleton trajectory according to any one of claims 11 to 13, so as to enable the exoskeleton to walk.

15. 15. A system comprising a first server, a second server and an exoskeleton, each comprising a data processing means, the data processing means configured to perform the method for learning parameters of a neural network for generating a trajectory of the exoskeleton according to any one of claims 1 to 10, the method for generating a trajectory of an exoskeleton according to any one of claims 11 to 13 and / or the method for moving an exoskeleton according to claim 14.

16. 15. A computer program comprising code instructions for carrying out the method for learning parameters of a neural network for generating a trajectory of an exoskeleton according to any one of claims 1 to 10, the method for generating a trajectory of an exoskeleton according to any one of claims 11 to 13 and / or the method for moving an exoskeleton according to claim 14, said program being executed on a computer.

17. 15. A storage means readable by an IT device, storing a computer program comprising code instructions for executing the method for learning parameters of a neural network for generating a trajectory of an exoskeleton according to any one of claims 1 to 10, the method for generating a trajectory of an exoskeleton according to any one of claims 11 to 13 and / or the method for moving an exoskeleton according to claim 14.

Images