Articulated robotic arm device comprising a system for controlling movement of the arm segments by means of a neural network

Abstract

The invention relates to an articulated robotic arm device with a neural-network system for movement of a robotic arm, the neural-network system being capable of learning online and in real time the forces for moving its joints towards a target position by simulation of virtual springs implementing impedance control, wherein learning the rest elongations of the virtual springs for a selected state defines the equilibrium position of the arm corresponding to a zero resultant of the forces applied to the arm, thereby giving the arm elasticity properties as a function of the stiffness selected for the springs, at equilibrium, wherein the neural network learns these elongations for each transition from one target position to another target position in the joint space, making it possible to control movement of the arm in order to learn a sequence of movements and define the behavior of the arm for a given task, each joint of the arm having a target position.

Description

[0001] Articulated robotic arm device incorporating a neural network control system for arm segment movement

[0002] technical field

[0003] The technical context of the present invention is that of the control of multi-articulated robotic arms. More particularly, the invention relates to a control system for a multi-articulated robotic arm using techniques derived from artificial neural networks.

[0004] State of the art

[0005] In the state of the art, several servo control and control techniques for a multi-articulated robotic arm are known, among which we distinguish in particular

[0006] - The "classic" control of robotic arms in position and speed very often uses PID (Proportional, Integral, and Derivative) controllers cascaded one after the other. In particular, the use of such PID controllers is known, with a first PID controller configured to control the robotic arm's position and a second PID controller configured to control the robotic arm's speed along the desired trajectory. These control techniques are very effective and ensure that the robotic arm's inertia respects the usage constraints in terms of speed during an impact with a human, for example. However, the control is not elastic. The motor torque is not directly controlled, which can be dangerous in the case of physical interaction with humans. Such control systems must therefore be equipped with numerous sensors, making them complex and expensive.

[0007] - An approach known as optimal control aims to implement a dynamic model of the robotic arm and optimize the control law to guarantee a set of properties along the desired trajectory. This approach is clearly the most powerful but requires a powerful computer and, above all, that the dynamic model of the robotic arm does not change too rapidly during its use. For example, a change in weight at the end effector of the robotic arm must be known or quickly estimated to ensure continued optimal operation. Furthermore, such approaches are not easily deployed in open environments where the conditions of use or interaction with the robotic arm are not perfectly controlled and constant.

[0008] In the field of mechanical compliance, numerous torque limiting devices are also known, such as those using elastic mechanical parts or springs, or strain gauges combined with impedance control to regulate the forces exerted by and on each joint of the robotic arm. The objective of these devices is to allow the robotic arm to adjust to the stresses and forces acting upon it, and thus correct the positioning and orientation errors resulting from these stresses and forces.

[0009] Indeed, in the pursuit of this objective and to function effectively, taking into account the significant inertia of a robotic arm is essential to maintain its compliance and to be able to use it in a semi-passive mode or to allow an operator to interrupt the movement of the robotic arm by obstructing its movement.

[0010] As is well known, the least computationally expensive solutions use cascades of PID controllers to control the position, speed, and force of all or part of the robotic arm's joints. These technologies have subsequently been adapted for controlling collaborative robots.

[0011] Invention

[0012] The present invention aims to provide a new control system for a robotic arm in order to address at least largely the previous problems and to lead to other advantages.

[0013] Another objective of the invention is to learn to control a multi-articulated robotic arm in real time and by force. The invention allows for the pre-calculation of solutions, which are then injected and adjusted according to the current state so as to solve the problem without delay (the delay only occurs during learning and / or if significant adaptation is required) and with the least possible effort.

[0014] Another objective of the invention is to limit the effort required by the robotic arm in order to use it as a collaborative robot. A further objective is to provide a robotic arm control system that is less complex and more economical than those currently available, notably by avoiding the use of force sensors.

[0015] The present invention describes an articulated robotic arm device comprising a neural network control system for the movement of segments connected by joints i, capable of learning online and in real time its efforts to move the joints i towards one or more target positions which are arrival and / or passage positions of the robotic arm, thanks to the simulation of virtual springs realizing the elasticity of each joint i by impedance control.

[0016] The device includes for each joint of the robotic arm a motor, a reducer and a 0mi joint position sensor.

[0017] The control system comprises a low-level controller commanding the movement of the robotic arm joints and a high-level controller driving the low-level controller, which is the neural network calculating, throughout the movement, for each joint i, a current resting elongation sent to the low-level controller.

[0018] the low-level controller simulating for each joint i, at least one main virtual spring ri of stiffness Kn and current rest elongation 0ori, the low-level controller calculating, in real time, a control command Ti which includes at least one main sub-command to control the elongation of the main virtual spring r: Kn x (0OH - 0mi) , which is sent to the actuator associated with the joint i.

[0019] And for the high-level controller, learning the resting elongations of the virtual springs for a chosen state, which represents a joint configuration of each segment of the robotic arm, defines the equilibrium position at the target position of the arm corresponding to a zero resultant of the forces applied to the arm, giving it properties of elasticity or compliance depending on the stiffness chosen for the springs,

[0020] at equilibrium in the target position, the forces exerted by the virtual springs are calculated so that the resultant of all applied forces is zero (and the spring forces are therefore minimal, thus reducing the arm's power consumption compared to a PID type control),

[0021] This learning process is carried out in order to achieve this balance of forces at the target position using a learning control signal corresponding to the error between the positions measured by the position sensor and the target position.

[0022] Learning by the neural network of these elongations for each transition from one target position to another target position in the joint space makes it possible to control the movements of the robotic arm to learn a sequence of movements and define the behavior of the robotic arm for a given task, each joint of the robotic arm having a target position.

[0023] For a given starting point (the previous point on the trajectory), reaching the next point requires finding the resting elongations of the virtual springs for the state Ej characterized by the transition from the previous target position to the current target position, which we will call a transition. For example, if we want to go from A to B, we must activate a state associated with the transition AB, which can be seen as the activity of a neuron recognizing this transition. If the trajectory is an infinite loop, with the transitions being unique, we can return to states associated with the target positions (and the notion of a transient state can be neglected since there is no alternative).

[0024] In other words, the invention makes it possible, through the simulation of virtual springs, to define an elasticity for each joint i within the framework of an impedance control.

[0025] This control relies on learning the resting elongations of the virtual springs of each joint to reach a target position corresponding to an equilibrium position where the resultant of the applied forces is zero. This learning takes place in real time as the robot moves.

[0026] The target position can be either a destination position or a transit position. When it is a transit position, time and / or velocity conditions are given so that the resting elongation calculation allows the robot to approach or reach equilibrium at the target transit position. The controller uses the associated force to reach this target transit position while maintaining a certain velocity.

[0027] Once the target position is reached, the calculation switches to calculating the resting elongation for the next target arrival position, thus moving the robotic arm from the transit position towards the final target arrival position. This control is achieved through the definition of transient states, which serve as inputs for the neural network responsible for learning the elongations.

[0028] This learning is achieved by a reinforcement signal based on the difference between the current measured position of each joint and each of the target arm positions approached or reached one after the other.

[0029] This learning by the neural network of these elongations for each transition from one target position to another target position in the joint space makes it possible to control the movements of the robotic arm to learn a sequence of movements and define the behavior of the robotic arm for a given task, each joint i of the robotic arm having a target position.

[0030] This learning process allows the system to be given elastic properties depending on the stiffness that will be chosen for the virtual springs.

[0031] Preferably, the device according to the invention does not include an additional system providing the elasticity of the robotic arm. In other words, the articulated robotic arm device is free of any system additional to the virtual springs that provides the elasticity of the robotic arm. Advantageously, the robotic arm device comprises:

[0032] - joints i connecting different segments of the robotic arm, each joint i being associated with one or more degrees of freedom, with i being natural numbers, each degree of freedom being directly or indirectly associated with one or more virtual springs whose elongations control the position of the degree of freedom in question and ultimately the position of the robotic arm. Control can be achieved degree of freedom by degree of freedom or by directly connecting effectors to the segments; for example, the control of our shoulder involves a large number of muscles that do not "connect" to the degrees of freedom (abstraction);

[0033] - at least one actuator associated with each joint i and controlling the elongations of the corresponding virtual spring by force or torque;

[0034] - at least one position sensor associated with each joint i in order to periodically measure the measured position 0mi (u) of the corresponding joint i, with a given first fixed period T1, so as to simulate the dynamics of the virtual springs

[0035] - a human-machine interface (HMI), configured in particular to communicate target positions, and to emit signals transmitted to a low-level controller and a high-level controller, in order to control the trajectory of the robotic arm. The target positions are the arrival positions of the robotic arm.

[0036] - computing and storage units connected to actuators, position sensors and the human-machine interface.

[0037] The computing and storage units include the low-level controller and the high-level controller.

[0038] The low-level controller controls the movement of the robotic arm and is driven by the high-level controller, the low-level controller simulating for each joint i, at least one main virtual spring ri of stiffness Kn and current rest elongation 0Ori.

[0039] More specifically, the low-level controller calculates, in real time, at each first period T1, for each joint i and from the positions measured 0m i (u) by the position sensors associated with the joints i, a control command n (u) which includes at least one main sub-command controlling the elongation of the main virtual spring r: Kri x (0Ori - 9mi(u)), which is sent to the actuator associated with the joint i, in order to reach, from a starting position, a target position 0i_target, for each joint i, thanks to a current rest elongation 0Ori / Ej calculated by the high-level controller,

[0040] The high-level controller produces, in real time, with a given fixed second period T2, output values ​​between 0 and 1 multiplied by a maximum elongation L r _max allowed for each virtual spring r, which gives the current rest elongations 0OH / Ej (n) of the virtual springs, the period T2 being an integer multiple of T1, to reach the target position 0i_cibie, for each joint i, to an accuracy of V. The high-level controller has, as input data at each iteration:

[0041] a given stiffness of each spring ri,

[0042] The user instructions are:

[0043] the target positions 0i_cibie / Ej, each associated with at least one state Ej of a collection of states defined according to the given stiffness of each spring ri,

[0044] a state Ej corresponding to a transition between a chosen starting position and a target position;

[0045] the measured positions 0mi (n-1 ) at each period T2 for each joint i;

[0046] a learning control signal corresponding to the error between the measured positions 0mi (n-1) and the target positions 0i_target / Ej.

[0047] The input signal is calculated based on the user's instructions and the actual arm position. It is used to modify the synaptic weights of the neural network and thus propose new elongations for the current state.

[0048] The low-level controller calculates, in real time, at each first period T1, for each joint i and from the positions measured 0mi (u) by the position sensors associated with each joint i, a control command Ti (u) which includes at least one main sub-command for controlling the elongation of the main virtual spring r: Kn x (Son - 0mi(u)) , which is sent to the actuator associated with the joint i, in order to reach, from a starting position, a target position 0i_cibie , for each joint i, thanks to the rest elongation 0on / Ej calculated by the high-level controller.

[0049] The choice of T2 being greater than T1 ensures a smooth simulation of the dynamic spring simulation for arm control without overly constraining the speed of the neural network which can update the rest elongations of the springs with a much lower frequency (typically ten times slower: 1ms for T1 and 10ms for T2), which ensures smooth control of the different degrees of freedom.

[0050] T1 can for example be less than or equal to 10ms, advantageously equal to or less than 1ms, and / or T2 can for example be less than or equal to 50ms, advantageously equal to or less than 10ms.

[0051] The main subcommand of the low-level controller is Kn. (©OH / EJ (n) - 0mi(u)). This subcommand allows for the real-time movement of joints i during a subsequent displacement from the starting position to the target position 0i_cibie / Ej, with a precision of V. During the initial displacement, the neural network learned, for each joint i, a current resting elongation 0on / Ej from the starting position to the target position 0i_cibie / Ej, with a precision of V. The neural network is capable of modifying 0ori / Ej, based on the reinforcement signal every n, to adjust the control of 0on / Ej (n) to reach the target position 0i_cibie / Ej during this subsequent displacement from the starting position.

[0052] Advantageously, the neural network learns in real time to reach the target positions 0i_cibie / Ej without taking into account as input data: the weight of the different segments of the arm, the weight of the objects carried by the robot, the weight of the effectors or tools, and without taking into account their dimensions, allowing the controller to adapt to the variations of all these weights and lengths by calculating the current resting elongation 0Ori.

[0053] The human-machine interface (HMI) can emit signals to change the spring stiffness values ​​Kn of the joints i of the robotic arm, thus offering several operating modes for the robotic arm controlled by the control system. Once the control system is trained, each operating mode has a set of associated states Ej specific to the chosen spring stiffness values ​​Kn—for each joint—and the corresponding target positions. Conversely, the control system according to the invention can present virtual springs r with spring stiffnesses K r fixed, that is to say whose value is constant and invariant during the use of the robotic arm.

[0054] In the context of the present invention, a neural network is defined as a mathematical model used to define control laws for the robotic arm through iterative and statistical calculation. The control system according to the invention comprises one or more neural networks operating in parallel. The control system according to the invention comprises one or more neural networks used sequentially and / or in parallel to control the robotic arm. In the context of the present invention, the high-level controller is defined as comprising, for example, an electronic board with computing means, such as microprocessors, and / or storage means, such as RAM or ROM, enabling the deployment of the neural network.Typically, the resting elongations of the virtual springs of each joint are obtained through successive iterations and / or iterative calculation by the neural network and correspond to the neuronal activity outputs of the network. These neuronal activities are all between 0 and 1. To obtain an elongation value usable by the low-level controller, these activities are multiplied by the maximum allowed elongation L. r _max for each virtual spring r, for each joint i.

[0055] In the context of the present invention, the low-level controller is defined as a control unit for the actuators, interacting with the neural network. More generally, the low-level controller is configured to perform proportional control of the elongations sent as commands by the neural networks. In the event of disconnections with the neural networks, the low-level controller maintains the last received elongation command for the different degrees of freedom of the robotic arm. Therefore, a force applied to the arm is sufficient to momentarily move it away from its equilibrium position. By way of non-limiting example, the low-level controller comprises an electronic board interfacing the actuators and the computing unit hosting the neural network. In the context of the present invention, the low-level controller and the high-level controller may be separate or integrated into the same electronic board.

[0056] In the context of the present invention, the robotic arm is defined as a multi-articulated arm in which each segment is connected to at least one directly adjacent segment by a motorized joint.

[0057] The robotic arm can be of the serial type, but not exclusively.

[0058] In general, the control method according to the invention is applicable to any robotic device designed to control the position and force of a mechanical system. By way of non-limiting example, the robotic arm comprises six degrees of freedom arranged in series.

[0059] The robotic arm can be used in all types of applications and industrial fields.

[0060] For example, the robotic arm may be of the type of a serial robotic arm used in "pick and place" tasks, cutting, welding, polishing... requiring greater or lesser precision in controlling the position and / or speed of movement of the end of the robotic arm.

[0061] In addition, the robotic arm can be of the type of an electrically, pneumatically, or hydraulically actuated arm.

[0062] In the context of the present invention, a joint is defined as a connection between two directly adjacent segments of the robotic arm. A joint thus provides at least one degree of freedom to the segment to which it is attached. Each joint is represented by the index i in the terms and equations below.

[0063] In the context of the present invention, an actuator is defined as a motor that controls each joint and its elongation, that is, its movement relative to its degree(s) of freedom. In the context of the present invention, a virtual spring is defined as a mathematical model simulating the behavior of a joint of the robotic arm, using biomimicry. In the terms and equation below, each spring is represented by the variable r. In particular, the present invention defines primary virtual springs and secondary virtual springs, corresponding respectively to simplified or more complex models of the joint and its control.

[0064] In the context of the present invention, stiffness is defined as a variable representing the rigidity of a given joint, that is, the rigidity of the virtual spring representing the corresponding actuator. In the terms and equations below, stiffness is denoted K, indexed by the variable r to indicate the spring to which it is associated.

[0065] In the context of the present invention, elongation, denoted 0, is defined as representing the deformation - by axial or helical elongation for example - of the virtual spring representing the joint and the actuator.

[0066] In the present invention, the elongation of a given virtual spring is the basis of the control law of the corresponding actuators.

[0067] In the terms and equations below, elongation is indexed by i to indicate the joint to which it refers.

[0068] In particular, in the present invention, the following are defined:

[0069] - the word compliance which is the robot's ability to adapt the rigidity of its movements to take into account the forces applied to it; compliance brings flexibility, elasticity to robotic arms to interact more easily in an environment with humans or for tasks requiring flexibility to cushion the forces on a mechanical part for example, or to manipulate fragile objects like an apple; a humanoid robot in a house for example must be compliant in order to interact safely with humans; unlike industrial robots (robotic arms) which are very rigid and therefore very dangerous;

[0070] - The word "current" applied to elongation means elongation calculated or updated during the current movement period; the elongation can be learned, but even when learned, the neural network is constantly updating the elongation during the movement based on what it has learned and the situation (efforts, applied forces, etc.) of the current movement; the period is defined by the update frequency of the different time constants used; for torque calculation, it is the low-level controller period; for learning the elongation, it is the period of the neural network; for states, it is the human-machine interface that imposes the states.

[0071] - Self as being the unloaded elongation of the virtual spring associated with a joint i, corresponding to the elongation determined during learning to obtain a given position of the robotic arm, for each joint.

[0072] - 0mi as being the measured position of joint i, at each instant, by a sensor of a given joint.

[0073] - 0cibie as being the target position, the one we aim to reach for a segment of the robot, that is to say for a given joint; in this case, it is a target position to be reached, that is to say a position towards which we want the joint to move.

[0074] We will distinguish two cases for 0 c ibie:

[0075] - 0 target equilibrium as being the target equilibrium position, the one that we aim to achieve for a segment of the robot, that is to say for a given joint; in this case, it is a target equilibrium position to be achieved, that is to say a position in which the joint will be maintained in a static equilibrium position for a given duration.

[0076] - ©transitory as being the transitional position for a joint of the robotic arm, that is to say a transitional / and temporary transitional position from one position - or set of joint positions - to another without stopping, unlike an equilibrium position (useful for controlling the arm on a specific trajectory without stopping at intermediate points).

[0077] - ©ori / Ej as the learned, or currently being learned or adjusted, resting elongation for a joint i, for a spring r; when learned, it is obtained at the end of a sequence of iterations during the initial learning of the robotic arm for a state Ej; this resting elongation at the end of a sequence to move from A to B is the current command that the low-level controller will use when the robot arm needs to move from A to B again during a second movement. It corresponds to the output of the neural network that will be calculated by the activation of neuron j, enabling the AB transition. We will call it transition Ej. It constitutes the current context defining the action of moving from A to B for a given trajectory from a given starting position and within a given context of applied forces.

[0078] This elongation ©OH / EJ can be seen as a value which, once learned, becomes "approximately constant" and on which the robot relies throughout the second movement from A to B, knowing that this elongation ©On / Ej can again vary, most often marginally compared to its learned value during the first movement from A to B, because the neural network is constantly learning so that the reinforcement signal between the current measured position and the target position decreases constantly during this new movement.

[0079] The EJs are the labels associated with the sequence of positions to be reached.

[0080] If we return to B several times from the same starting position and the same constraints, it is the same Ej that must be used.

[0081] If we return to point B multiple times from different positions / trajectories (with a different past than what has been learned), we must use a different Ej each time. Indeed, depending on the constraint on the starting position, as well as whether or not we stop at B, we will need to use a different Ej in order to learn different elongations so as to maintain inertia during the transition to B (transient) or, conversely, to stop at B.

[0082] If there is an unforeseen event on the path from A to B then the neural network can significantly change 0Ori / Ej current to take into account this new constraint, knowing that what always matters is that this change respects the balance of forces applied to each joint of the robot.

[0083] The elongations can be angular (e.g., electric motor or rotary hydraulic or pneumatic cylinder) or linear (e.g., hydraulic or pneumatic cylinder or other). For greater control, a pair of virtual springs can be used to simulate antagonistic muscles to allow symmetrical movements relative to the rest position. In the context of the present invention, a state E of the robotic arm is defined, representing a joint configuration of the robotic arm or one of its segments. The state is indexed to indicate one of the states—that is, one of the joint configurations—chosen from among all the possible states of the robotic arm—that is, from among all the joint configurations that the robotic arm can assume within its workspace.

[0084] In the context of the present invention, an equilibrium position or state of equilibrium is defined as a position where the sum of the external forces applied to the robotic arm and / or to the joint i under consideration is zero. Conversely, a transient position or state of transience is defined as a position where the sum of the external forces applied to the robotic arm and / or to the joint i under consideration is not zero.

[0085] In the context of the present invention, a reinforcement signal is defined, obtained from the difference in elongation - for a given joint - between that measured and the desired elongation, and allowing the neural network to be fed for the next iteration.

[0086] In the context of the present invention, the speed of a joint is defined as the speed of the corresponding actuator as measured by the associated sensor. This can be a linear speed, for example, in the case of a cylinder-type actuator, or a rotational speed, in the case of a pivoting actuator. The present invention proposes to apply similar principles to the control system of the robotic arm according to the first aspect of the invention. In particular, to provide a certain degree of elasticity to the robotic arm, the invention employs various technical features:

[0087] - a model of the muscle viewed as a spring whose elongation can be controlled, possibly associated with a parallel damper to limit its speed. Each joint is thus modeled by such a model and controlled via it;

[0088] - a learning inspired by cortical control for the creation of states and basal ganglia for reinforcement learning of the elongation to be used; - generalization capabilities of a control inspired by cortico-striatal loops to improve the generalization capabilities of the robotic arm, by interpolation between different learned states for example, during movement in manual control or when speed control is required.

[0089] Thus, the robotic arm control system according to the invention simulates the presence of springs at each joint, with variable stiffness, to obtain a compliant robotic arm without using force sensors to control the arm's ability to be pushed back by an operator. This allows for control over the "flexibility" of the control, enabling an operator to easily push back the arm with low stiffness, or conversely, ensuring high precision and "hardness" of the control with high stiffness once the trajectory has been learned at the selected stiffness.

[0090] We can increase the maximum elongation to allow for a reduction in the minimum stiffness Kri_min. Indeed, neurons have a bounded activity between 0 and 1. To obtain the elongations, we multiply this value by a constant which is the maximum elongation (increasing this constant thus allows us to achieve a greater maximum force or torque).

[0091] The control system conforming to the first aspect of the invention solves the technical problems mentioned above by ensuring a predetermined stiffness of the robotic arm in all circumstances and without the need for additional instrumentation, thus reducing its complexity, costs and risks of malfunction.

[0092] The control system according to the invention thus makes it possible to use robotic arms equipped with reversible end effectors (e.g., reversible geared motor systems) as collaborative robots, that is, to make them less dangerous in the event of physical contact with a human in an unforeseen or undesired interaction. More specifically, the control system according to the invention provides torque control of the different degrees of freedom of the robotic arm in order to:

[0093] - Increase safety by limiting the effort required if the robotic arm, controlled by the control system according to the invention, touches a human or another object in its workspace; - Enable interactions between the robotic arm and a contact surface in cases where said contact surface is not known beforehand or not perfectly known. Thus, if, for example, the contact surface is closer than expected, then the robotic arm will exert a higher torque than expected, but this will still be limited by the control system, unlike a robotic arm controlled by a cascaded PID controller that regulates the position and speed of the arm's end; - Limit the energy consumption of the robotic arm by using minimal effort to achieve a given effect.

[0094] The control system according to the invention makes it possible to control and obtain a compliant robotic arm—making it suitable for collaborative use—when the robotic arm's stiffness is not too high; but also to obtain and control the robotic arm much more precisely in other circumstances by imposing a high stiffness. To this end, the neural network of the control system is configured to learn—in either of the aforementioned cases—to find the correct elongation of the corresponding joints for a given stiffness and mass.

[0095] The reinforcement learning mechanism for learning to control the robotic arm is particularly innovative compared to other known robotic servo systems because:

[0096] - it allows for low-precision learning of the elongation associated with the desired state considered as an equilibrium point;

[0097] - it allows learning of the minimum force required to initiate a minimum movement capable of overcoming dry friction forces and avoiding each time a long integration time, as would be the case with a proportional and integral (PI) controller.

[0098] The control system conforming to the first aspect of the invention further comprises the following capabilities, which will be described in more detail in the following paragraphs, each of these technical characteristics offering superior advantages to previously known servo technologies:

[0099] - The control system according to the invention allows for reinforcement learning of the command without an explicit dynamic model of the robotic arm; reinforcement learning of the elongation and its adaptation implicitly takes into account the dynamics of the robotic arm during learning; the neural network discovers for itself the forces to be applied to each joint to go from one given position to another; in particular, depending on the starting point considered, these forces to be applied - i.e. the control of each joint - can be very different; for example, a zero force is sufficient to bring the robotic arm down to the desired position - due to Earth's gravity - whereas a high force will be required to bring it up to this position if it starts from a lower position;

[0100] In practical terms, the neural network learns in real time to reach the target positions 0i_cibie / Ej without explicitly knowing the weight of the different arm segments, the weight of the objects carried by the robot, the weight of the end effectors or tools, or the dimensions and geometry of the arm, such as the length and width of the segments. Real-time learning allows the controller to adapt to variations in all these weights. Therefore, the present invention does not require an inverse dynamics model that explicitly takes this data (weight, positioning, inertia, and geometry) into account, allowing the controller to adapt quickly when the end effector changes or when the weight of the carried objects varies.

[0101] Adaptation requires a limited number of retraining cycles depending on the chosen speed / rate of change of synaptic weights and the desired accuracy (from 1 cycle for a small variation in weight to about ten cycles for large variations in weight or the need for high accuracy).

[0102] The control system according to the invention may optionally allow the addition of an adaptation mechanism to ensure good accuracy in pseudo-statics; to allow the precise adaptation of the position of a given joint, the same mechanics are used to learn - as a function of the starting position and / or the desired target position - the minimum force to be applied to the joint to initiate a small displacement in the direction of the desired target position.

[0103] The control system according to the invention optionally allows the addition of an error correction mechanism that takes into account the actual dynamics following a few reproductions of the trajectory; The control system according to the invention optionally allows generalization by interpolation to intermediate states to explore the working environment (manual control), or to systematize a trajectory (for example in the case of a palletizing task); the interpolation mechanism thus makes it possible to control the speed and to generalize the movements to joint positions of the robotic arm never learned before.

[0104] The control system conforming to the first aspect of the invention advantageously comprises at least one of the improvements presented below, the technical characteristics forming these improvements being able to be taken alone or in combination.

[0105] According to an initial refinement, the maximum elongation and stiffness of the spring(s) are selected and / or defined by an operator via the human-machine interface, enabling them to define the collaborative behavior of the robotic arm. More specifically, defining the maximum stiffness and / or elongation of the springs associated with each joint allows the force and / or torque of the robotic arm to be limited for a given state Ej. This advantageous configuration allows the operator to define how the robotic arm interrupts its trajectory in the event of an interaction with an object or person present in its path when not initially anticipated. This selection or definition of the virtual spring parameters also determines how the robotic arm can be pushed backward by the operator within the framework of a collaborative and compliant robotic arm.

[0106] The reinforcement mechanism used in the present invention exploits features of supervised and unsupervised learning, and can be described as self-supervised insofar as no external "teacher" intervenes.

[0107] The main framework of the invention is that of online learning. There is no offline learning phase performed on a database, as is the case in many neural networks.

[0108] In the present invention, the inputs of the neural network are: 1) A state vector allowing each point of the trajectory to be associated with a transition to be learned (elementary movement from a starting point to an arrival point associated with a label).

[0109] 2) The desired positions of each joint are used to generate a reinforcement signal

[0110] The outputs of the neural network are the values ​​of the resting elongations which are sent to the low-level controller to deduce the torques applied to the motors as a function of the joint position of each joint (2nd input of the low-level controller).

[0111] In summary, the present invention uses an online learning neural network (without backpropagation of gradient).

[0112] The network includes at least one input layer, a hidden layer similar to a WTA (winner takes all) or softmax layer, and an output layer learning the elongations to provide for the desired angular configuration provided as input.

[0113] A reinforcement signal is generated from the difference between the desired joint position and the current joint position (which can be viewed as an accessory input to the network).

[0114] This is a typical architecture for learning nonlinear functions in online learning neural networks without backpropagation of gradients, of the type for example:

[0115] - counterpropagation architecture (Hecht-Nielsen87)

[0116] Hecht-Nielsen, R. (1987). Counterpropagation networks. Applied optics, 26(23), 4979-4984.

[0117] - or of the PerAc type (Gaussier&Zrehen95)

[0118] Gaussier, P., & Zrehen, S. (1995). Perac: A neural architecture to control artificial animals. Robotics and Autonomous Systems, 16(2-4), 291-320.

[0119] The low-level controller then uses the current measured joint position and the resting elongation values ​​provided at different times from the output of the neural network to calculate the torques to be applied at each instant to the motors associated with the different degrees of freedom of the arm.

[0120] Choice of stiffness

[0121] To check the compliance or elasticity of the arm:

[0122] -the storage unit comprises several collections of states, each defined according to a fixed stiffness specific to each virtual spring of the joints i,

[0123] The human-machine interface (HMI) then allows the selection of a robotic arm operating mode associated with a collection of states, where the stiffness Kn of each joint i is between a minimum stiffness Kri_min and a maximum stiffness Kri_max, which is at least 5 times greater than Kri_min and at most 20 times greater than Kri_min. For example, there is one neuron per state in the network, for the group of neurons responsible for state recognition. For a state activated at 100%, the corresponding neuron is set to 1, and to 0 if it is not activated at all.

[0124] In other words, at a fixed maximum elongation, a minimum stiffness Kri_min per joint i is calculated to allow reaching the position associated with a state Ej. The stiffness Kn of joint i in the system is between a low stiffness Kri_min ("soft system") and a high stiffness Kri_max ("hard system"), the high stiffness value Kri_max being, for example, more than 100 times Kri_min. At the fixed maximum elongation, which cannot exceed a certain value defined in advance for each application, if the stiffness Kn of joint i is taken below Kri_min, joint i will not move (elastic force too low).

[0125] According to a preferred embodiment of the invention, the stiffness Kn of the joint i of the control system according to the invention is between a low stiffness Kri_min = 10 and a high stiffness Kri_max = 5 to 10 x Kri_min = 50 or 100, to meet the constraints of collaborative robot type applications or when the robotic arm comes into contact with a surface whose curvature is variable and / or unknown (low stiffness).

[0126] Here we do not provide the unit for stiffness because torque itself has no explicit unit. Indeed, the torque output from the low-level controller is sent to an electronic control board where 0 corresponds to no torque and 1000 represents the maximum permissible torque based on the current the board can deliver. Above 1000, the board supplies the maximum permissible current.

[0127] In other words, no unit is specified here for the stiffnesses K because they depend on the electronic amplifiers and motors used.

[0128] Low values ​​for the Kn stiffness of joints i are chosen if a compliant robotic arm is desired, capable of reacting to an unforeseen external interaction in a moderate, damped, and smooth manner—that is, by being able to stop in its trajectory and not exert a significant torque or force. Conversely, high values ​​for the Kn stiffness of joints i are chosen if a more dynamic and less compliant robotic arm is desired—that is, one capable of resisting or opposing an unforeseen external interaction by imposing a high torque or force, or even continuing its trajectory, or when the robotic arm needs to be very precise along its trajectory.

[0129] Learning mechanisms

[0130] The learning mechanisms of the neural network are based for elongations and / or states Ej on at least one of the following three learnings: -associative learning, such as for example of the Hebb rule type, modulated by a reinforcement or error correction signal;

[0131] -classical conditioning learning corresponding to least mean squares (LMS) error minimization (example publication Widrow, B., & Hoff, ME (1988). Adaptive switching circuits. In Neurocomputing: foundations of research (pp. 123-134));

[0132] - a category learning (used for state recognition) of the WTA (Winner Takes All) type with an update rule directly inspired by K-Mean algorithms and Kohonen maps.

[0133] Advantageously, associative learning is free of backpropagation of gradients. Functioning of the neural network and the arm die

[0134]

[0135] towards a

[0136]

[0137] of

[0138] After training, updating the neural network allows the calculation of the rest elongations of the virtual springs as a function of the different inputs of the network, the neurons of the output layer of the neural network being used at each period T2 to calculate the rest elongation of the associated spring i 0ori / E(n) such that:

[0139] 0Ori / E(n) = Lr_max_i. f(Sj Wij(n) . Ej(n)), with:

[0140] Wij is the weight of the synapse linking the input Ej to the output associated with the joint i of the neural network; Ej is a state linked to the target position;

[0141] Given that E is a vector containing the set of Ej, E = [Ei, E2, ..., Ej, ...]

[0142] Ej(n) = 1 to move towards the position associated with Ej at iteration n, or

[0143] Ej(n) = 0 when we do not want to go towards the position associated with Ej,

[0144] otherwise Ej(n) = Ej(P(n)) for position P(n), Ej(n) then corresponding to the output of neurons whose activity corresponds to the recognition level of state j for position P(n),

[0145] The index i corresponds to the joint and the number of the output neuron, and the index j corresponds to the state number Ej, f being a function of the activation of neurons having, by convention, for example, output values ​​between 0 and 1. The high-level controller selects, for a setpoint position P:

[0146] - The learned state Ei closest to the setpoint P, based on a calculated distance between the setpoint position P and the positions of the learned states Ej in the considered state collection Ej, for example, using a K-means algorithm. The low-level control unit then calculates, at each iteration u, the main sub-command controlling the spring Kn (SON / EI(n) - ©mi(u)) for each joint i in order to reach the target position associated with state Ei. For a given starting point (the previous point on the trajectory), reaching the next point requires finding the resting elongations of the virtual springs for state Ej, characterized by the transition from the previous target position to the current target position, which we will call a transition. For example, if we want to go from A to B, we must activate a state associated with the transition AB, which can be seen as the activity of a neuron recognizing this transition.If the trajectory is an endless loop with unique transitions, we can return to states associated with target positions (and the notion of a transient state can be neglected since there is no alternative).

[0147] When the distance between the desired target position (associated with state Ej) and the current measured position increases, the neural network learns to reach the defined target position, thanks to a calculated reinforcement signal used to deliver, at the output, a new current resting elongation 0ori / Ej(n+1) in order to decrease the error between the target position 0i_cibie / Ej and the measured position ©mi(n), at each iteration n of period T2, with n a natural number, to obtain 0on / Ej for the target position which allows the low-level controller to go to the target position with the command Kn. (©OH / EJ (n) - ©mi(u)).

[0148] The modification of a synaptic weight Wij of the neuron associated with each joint i is done according to a variant of Hebb's rule taking into account an error or reinforcement term Ri(n), preferably:

[0149] dWij(n) = Ej(P(n)) . ©ori / Ej (n) . Ri(n)

[0150] with Wij(n+1) = Wij(n) + 8 . dWdynj(n) with 8 learning rates between 0 and 1,

[0151] The error Ri(n) for spring i is defined by:

[0152] Ri(n) = f((0i_cible / Ej(n) — ©mi(n)) . a1 ) - f((0mi(n) — 0i_cible / Ej(n)) . a1 ) ,

[0153] a1 is chosen so that the reinforcement signal saturates or plateaus for an angular difference greater than a given angular threshold, with the modification of synoptic weights ceasing as soon as the absolute value of the error signal decreases. This restriction prevents the synaptic weights from changing when the joints move in the direction that reduces the error signal from their current starting positions to their target positions.

[0154] The weights Wij of these neurons are initialized to the same value, advantageously, Wij(0) = 0.5 in the middle of the elongation dynamics which we assume to be normalized between 0 and 1.

[0155] Moving the robotic arm to a target position not learned by tiling the environment

[0156] After an initial training phase known as "3D environment tiling", the high-level controller uses, for each new state Ef to be learned and associated with a target position P(n), several learned states E mapping_j to interpolate the response to the new state Ej';

[0157] The low-level control unit calculates, at each iteration u, for each joint i, the main sub-control of the elongation of the virtual spring r: Kn . (Qori (n) - 0mi(u)), with Son obtained by weighting the learned state / elongation pairs around Ey, such that the activity Ej(P(n)) of the learned states E mapping_j corresponds to their distance from the position P(n), after softmax normalization.

[0158] the activity Ej(P(n)) becomes Dj such that:

[0159] Dj(P(n)) = exp(y . Ej(P(n))) / Si=i to Ne exp(y . Ei(P(n))) with Ne the number of learned states, we then obtain a current learned elongation 0OH (n) = Lr_max_i . f(L=1 to Ne Wij (n) . Dj(P(n)))

[0160] With y a constant used to boost the most active states and set the others to 0, f is a bounded ramp function y = f(x) such that: y = 0 if x < 0, y = 1 if x > 1, and y = x otherwise, and Ej(P(n)) = 1 - dist(Emapping_j, P(n)) / dmax, where dmax is a normalization term corresponding to the maximum possible distance between the learned and tested positions, and Wij is a learned synaptic weight of a neuron in the neural network. Ej' is a new state that has not yet been learned. To avoid starting from scratch to find the associated elongations, we use factory-prepared learning, which consists of tiling the environment. We reactivate the learned states in proportion to their proximity to the new target position. Depending on the level of recognition of the Emappingj(P(n)) states, the arm is pulled more or less in that direction.

[0161] The formula guarantees a normalization of forces which brings the arm to an intermediate position between the learned positions: interpolation of the learnings carried out during the mapping phase (see fig. 5).

[0162] To reach any unlearned target position P defined in joint space by (0o_cibie, ...0i_cibie, ... , 0M_cibie) or in Cartesian space, from the states E mappingj learned during tiling and associated with the 4 previously learned positions closest in joint space to the 3D target position defining a triangular-based pyramid around any unlearned target position P; and, for each joint i considered independently of each other, the proposed elongations are weighted by a level of recognition of the states E mappingj pOUT the target position, according to:

[0163] 0Ori(n)= Lr_maxJ . f (Sk=1..4 des l=top-k(EmappingJ(P(n)), k) Wij . Emappingj(P(n)))

[0164] where top-k(E,k) corresponds to the top-k of the activities of the Emappingj states as a function of the target position P(n),

[0165] Emapping_l(P(n)) = 1 - dist(EmappingJ, P(n)) / dmax

[0166] and Wij is a learned synaptic weight.

[0167] The target position P(n) depends on the joint positions that we wish to obtain and the stiffness of the joints i of the robotic arm controlled by the control system according to the invention.

[0168] Thus, the interpolation performed by the control system according to the invention partially compensates for the external forces applied to the different segments of the robotic arm so that the equilibrium position of the robotic arm corresponds as closely as possible to the target position associated with the unlearned context Cj'. The interpolation only provides an approximation of the solution, that is, of the desired target position. The fine-tuning mechanism is advantageously used in conjunction with the interpolation function to ensure a final correction that allows the arm to reach the target point without requiring relearning. This advantageous configuration drastically reduces the learning time of the robotic arm controlled by the control system according to the invention.

[0169] Optionally, the function Top-k(A) can be defined recursively as the set of elements of A such that:

[0170] Top-k(A, 1) = {max(A)}

[0171] Top-k(A, i + 1) = {max(A - top-k(A, i))} U top-k(A, i)

[0172] For tiling, the joints are considered "independently of each other," as this limits the number of states to be learned and therefore the computational power and complexity. This allows the system to operate at a first approximation, with learning refining the approximation.

[0173] Target equilibrium position

[0174] The target position reached by the robotic arm, accurate to within a value V, is a target equilibrium position 0i_equilibrium, representing the equilibrium of all forces acting on the robotic arm. In this case, the main subcommand of the low-level controller is Kn . (0ori / Ej (n) - 0mi(u)). This command allows for the real-time movement of joints i during a subsequent movement from the starting position to the target position 0i_dbie / Ej, accurate to within a value V. During the initial movement, the neural network, having learned a current resting elongation 0on / Ej for each joint i, moves from the starting position to the target position 0i_dbie / Ej, accurate to within a value V. The neural network is capable of modifying 0ori / Ej, based on the reinforcement signal every n, to adjust the control of 0on / Ej (n) to reach the target position 0i_cibie / Ej during this subsequent movement from the starting position.

[0175] Another secondary spring can be added to learn the minimum force required to initiate movement and overcome dry friction forces, which are generally greater than viscous friction forces, in order to directly generate this force when the robotic arm stops approaching the target position.

[0176] Discretized movement

[0177] According to another improvement, the high-level controller is trained to learn different resting elongations for each joint depending on the dynamics of the movement.

[0178] The movement is discretized into different passage points which constitute target positions for which specific resting elongations are determined.

[0179] This learning process is advantageously carried out in the order of the transition points, from state to state, each state corresponding in this case to a transition between two target positions depending on the direction of movement, so that the robot can learn to take into account the effects related to its inertia and the dynamics of the target trajectory. It should be noted that, in the context of the present invention, for the joints of the robotic arm, the learning of the elongations of each joint i allowing movement from state A of the robotic arm to state B of said robotic arm is different from the learning of the elongations of each joint i allowing movement of the robotic arm from state C to state B.In other words, the learning of the robotic arm by the control system according to the invention must consider a first succession of states between A and B and then a second succession of states between C and B rather than, more simply, a learning of state B and its neighboring states.

[0180] In other words, when the arm has to describe a movement passing, for example, through three points A, B, and C, the learned elongations for A, B, and C depend on the order of the sequence. Thus, learning to move from A to B or learning to reach B from C can involve very different elongations. It is therefore considered incorrect to say that one wants to go to B; one should rather speak of performing the AB transition or the CB transition. The two transitions must be encoded on different neurons (different states) so that each neuron is correctly associated with a well-defined set of elongations. There is therefore no difficulty in obtaining the "target transient position." It is identified as soon as the desired trajectory is entered, with the recruitment of one state per transition. For the ABC sequence, this would involve the recruitment of states AB and BC.If we also learn the CBA sequence, we will have to recruit 2 additional states for CB and BA unlike a purely kinematic system which would have considered in both cases only 3 states: A, B and C.

[0181] Real-time neural network training (i.e., with a precise update interval of every 10 ms) allows us to go far beyond what is possible with updates based solely on kinematic information. It enables the learning process to account for the effects of dynamics from one iteration to the next.

[0182] In other words, the high-level controller is trained to learn a current resting elongation sound – for each joint i and each transition – according to the desired movement dynamics.

[0183] the movement being discretized into different passage points which constitute target transient positions for which specific rest elongations are determined,

[0184] This learning process follows the order of transition points, from state Ej to state Ej+1,

[0185] Each state Ej in this case corresponds to a transition between two target positions depending on the direction of movement so that the robotic arm can learn to take into account the effects related to its inertia and the dynamics of the movement.

[0186] and once trained, the Ti control command comprising at least one main sub-command:

[0187] Kri . (0Ori / Ej (n) - 0mi(u)),

[0188] and allows the joints i of the robotic arm to be moved towards the target transient position 0i_ _transitoire_cible / Ej .

[0189] Adaptation Mechanism In an implementation, the high-level controller is trained, via an adaptation mechanism, during movement learning or during the usage phase, for a state Ej, with the following control command for the low-level unit:

[0190] n(u) = Kri. (0Ori / Ej (n)- 0mi(u)) + Kr'i. (0Or'i / Ej(n) - 0mi(u))

[0191] the high-level controller using the modeling of an additional elongation 0ori / Ej(n) of at least one virtual secondary spring r'i per joint i and located in series with the main virtual spring n, to allow from the measured position 0mi(u), to move the robotic arm towards the target position 0i_target / Ej.

[0192] After learning, the Ti control command allows, from the measured position, to move the robotic arm towards the target position 0i_target / Ej.

[0193] The maximum elongation of the secondary spring is much less than the maximum elongation of the main spring, typically more than 10 times less. This advantageous configuration limits the range over which adjustment is possible and avoids altering the position controlled by the main spring.

[0194] After learning, the Ti control command allows, from the starting position, the robotic arm to be moved towards the target equilibrium position 0i_cibie / Ej which will be reached with an error less than a threshold Sadapt less than the value V, in an iteration N,

[0195] The control command Ti at that moment compensates for the external forces applied to the segments for each of the joints i of the robotic arm, and maintains the position of the arm at an error below the threshold Sadapt for the iterations following iteration N.

[0196] The adaptation mechanism defined above differs from the learning mechanism only in the learning step size, which is much smaller for the adaptation mechanism. Typically, the adaptation step size used during adaptation is equal to, or approximately equal to, one-hundredth of the learning step size used during the learning mechanism. Optionally, if faster adaptation is desired, the learning step size used during adaptation can be equal to, or approximately equal to, one-tenth of the learning step size. Furthermore, the adaptation mechanism is only triggered when the distance between the end effector of the robotic arm and the target position is less than a threshold Sadapt.

[0197] Error correction mechanism

[0198] The computing and storage unit can implement, when the robotic arm is in dynamic operation and moves between several states Ej, a dynamic error correction mechanism using the error measured at the last iteration of the movement aimed at reaching the position associated with a state Ej to control for each joint i a secondary virtual dynamic error correction spring, located in series with the main virtual spring.

[0199] This secondary virtual spring either opposes or complements the effect of the primary virtual spring by proposing a correction term that integrates, through iterative corrections, the effect of inertias during the reproduction of the same movement.

[0200] The dynamic error correction mechanism in which the control unit performs the summation of the main control sub-command and the sub-command of the secondary spring associated with the dynamic adaptation, is:

[0201] n(u) = Kri. (0Ori / Ej (n)- 0mi(u)) + Kr'ï. (0O_dyn_i / Ej (n) - 0mi(u))

[0202] with n a natural number corresponding to the last programmed iteration to reach the target position Ej,

[0203] The update of 0o_dynj / Ej (n) is performed at each iteration n using:

[0204] 0O_dyn_i / Ej (n) = Lr”_max_i . f(Wdyn_ij(n) . Ej(n))

[0205] The elongation 0o_dynj / Ej of this spring is modified so that at the end of a movement segment between two states Eji and Ej2, the joint position of the joint in question gets closer and closer to the target joint position Ej2, according to the learning of the high-level controller as follows:

[0206] dWdynj(n) = Ri(n) . Ej(n)

[0207] ©t Wdynjj(n+1) — Wdynjj(n) + Gleam. dWdyn_ij(n) with Ri(n)= 0cibiej(n) - 0mi(u)

[0208] Muscle model

[0209] The muscle model for each joint i can be represented for each type of spring by a pair of antagonistic virtual springs.

[0210] In this case, the low-level control unit can sum two control sub-commands: T+i(u) associated with the first spring and Ti(u) associated with the second opposing spring, such that: T+i(u) = Kn . (0ori+(n) - 0mi(u)) and Ti(u) = -Kn . (0ori-(n) - 0mi(u)) The elongation learning is carried out in parallel on the two springs associated with each joint.

[0211] Speed ​​limitation mechanism

[0212] In one variant, the low-level controller can sum, from the measured positions 0mi, during the movement of the robotic arm:

[0213] - the sub-control of elongation and

[0214] - an additional sub-command to help synchronize the movements of the different joints by limiting the speed of each joint i as a function of the difference between a real measured speed of the joint and a target speed for said joint i, which is written: T'vi(u) = K'vi . (Vcibiej (n) - Vmi (u)) Where Vmi is the time derivative of 0mi, if this real measured speed Vmi is greater than the target speed Vcibiej (n).

[0215] This synchronization of movements between the different joints of the robotic arm is achieved by reducing those joints with excessive joint displacement—that is, an elongation speed. To do this, the control system identifies and regulates the movement speed of joints that extend too rapidly relative to a predetermined target speed (Vcibie). Optionally, the control system employs viscous damping.

[0216] The low-level controller can sum during movement, starting from the measured positions 0mi:

[0217] - the main sub-command for elongation control and

[0218] - a speed limiting sub-command for each joint i, which is written rvi(u) = Kvi . (VthresholdJ (n) - Vmi(u))

[0219] Where Vmi is the time derivative of 0mi, and which is activated if the measured velocity V m of the corresponding joint i is greater than a threshold velocity Vseuiij (n).

[0220] This force or torque is added to the previously calculated elastic force and / or torque. This advantageous configuration makes it possible to secure the control system according to the invention and the robotic arm by preventing the speed of the joints from becoming too high and / or too dangerous when transitioning from one state to another.

[0221] Speed ​​control mechanism

[0222] During movement between two target positions A and B, associated respectively with a starting state and an ending state, the high-level controller uses the learned rest elongations of the two states to calculate new rest elongations corresponding to their weighting during movement. This explicitly controls the speed of the robotic arm, Vt, which is either the target speed Vcibiej(n) or the threshold speed Vseuiij(n) during its movement along the trajectory AB.

[0223] To do this, for example, the human-machine interface can be defined using the following displacement discretization equation:

[0224] Es(n) = (n-no) / NAB and EA(H) = 1 - Es(n),

[0225] with n = n0 at the last iteration of A and NAB is the number of iterations to go from A to B, calculated as a function of the computation period T2 of the high-level controller: NAB = (dist(A,B) / Vt) / T2 in order to provide the high-level controller with the values ​​EA(H) and Es(n) which uses them in the equation 0ori / E(n) = Lr_max_i . f(Sj Wij(n) . Ej(n)), which becomes 0ori / E(n) = Lr_max_i . f( WiA .EA(H) + WiB .EB(H)) in order to:

[0226] - to reach, at each iteration n from no to no + NAB, a new equilibrium position between A and B;

[0227] - to control the speed of each joint i by imposing a speed greater than or equal to a target speed, the low-level controller also exploiting a speed limiting sub-command for each joint i, and which is written Tvi(u) = Kvi . (VthresholdJ (n) -Vmi(u)).

[0228] Vmi is the derivative of 0mi, which is activated if the measured velocity V mof the corresponding joint i is greater than a threshold velocity Vseuiij(n) which is equal in this case to Vt.

[0229] Here, the purpose of limiting the speed of each joint i is to ensure the safety of the robotic arm and / or operators located nearby by preventing the speed of movement of a joint from exceeding the threshold speed Vseuiij.

[0230] Naturally, the speed of each joint depends, firstly, on spatial sampling, which is essentially a certain density of intermediate states selected between two successive target states, or even a certain density of target states used to guide the robotic arm along a trajectory. Secondly, the speed of each joint also depends on the robotic arm's control frequency, which is essentially the frequency at which successive target states are guided to the robotic arm. Based on these two parameters—the control frequency and spatial sampling—the definition of states and the stabilization of the robotic arm—and each of its joints—to the joint configurations representing these states allows the robotic arm to be controlled more or less dynamically.

[0231] Learning about stiffness

[0232] The system according to the invention can also learn the stiffnesses Kri(n). The stiffnesses Kri(n) can be learned and adjusted by the neural network, via the reinforcement learning mechanism, the learning rate 8 of the elongation 0ori(n) being faster than the learning rate 8' of the stiffness Kri(n), the neural network modifying the stiffnesses Kn of the virtual springs, with:

[0233] Kri / E(n) = Kr_max_i. f(Sj W'ij(n). Ej(n)), with:

[0234] dWij(n) = Ej(P(n)) . K ri / Ej(n) . R'i(n)

[0235] R'i(n) = f((0i_cibie / Ej(n) - 0mi(n)) . a1) - f((0mi(n) - 0i_cibie / Ej(n)) . a1), a1 being chosen so that the reinforcement signal saturates for an angular difference greater than a given angular threshold,

[0236] Wij(n+1) = W'ij(n) + 8' dWij(n) with 8' the learning rate between 0 and 1.

[0237] f being a function of the activation of neurons having, by convention, for example, output values ​​between 0 and 1.

[0238] As a reminder, for the error learning rate on elongation, we have:

[0239] 0Ori / E(n) = Lr_max_i. f(Sj Wij(n) . Ej(n)), with:

[0240] dWij(n) = Ej(P(n)) . 0ori / Ej(n) . Ri(n)

[0241] with Ri(n) = f((0i_cibie / Ej(n) - 0mi(n)) . a1) - f((0mi(n) - 0i_cibie / Ej(n)) . a1), a1 being chosen so that the reinforcement signal saturates for an angular difference greater than a given angular threshold

[0242] f being a function of neuronal activation having, by convention, output values ​​for example between 0 and 1

[0243] Wij(n+1) = Wij(n) + 8 . dWij(n) with 8 learning rates between 0 and 1,

[0244] Thus, the stiffness values ​​Kri(n) are learned and tuned by the neural network and used by the low-level controller via reinforcement learning, with the learning rate of the elongation 0ori(n) being faster than that of the stiffness Kri(n). In the context of the present invention, the learning rate controls the neural network's ability to find the correct solution to converge a given joint towards an expected joint configuration, i.e., a target elongation. Ultimately, the learning rate corresponds to a greater or lesser number of iterations and a greater or lesser degree of accuracy in the solution.

[0245] The elongations can be angular - in the case of a virtual torsional or helical spring, or linear - in the case of a virtual linear spring.

[0246] Thanks to the invention, the control system can be:

[0247] - without force sensors, such as strain gauges; and / or - without additional mechanics - that is, without a clutch (also known as a "torque control drive") and / or without a mechanical spring to limit the forces; it should be noted that, particularly advantageously, the robotic arm control system according to the invention makes it possible to limit the torques and forces generated by the effector of the robotic arm simply by defining a maximum stiffness and / or elongation for each spring associated with each joint; and / or

[0248] - without a dynamic model of the robotic arm to control its operation. In the control system according to the invention, dynamic effects are taken into account by learning the resting elongations of the springs associated with each joint. Complementarily or alternatively, the characterization of the corresponding states of the robotic arm results in transitions between two successive states that take into account the direction and speed of movement of the robotic arm and each of its joints. Thus, for example, the joint states of the robotic arm EAB that allow transition from state A to state B differ from the joint states ECB that allow transition from state C to state B, even if these two joint configurations lead to the same state B.

[0249] According to another improvement of the invention, the system is free of mechanical systems for impedance measurement, such as springs, and / or without sensors for measuring forces, and / or without a disengagement mechanism triggered by torque overshoot. According to another improvement of the invention, the robot is a collaborative robot or cobot, that is, a robot that must not exceed a maximum speed and must limit its impact force to comply with the relevant ISO standards.

[0250] According to another improvement of the invention, the robotic arm is able to adapt to any type of non-planar surface, by the use of virtual springs, without calculating the equation of the non-planar surface.

[0251] The robotic arm can be configured for use in the following applications: manufacturing, automotive, aerospace, healthcare, and construction.

[0252] In a realization, a default position of the robotic arm is the position obtained by the robotic arm for zero elongation or zero angle of the actuators of each joint, linked to a calibration position.

[0253] Various embodiments of the invention are envisaged, incorporating, according to all their possible combinations, the different optional features described herein.

[0254] Other features and advantages of the invention will become apparent from the following description on the one hand, and from several illustrative and non-limiting examples of embodiments given with reference to the attached schematic drawings on the other hand, in which:

[0255] [Fig.1] illustrates a representative diagram of the mathematical model applied to model a joint of the robotic arm controlled by the control system according to the invention;

[0256] [Fig.2] illustrates a first joint configuration of the model presented in FIGURE 1;

[0257] [Fig.3] illustrates a second joint configuration of the model shown in FIGURE 1;

[0258] [Fig.4] illustrates a representative diagram of an interpolation mechanism implemented by the control system according to the invention;

[0259] [Fig.5] illustrates a representative diagram of a weighting mechanism for the recognition of 3 states of the robotic arm by the control system according to the invention; [Fig.6] illustrates a time diagram of the learning of the robotic arm for a given joint configuration and according to a low value of joint stiffness;

[0260] [Fig.7] illustrates a time diagram of the learning of the robotic arm for a given joint configuration and according to a high value of joint stiffness; [Fig.8] illustrates a time diagram of the learning on the dynamics of a trajectory defined between two angular positions of the robotic arm and according to a low value of joint stiffness;

[0261] [Fig.9] illustrates a time diagram of learning on the dynamics of a trajectory defined between two angular positions of the robotic arm and according to a high value of joint stiffness;

[0262] [Fig.10] illustrates a time diagram of the robotic arm controlled by the control system according to the invention and to which a weight is added, according to a low value of joint stiffness;

[0263] [Fig.11] illustrates a time diagram of the robotic arm controlled by the control system according to the invention and to which a weight is added, according to a high value of joint rigidity;

[0264] [Fig.12] illustrates an adaptation mechanism of the robotic arm controlled by the control mechanism according to the invention;

[0265] [Fig.13] illustrates an interpolation mechanism of the robotic arm controlled by the control mechanism according to the invention and along a trajectory initially defined by only two learned positions.

[0266] [Fig.14] illustrates the principle of the invention; it represents the general architecture of the system which comprises 3 levels working with 3 different time constants; the low-level controller simulates the resultant forces of the simulated springs as a function of the position of the arm and the values ​​of the rest elongations of the arms provided by the neural network; the output of the low-level controller is sent directly to the motors via an ethercat link in order to guarantee real time and a high operating frequency necessary for a good quality simulation; the second level consists of the neural network which mainly calculates the rest elongations of the virtual springs in order to satisfy the commands from the human-machine interface: direct commands (target theta position) or a pre-recorded sequence of points to be joined associated with states Ej;The neural network learns through reinforcement to minimize the error between the desired angle and the actual position of the arm for a given state Ej.

[0267] Of course, the features, variants, and different embodiments of the invention can be combined in various ways, provided they are not incompatible or mutually exclusive. In particular, variants of the invention may include only a selection of features, described hereafter in isolation from the other described features, if this selection of features is sufficient to confer a technical advantage or to differentiate the invention from prior art.

[0268] In particular, all the variants and embodiments described are combinable with each other provided there are no technical obstacles to such combination. In the figures, elements common to several figures retain the same reference.

[0269] Figure 1 schematically illustrates the muscle model implemented in the control system according to the invention, in which each joint i of the robotic arm is modeled by a spring system. In particular:

[0270] - The joint position—that is, the elongation—of each joint of the robotic arm is determined by a position sensor. Thus, the angular or linear velocity of each joint can be obtained by time derivative of the corresponding angular or linear elongation, or by using a position sensor connected to an actuator controlling the joint in question. - Each joint—and its associated actuator—is controlled by torque, particularly when controlling the rotation of a pivoting joint, or by force, particularly when controlling the extension of a linear joint, or if, for example, a pivoting joint is controlled by the extension of a cylinder.

[0271] Consequently, each joint is modeled as a system comprising at least one spring – as shown on the right-hand side of FIGURE 1. Thus, to control the elongation of the joint, it suffices to control the elongation of the spring(s) associated with said joint. The control system according to the invention implements a neural network to control the robotic arm. This control is – in principle – associated with a control of the rigidity of each joint via the neural network. Thus, depending on the state or context Ei, the neural network proposes a current resting elongation 0o. In the absence of any external force applied to the robotic arm, the equilibrium position of a given segment of the robotic arm is then obtained when the force of the spring(s) associated with the corresponding joint balances the force of gravity acting on said segment, as shown in the diagram on the right-hand side of FIGURE 1.

[0272] The diagram on the left of Figure 1 illustrates a neuron controlling the elongation of a spring as a function of the selection of a state Ej of the neural network implemented to control the robotic arm using the control system according to the invention. Advantageously, the neural network is configured to control the robotic arm via a learning mechanism whose purpose is to obtain – for each synapse of the neural network – a synaptic weight Wj such that the equilibrium position of the segment – ​​associated with an equilibrium elongation 0 of said joint – corresponds to the desired configuration of the robotic arm.

[0273] Naturally, the behavior of the robotic arm, for the same given command, will differ depending on the external forces applied to it. Figures 2 and 3 illustrate two different situations from the one shown previously in Figure 1, for the same robotic arm and the same command for its joints. Figure 2 illustrates a configuration in which the robotic arm is located outside the gravitational field. Therefore, in the absence of gravity, the equilibrium position of joint i of the segment shown for the robotic arm is directly associated with the value learned by the neural network. We also observe that, for this same command, the robotic arm segment is in a more upright position; that is, the equilibrium elongation calculated by the neural network is then greater than before.

[0274] Conversely, if an additional external force is applied to the robotic arm segment after the previous learning phase, as illustrated in Figure 3, then the robotic arm will not reach its "normal" equilibrium position—corresponding to the one it reached in Figure 1—but will instead have a lower equilibrium position. This means that the corresponding joint of the segment shown in Figure 3 will have a lesser elongation than it did for the joint configuration shown in Figure 1. In the context of the present invention, the additional external force can be of any type. By way of non-limiting example, it can be caused by a load carried by the robotic arm, or by an obstacle with which the robotic arm interferes in its workspace, such as the presence of a human being.

[0275] To return to the desired equilibrium position and therefore to a resultant at the robotic arm of all the old applied forces still acting as new (and which include the forces of the virtual springs and possibly virtual damping which will be calculated by the neural network thanks to this equilibrium equation resultant of the forces zero), that is to say that of FIGURE 1 but with a "set" of different external forces, it will therefore be necessary for the neural network to reduce the synaptic weight associated with its current state so that the resultant of the external forces applied to the considered segment of the robotic arm is zero at the desired equilibrium position.

[0276] In the context of the present invention, the external forces applied to a given segment of the robotic arm include, in particular, in addition to gravity, dry friction forces and / or viscous friction forces. These friction forces are those encountered at each joint. Dry friction forces can cause the robotic arm to stop at an equilibrium position slightly different from the desired target position if they are not taken into account.

[0277] In summary, the invention takes into account all the forces applied to the arm, which can change, in the calculation of current rest elongations so that the resultant of the forces is zero.

[0278] But it does not need to measure them explicitly, nor to know the weight of the segments or elements carried by the robotic arm, or of new external forces such as the presence of someone in front of the robot.

[0279] The control parameters of the robotic arm using the control system according to the invention include, in particular:

[0280] - Ei: a binary or analog vector such that the activated neuron of the neural network corresponds to the state of the robotic arm - and the corresponding joint segment - for a desired equilibrium position at a given instant in a sequence or during a trajectory;

[0281] - N: the maximum number of states, i.e., joint configurations of the robotic arm, that can be learned by the control system;

[0282] - p: the learning speed of the neural network;

[0283] - pr: the desired precision for obtaining a given equilibrium position;

[0284] - K: the stiffness of the virtual spring associated with a joint of the robotic arm.

[0285] A learning mechanism is implemented to identify and learn the necessary elongations in the spring model—for each joint—to achieve the desired equilibrium configuration of the robotic arm. As a non-limiting example, the learning mechanism is a reinforcement learning type based on Hebb's rule.

[0286] The neural network control is as follows, illustrated below for a joint control model using a two-spring, opposing spring model, indexed (+) and (-) in the following equations. Of course, this example is given only to illustrate and explain the learning and control of the robotic arm; and those skilled in the art could adapt it to other spring models, not all of which can be described here for the sake of clarity and conciseness.

[0287] The learning mechanism allows the robotic arm, controlled by the control system according to the invention, to reach the vicinity of the desired target position. However, it is possible that the robotic arm may not stop exactly at this equilibrium position due to frictional forces and the inertia of the robotic arm. The following paragraphs illustrate the control of the robotic arm using the control system according to the invention and via an initial learning mechanism, described below, for the movement of the robotic arm from a first equilibrium state EA corresponding to a first position A to a second equilibrium state EB associated with a second position B.

[0288] The neural network thus begins by activating the joint context associated with state EA and converges towards A. The elongations 0i of each joint i correspond to the joint configuration associated with state EA. The synaptic weights Wi learned for the elongations of each joint take into account this desired configuration and the mass of the robotic arm. Naturally, the external forces acting on the robotic arm—and on each joint segment—differ for each of its states E.

[0289] To configure the robotic arm in the EB state associated with position B, the control system deactivates the EA state of the robotic arm associated with position A and then creates a new EB state at the neural network level, this EB state being associated with the target equilibrium position B.

[0290] This learning process—illustrated here in a single iteration, for example of the "kmeans" type—will reactivate the same synaptic node of the neural network for the next request to move the robotic arm to position B associated with state EB. Recognition of the state associated with position A or B is performed based on the desired joint configuration of the robotic arm, initially, or on the activation of the corresponding synaptic node and that state.

[0291] The first time the neural network activates the EB state associated with position B, the synaptic weights are nulled; and the robotic arm therefore returns to its default joint configuration, corresponding to zero elongations of the joints defined for this so-called calibration position B. Subsequently, a measurement by the position sensors associated with the joints of the difference between the desired position of the robotic arm—or at least one of its joints—and its current position creates an error signal that triggers reinforcement learning of the neural network's synaptic weights, so as to provide the elongations associated with position B in the EB state.

[0292] During this phase, for each joint, the synaptic weight associated with the corresponding state Ej is modified via the previously defined equations dW+ij and dW-ij. According to one embodiment of the control system of the invention, the stiffness of the springs associated with each joint can be constant and predetermined by the operator, for example, via the human-machine interface. Naturally, the stiffness constant of the robotic arm joints must be chosen to be sufficiently high relative to the mass of the robotic arm and its inertia.

[0293] Once position B is learned, the triggering of position B associated with the state EB = 1 induces a new instruction for the rest elongations 0o of the springs of each joint, and therefore forces which will induce a displacement of the robotic arm until reaching an equilibrium point at which the forces generated by the springs of the joints are equal to the external forces applied and corresponding to the desired angular configuration.

[0294] Activating state EA, then state EB, and then state EA again induces a back-and-forth movement between A and B. States EA and EB are kept active to allow the robotic robot to reach these positions.

[0295] The control system according to the invention also provides an interpolation mechanism which is described below, with reference to FIGURE 4. This interpolation mechanism - represented in one dimension (for simplicity) in FIGURE 4 - allows the speed of movement on the trajectory AB to be controlled by forcing the robotic arm to make small movements - preferably of equal length - between A and B.

[0296] To achieve this, and to move more precisely from position A to position B, it is possible to activate EA and EB simultaneously, for example by setting EA = 0.5 and EB = 0.5. By doing so, the robotic arm's equilibrium point is located in a joint configuration midway between the angular configuration associated with A and that associated with B. By changing the activity levels of CA and CB over time, the control system performs an interpolation between A and B. This advantageous configuration is particularly clever because it also allows control of the minimum speed of each joint by controlling the interpolation step – defined by the activation / deactivation ramps of the two successive states EA and EB.

[0297] FIGURE 5 illustrates the effect of weighting the recognition of 3 states A, B and C in 2D for the definition of an equilibrium point P. By changing the weighting of each state EA, EB and Ec, we increase or decrease the force associated with the spring of each joint and we can thus position the robotic arm precisely inside the triangle (A, B, C).

[0298] The interpolation mechanism between several joint states of the robotic arm, as described above, allows for the generalization of scattered learning and provides two main advantages:

[0299] - The ability to quickly move to unlearned points, allowing the robotic arm controlled by the system to learn a configuration that enables it to reach those points. This advantage is particularly valuable when the robotic arm is used collaboratively or in the presence of a user, allowing the robotic arm to be moved to the positions to be learned. In this case, the control system can utilize a 1D, 2D, or 3D mesh of its workspace, as described below, a mesh created during an initial calibration step of the robotic arm within its workspace.

[0300] - the ability to finely control the movement of the robotic arm along its trajectory by creating virtual intermediate points between two previously learned positions or states. This advantageous configuration also allows, as mentioned previously, for controlling the speed of the robotic arm along the trajectory. Furthermore, the interpolation mechanism described earlier is advantageously used by the control system according to the invention to perform an initial tiling of the robotic arm's workspace, thus providing a one-dimensional, two-dimensional, or three-dimensional mesh that will serve as the basis for such interpolation and fine control of the robotic arm through the use of the weightings described above.

[0301] Of course, for 2D interpolation, the control system according to the invention uses at least three learned positions around the target position to be reached. For 3D interpolation, the control system uses at least four learned positions that together define a tetrahedron. Preferably, a hexagonal tiling can be used to evenly distribute the learned positions that serve as support points for interpolated "intra-mesh" movements. In this way, the robotic arm's control system learns—during initial calibration—to reach all the positions defined by the hexagonal mesh itself, so that it can then—during operational use—be able to wait for any "intra-mesh" position by activating the positions associated with the 3D vertices closest to the desired position.This interpolation is advantageously achieved using a k nearest neighbors algorithm, for example.

[0302] Figures 6 to 13 described below illustrate the performance of the control system according to the invention. In particular:

[0303] Figure 6 illustrates the learning method for the robotic arm to achieve a given elongation—here, a joint position of -26°—by modeling a segment of the robotic arm with two opposing springs and a low stiffness, taken here to be equal to 10. The top graph represents the evolution of the elongations of the first "positive" spring, the middle graph represents the evolution of the elongations of the second "negative" spring, and the bottom graph represents the evolution of the joint configuration of the segment in question, by representing the evolution of the elongation of the corresponding joint. The x-axis of the three graphs is a time axis, with a time step of 15 ms.

[0304] The learning of the opposing spring elongations (Elong_minus and Elong_plus) occurs each time the corresponding joint stops moving in the correct direction. Thus, at each horizontal or nearly horizontal stage of the joint (bottom graph), the opposing spring elongations progress and converge towards the configuration—the state—that allows the desired joint elongation to be achieved. The horizontal stages of the two opposing spring elongations correspond to periods when the robotic arm is not learning because it is moving in the correct direction, minimizing the error between the measured angle 0mi and the target angular position 0i_cibie (here i = 2 corresponds to the second joint of the robotic arm—that is, its second joint—and to the first joint in the vertical plane). The learning rate is 8 = 0.01.The cessation of joint movement is linked to the force developed by the springs becoming too weak compared to the learned elongation, or possibly due to movement in the wrong direction.

[0305] FIGURE 7 is analogous to FIGURE 6, but the stiffness constant of the opposing springs is now fixed at a high value, here equal to 100. We observe that the movement of the joint and the two opposing springs is more continuous than that observed in FIGURE 6. The learning time is dependent on the learning rate used (here 8 = 0.01) multiplied by the error level (R).

[0306] Figures 8 and 9 illustrate the effect of learning on the dynamics of a trajectory defined between two angular positions of the robotic arm: a first position corresponding to an angle of -60° and a second position corresponding to an angle of -25°. Furthermore, the control system is configured to limit the joint movement speed to a threshold angular velocity of 0.3 m / s. Finally, in Figure 8, the spring constant associated with the corresponding joint is set to a low value of 10, while in Figure 9, the spring constant associated with the corresponding joint is set to a high value of 100.

[0307] In Figures 8 and 9, the top graph represents the evolution of the elongation of the first "positive" spring, the middle graph represents the evolution of the elongation of the second "negative" spring, and the bottom graph represents the evolution of the joint configuration of the segment under consideration, by showing the evolution of the elongation of the corresponding joint. In the context of the present invention, the "positive" and "negative" springs are opposing springs attached to the joint considered here. The x-axis of the three graphs is a time axis, with a time step of 15 ms.

[0308] Figure 8 highlights the primary learning process: as soon as the robotic arm reaches the desired position, it is instructed to return to the previous position. The oscillation frequency therefore depends on the robotic arm's speed along the trajectory, and thus on the stiffness defined for the opposing springs. We can see that the learned elongations for the two positions are 260° and 290°. These values ​​differ from the desired angles. These values ​​take into account the effect of gravity on the controlled joint (here, the second joint of the arm). In this first example, the duration of one cycle is 2.4 seconds.

[0309] In FIGURE 9, the speed achieved by the robotic arm is faster. It completes 1 cycle in 1.6 s.

[0310] In the experiments illustrated in Figures 8 and 9, the speed of the robotic arm is limited by a speed damping mechanism such as the one described previously. The parameters used here for this limitation are Kvi = 600 and Vseuiij = 0.

[0311] Figures 10 and 11 illustrate the effect of adding a weight – here 5 kg – to the robotic arm controlled by the control system according to the invention. The segment considered – here the 2 èmeof the robotic arm – is maintained in its initial configuration. In Figure 10, the spring constant of the opposing springs associated with the corresponding joint is set to a low value of 10, while in Figure 11, the spring constant of the springs associated with the corresponding joint is set to a high value of 100. In Figures 10 and 11, the top graph represents the evolution of the elongation of the first "positive" spring, the middle graph represents the evolution of the elongation of the second "negative" spring, and the bottom graph represents the evolution of the joint configuration of the segment under consideration, by showing the evolution of the elongation of the corresponding joint. In the context of the present invention, the "positive" and "negative" springs are opposing springs attached to the joint considered here.The x-axis of the three graphs is a time axis, with a time step of 15 ms.

[0312] We can thus observe that, when the weight is removed, near the 3400 ème iteration on FIGURE 10 and near the 5200 ème Upon iteration on FIGURE 11, the joint configuration of the robotic arm evolves rapidly: the robotic arm cannot return to the initial position because adaptation has been inhibited. Only the main springs function for the joint in question. The movement stops when the frictional forces compensate for the spring force of the joints.

[0313] In Figure 10, the stiffness constant is too low, and the robotic arm—and its joint in this experiment—cannot return to its original position. In contrast, as seen in Figure 11, the return to the original position is possible thanks to the greater stiffness of the joint spring.

[0314] Furthermore, it is noted that the variation in angle associated with the addition of weight on the robotic arm is lower in the case of high stiffness - FIGURE 11 - than in the case of low stiffness - FIGURE 10.

[0315] Finally, we observe that the robotic arm is more precise in the experiment in FIGURE 11, even if it does not return exactly to the target original position, due to friction forces and the absence of the adaptation mechanism in the control system implemented here.

[0316] The influence of the adaptation mechanism is highlighted in Figure 12. As before, the top graph represents the evolution of the elongation of the first "positive" spring, the middle graph represents the evolution of the elongation of the second "negative" spring, and the bottom graph represents the evolution of the joint configuration of the segment under consideration, by showing the evolution of the elongation of the corresponding joint. In the context of the present invention, the "positive" and "negative" springs are opposing springs attached to the joint considered here. The x-axis of the three graphs is a time axis, with a time step of 15 ms.

[0317] In the experiment shown in FIGURE 12, the robotic arm control system is implemented with high spring stiffness for the 6 joints, here set at K = 100. A 5kg weight is applied to the robotic arm near the 100ème iteration. The robotic arm is then configured in a state where it must always maintain the 2 ème The joint is elongated at an angle of -26.8 degrees. The adjustment mechanism corrects the error between the measured position and the desired elongation.

[0318] After the robotic arm's height drops due to the application of weight, the control system according to the invention guides the robotic arm to adapt by controlling the resting elongations of the opposing springs associated with the joints. More specifically, the current resting elongation of the first spring associated with the joint is increased, while the current resting elongation of the second spring associated with the joint is decreased. This modification ultimately generates sufficient force on the joint—that is, greater than the external forces applied to the joint—thus enabling the joint to move.

[0319] The positioning error is significantly reduced, but we also observe that the correction applied is slightly too strong: the robotic arm joint extends beyond the desired target position. The elongation of the two opposing virtual springs associated with the joint is then corrected again, and the joint finally stabilizes in the desired target position, despite the presence of the 5kg weight at the end of the robotic arm.

[0320] In this experiment, the robotic arm is 1m long and the controlled joint is the 2 ème joint, working in the vertical plane.

[0321] Finally, FIGURE 13 illustrates the ability of the control system according to the invention to pilot the robotic arm to reach an unlearned angular position through the interpolation mechanism.

[0322] In this experiment, two joint states of the robotic arm were learned, these two corresponding states, for the 2ème The joint is positioned at 0° and 80°. The learning mechanism is then stopped, so the robotic arm, via its control system, cannot adapt to improve its performance. A VU meter – the top graph – is then used in the neural network simulator's human-machine interface to change the desired angular position between 0 and 40 degrees.

[0323] The position of the joint - measured by its position sensor - is illustrated in the graph at the bottom of FIGURE 13.

[0324] Figure 13 shows that at the beginning of the experiment, the target position (visible on the top graph) is rapidly changed from 0° to 40°. It can be seen that the robotic arm follows the instructions with a certain latency, while generally reproducing the evolution of the desired angular values.

[0325] The robotic arm is then returned to 0° and the angle is changed in successive steps to verify that the robotic arm is indeed capable of reaching the positions labeled 1 to 8 in FIGURE 13. It is observed that linear interpolation, with only two states 80 degrees apart and despite the applicable nonlinear effects, allows the robotic arm to be stabilized in a position close to the desired position. The position error is less than 5°. This position error is completely correctable by the learning mechanism described previously.

[0326] In summary, the invention relates to a control system for a robotic arm in which each joint is modeled by a mathematical model inspired by a mammalian muscle, represented by at least one spring. Controlling the elongation of this spring allows control of the elongation of the corresponding joint. This model defines a stiffness associated with each joint spring, thus enabling the robotic arm to exhibit varying degrees of compliance. The control system implements a neural network which, through learning, determines the joint elongation required to achieve a given joint configuration of the robotic arm. The control system according to the invention then deploys several secondary control mechanisms that optimize the control of the robotic arm, making it more precise, faster, and / or more compliant, for example.

[0327] Of course, the invention is not limited to the examples just described, and many modifications can be made to these examples without departing from the scope of the invention. In particular, the various features, forms, variants, and embodiments of the invention can be combined in various ways, provided they are not incompatible or mutually exclusive. Specifically, all the variants and embodiments described above are combinable.

[0328] The neural network consists of two main parts. The first part identifies a state based on the coordinates of the target position provided by the human-machine interface. A competition mechanism inspired by the K-means algorithm can perform this task by considering as input the concatenation of the previous target position (starting point) and the current target position. For a 6-degree-of-freedom arm, the output group comprises at least 6 neurons associated with the current resting elongations for the configuration of the 6 virtual springs. The memory of the current resting elongation required to perform the movement from point A to point B is stored in the weights linking the neuron associated with the recognition of this transient (i.e., state AB) with the 6 output neurons.

Claims

Demands 1. Articulated robotic arm device comprising a neural network control system for the movement of segments connected by joints i, capable of learning online and in real time the forces to move the joints i towards one or more target positions which are arrival and / or passage positions of the robotic arm, thanks to the simulation of virtual springs realizing the elasticity of each joint i by impedance control, in which the device comprises, for each joint of the robotic arm, a motor, a reducer, and a joint position sensor 0mi in which the control system includes a low-level controller controlling the movement of the joints i of the robotic arm, and a high-level controller driving the low-level controller and which is the neural network calculating throughout the movement, for each joint i, a current rest elongation Son sent to the low-level controller, the low-level controller simulating for each joint i, at least one main virtual spring ri of stiffness Kn and current rest elongation 0ori, the low-level controller calculating, in real time, a control command Ti which includes at least one main sub-command controlling the elongation of the main virtual spring r: Kn x (0OH - 0mi) , which is sent to the actuator associated with joint i, and for the high-level controller: - learning the resting elongations of the virtual springs for a chosen state that represents a joint configuration of each segment of the robotic arm and that defines the equilibrium position of the arm at the target position corresponding to a zero resultant of the forces applied to the arm, giving it elastic properties depending on the chosen stiffness of the springs, at equilibrium in the target position, the forces exerted by the virtual springs are calculated so that the resultant force is zero. This learning process is carried out in order to achieve this balance of forces at the target position using a learning control signal corresponding to the error between the positions measured by the position sensor and the target position. - learning by the neural network of these elongations for each transition from one target position to another target position in the joint space allowing control of the movements of the robotic arm to learn a sequence of movements and define the behavior of the robotic arm for a given task, each joint of the robotic arm having a target position.

2. Articulated robotic arm device according to claim 1, wherein the device is without an additional system to the virtual springs realizing the elasticity of the robotic arm.

3. An articulated robotic arm device according to any one of the preceding claims, wherein: - each joint i is associated with one or more degrees of freedom, with i a natural number, each degree of freedom being associated with one or more virtual springs whose elongations control the position of the robotic arm; - at least one actuator is associated with each joint i and controls the elongations of the corresponding virtual spring by force or torque; - at least one position sensor is associated with each joint i in order to periodically measure the measured position Qmi(u) of the corresponding joint i, with a first period T1 fixed given, so as to simulate the dynamics of the virtual springs; - a human-machine interface (HMI), configured in particular to communicate target positions and emit signals transmitted to the low-level controller and the high-level controller, notably to control the trajectory of the robotic arm; - computing and storage units connected to the actuators, position sensors and the human-machine interface, including: • the low-level controller, which calculates, in real time, at each first period T1, for each joint i and from the positions measured 0mi(u) by the position sensors associated with the joints i, a control command Ti(u) which includes at least one main sub-command for controlling the elongation of the main virtual spring r: Kn x (Son - 0mi(u)), which is sent to the actuator associated with the joint i, in order to reach, from a starting position, a target position 0i_cibie, for each joint i, thanks to a current rest elongation 0on / Ej calculated by the high-level controller, The high-level controller produces, in real time, with a given fixed second period T2, output values ​​between 0 and 1 multiplied by a maximum elongation L r_max allowed for each virtual spring r, which give the current rest elongations 0ori / Ej (n) of the virtual springs, the period T2 being an integer multiple of T1, to reach the target position 0i_cibie, for each joint i, to an accuracy V, the high-level controller having the following input data at each iteration n: a given stiffness of each spring ri, The user instructions are: the target positions 0i_cibie / Ej, each associated with at least one state Ej of a collection of states defined according to the given stiffness of each spring ri, a state Ej corresponding to a transition between a chosen starting position and a target position; the measured positions 0mi (n-1) at each period T2 for each joint i; a learning control signal corresponding to the error between the measured positions 0mi (n-1) and the target positions 0i_target / Ej.

4. Articulated robotic arm device according to claim 3, wherein after training, updating the neural network allows the calculation of the rest elongations of the virtual springs as a function of the different inputs of the network, the neurons of the output layer of the neural network being used at each period T2 to calculate the current rest elongation of the associated spring i 0ori / E(n) such that: 0Ori / E(n) = Lr_max_i. f(Sj Wij(n) . Ej(n)), with: Wij is the weight of the synapse linking the input Ej to the output associated with the joint i of the neural network; Ej is a state linked to the target position; Given that E is a vector containing the set of Ej, E = [Ei, E2, ..., Ej, ...] Ej(n) = 1 to move towards the position associated with Ej at iteration n, or Ej(n) = 0 when we do not want to move to the position associated with Ej; otherwise, Ej(n) = Ej(P(n)) for position P(n). Ej(n) then corresponds to the output of neurons whose activity corresponds to the recognition level of state j for position P(n). The index i corresponds to the articulation and number of the output neuron, and the index j corresponds to the number of state Ej. f is a function of the activation of neurons having, by convention, for example, output values ​​between 0 and 1. The high-level controller selects, for a setpoint position P: the learned state Ei closest to the setpoint P as a function of a distance calculated between the setpoint position P and the positions of the learned states Ej of the considered collection of states Ej, for example with a K-means algorithm, - the low-level control unit then calculates, at each iteration u, the main sub-command for controlling the spring Kn . (SON / EI (n) - 0mi(u)) for each joint i in order to reach the target position associated with the state Ei.

5. Articulated robotic arm device according to the preceding claim, wherein the main sub-control of the low-level controller is Kn . (0ori / Ej (n) - 0mi(u)), this sub-control enabling the real-time movement of the joints i during a new displacement from the starting position to the target position 0i_cibie / Ej to an accuracy value of V, during a first movement, the neural network having learned for each joint i, a current resting elongation learned 0on / Ej from the starting position to the target position 0i_cibie / Ej to the value of precision V, the neural network being able to modify the current resting elongation learned ©ON / EJ , according to the reinforcement signal every n, to adjust the control of ©ON / EJ (n) to reach the target position 0i_cibie / Ej during this new movement from the starting position.

6. Articulated robotic arm device according to any one of the preceding claims 4 to 5, wherein as the distance between the desired target position and the current measured position increases, the neural network learns to reach the defined target position, thanks to a calculated reinforcement signal used to output a new current resting elongation 0ori / Ej(n+1) in order to decrease the error between the target position 0i_cibie / Ej and the measured position ©mi(n), at each iteration n of period T2, with n a natural number, to obtain 0on / Ej for the target position which allows the low-level controller to go to the target position with the command Kn. (©OH / EJ (n) - 0mi(u)), the modification of a synaptic weight Wij of the neuron associated with joint i occurs according to a variant of Hebb's rule taking into account an error or reinforcement term Ri(n), preferably: dWij(n) = Ej(P(n)) . Qori / Ej (n) . Ri(n) with Wij(n+1) = Wij(n) + 8 . dWdynj(n) with 8 learning rates between 0 and 1, The error Ri(n) for spring i is defined by: Ri(n)= f((0i_cible / Ej(n) — 0mi(n)) . a1 ) - f((0mi(n) — 0i_cible / Ej(n)) . a1 ) , a1 being chosen so that the reinforcement signal saturates for an angular difference greater than a given angular threshold, the modification of the synaptic weights stopping as soon as the absolute value of the error signal decreases, the weights Wij of these neurons being initialized to the same value, advantageously, Wij(0) =0.5 in the middle of the elongation dynamics which we assume to be normalized between 0 and 1.

7. Articulated robotic arm device according to the preceding claim, in which the neural network learns in real time to reach the target positions 0i_cibie / Ej without taking into account as input data: the weight of the different segments of the arm, the weight of the objects transported by the robot, the weight of the effectors or tools, and without taking into account their dimensions, allowing the controller to adapt to the variations of all these weights and lengths by calculating the current resting elongation 0on.

8. Articulated robotic arm device according to any one of the preceding claims, wherein, for controlling the compliance and / or elasticity of the arm: - The storage unit comprises several collections of states, each defined according to a fixed stiffness specific to each virtual spring of joints i. - The human-machine interface (HMI) then allows the selection of a robotic arm operating mode associated with a collection of states, where the stiffness Kn of each joint i is between a minimum stiffness Kri_min and a maximum stiffness Kri_max, which is advantageously at least 5 times greater than Kri_min and at most 20 times greater than Kri_min.

9. Articulated robotic arm device according to any one of claims 4 to 8, wherein: - after an initial training phase called "3D environment tiling", the high-level controller uses, for any new state Ej' to be learned and associated with a target position P(n), several learned states Emapping_j to interpolate the response to the new state Ej'; - the low-level control unit calculates, at each iteration u, for each joint i, the main sub-control of the elongation of the virtual spring r: Kn . (Son (n) - 0mi(u)), with 0OH obtained by weighting the learned state / elongation pairs around Ej', such that the activity Ej(P(n)) of the learned states E mapping_j corresponds to their distance from the position P(n), after softmax normalization, the activity Ej(P(n)) becomes Dj such that: Dj(P(n)) = exp(y . Ej(P(n))) / Si=i to Ne exp(y . Ei(P(n))) with Ne the number of learned states, and 0OH (n) = Lr_max_i . f(S=i to NeWij(n) . Dj(P(n))) with y a constant allowing to boost the most active states and set to 0 the others, f is a bounded ramp function y = f(x) such that: y = 0 if x < 0, y = 1 if x > 1 and y = x otherwise, and Ej(P(n)) = 1 - dist(Emapping_j, P(n)) / dmax with dmax a normalization term corresponding to the maximum possible distance between the learned positions and the tested positions, and Wij is a learned synaptic weight of a neuron in the neural network.

10. Articulated robotic arm device according to claim 9, wherein to reach any unlearned target position P defined in joint space by (0o_cibie, ... 0i_cibie, ... , 0M_cibie) or in Cartesian space, from the EmappingJ states learned during tiling and associated with 4 previously learned positions closest in joint space to the 3D target position defining a triangular-based pyramid around any unlearned target position P; and, for each joint i considered independently of each other, the proposed elongations are weighted by a level of recognition of the EmappingJ states for the target position, according to: 0Ori(n)= Lr_max_i. f (Sk=1..4 des l=top-k(Emapping_j(P(n)), k) Wij . Emapping_l(P(n))) where top-k(E,k) corresponds to the top-k of the activities of the Emappingj states as a function of the target position P(n), Emapping_l(P(n)) = 1 - dist(Emapping_l, P(n)) / dmax and Wij is a learned synaptic weight.

11. Articulated robotic arm device according to any one of claims 1 to 10, wherein another secondary spring is added to learn the minimum force required to initiate movement of the robotic arm and overcome dry friction forces, in order to directly generate this force when the robotic arm stops approaching the target position.

12. Articulated robotic arm device according to any one of claims 1 to 11, wherein the high-level controller is trained to learn a current resting elongation S0 - for each joint i and each transition - as a function of the desired movement dynamics, the movement being discretized into different passage points which constitute target transient positions for which specific rest elongations 0OH are determined, This learning process follows the order of transition points, from state Ej to state Ej+1, Each state Ej in this case corresponds to a transition between two target positions depending on the direction of movement so that the robotic arm can learn to take into account the effects related to its inertia and the dynamics of the movement. and once trained, the Ti control command comprising at least one main sub-command: Kri . (0Ori / Ej (n) - 0mi(u)), and allows the joints i of the robotic arm to be moved towards the target transient position 0i_ _transitoire_cible / Ej .

13. Articulated robotic arm device according to any one of the preceding claims 1 to 12, wherein the high-level controller is driven, via an adaptation mechanism, during movement learning or in the usage phase, to a state Ej, , with the following control command for the low-level unit: n(u) = Kri. (0Ori / Ej (n)- 0mi(u)) + Kr'i. (0Or'i / Ej(n) - 0mi(u)) the high-level controller using the modeling of an additional elongation 0ori / Ej(n) of at least one virtual secondary spring r'i per joint i and located in series with the main virtual spring n, to allow from the measured position 0mi(u), to move the robotic arm towards the target position 0i_target / Ej.

14. Articulated robotic arm device according to any one of claims 4 to 13, wherein the computing and storage unit implements, when the robotic arm is in dynamic operation and moves between several states Ej, a dynamic error correction mechanism using the error measured at the last iteration of the movement aimed at reaching the position associated with a state Ej to control, for each joint i, a secondary dynamic error correction virtual spring, located in series with the main virtual spring, This secondary virtual spring opposes or complements the effect of the primary virtual spring by proposing a correction term that integrates, through iterative corrections, the effect of inertias during the reproduction of the same movement. the dynamic error correction mechanism in which the control unit performs the following summation of the main control sub-command and the sub-command of the secondary spring associated with the dynamic adaptation: n(u) = Kri. (0Ori / Ej (n)- 0mi(u)) + Kr'ï. (0O_dyn_i / Ej (n) - 0mi(u)) With n a natural number corresponding to the last programmed iteration to reach the target position Ej, The update of 0o_dynj / Ej (n) is performed at each iteration n using: 0O_dyn_i / Ej (n) = Lr”_max_i . f(Wdyn_ij(n) . Ej(n)) The elongation 0o_dynj / Ej of this spring is modified so that at the end of a movement segment between two states Eji and Ej2, the joint position of the joint considered gets closer and closer to the target joint position Ej2, according to the learning of the high-level controller as follows: dWdynjj(n) = Ri(n) . Ej(n) and Wdyn_ij(n+1) = Wdynjj(n) + Slearn. dWdyn_ij(n) with Ri(n) = 0cibiej(n) - 0mi(u) 15. Articulated robotic arm device according to any one of the preceding claims 1 to 14, wherein the low-level controller sums, from the measured positions 0mi, during the movement of the robotic arm: - the sub-control of elongation and - an additional sub-command to help synchronize the movements of the different joints by limiting the speed of each joint i as a function of the difference between a real measured speed of the joint and a target speed for said joint i, which is written: T'vi(u) = K'vi . (Vcibiej (n) - Vmi (u)) Where Vmi is the time derivative of 0mi, if this real measured speed Vmi is greater than the target speed Vcibiej (n).

16. Articulated robotic arm device according to any one of the preceding claims 1 to 15, wherein the low-level controller sums during movement, from the measured positions 0mi: - the main sub-command for elongation control and - a speed limiting sub-command for each joint i, which is written rvi(u) = Kvi . (VthresholdJ (n) - Vmi(u)) Where Vmi is the time derivative of 0mi, and which is activated if the measured velocity V m of the corresponding joint i is greater than a threshold velocity Vseuiij (n).

17. Articulated robotic arm device according to claim 15 or 16, wherein during movement between two target positions A and B associated respectively with a starting state and an ending state, The high-level controller uses the learned rest elongations of the two states to calculate new rest elongations corresponding to their weighting during movement to control the speed of the robotic arm Vt which is either the target speed Vcibiej (n) , or the threshold speed Vseuiij (n) during its movement along the trajectory AB.

18. Articulated robotic arm device according to claim 17, wherein the human-machine interface uses the following state activity discretization equation: Es(n) = (n-no) / NAB and EA(H) = 1 - Es(n), with n = no at the last iteration of A and NAB is the number of iterations to go from A to B, calculated as a function of the computation period T2 of the high-level controller: NAB = (dist(A,B) / Vt) / T2 in order to provide the high-level controller with the values ​​EA(H) and Es(n), which uses them in the equation 0ori / E(n) = Lr_max_i . f(Sj Wij(n) . Ej(n)), which becomes 0ori / E(n) = Lr_max_i . f( WiA .EA(H) + WiB .EB(H)) so that: - to reach, at each iteration n from no to no + NAB, a new equilibrium position between A and B; - to control the speed of each joint i by imposing a speed greater than or equal to a target speed, the low-level controller also using a speed limiting sub-command for each joint i, which is written Tvi(u) = Kvi . (VthresholdJ (n) -Vmi(u)) Where Vmi is the derivative of 0mi, and which activates if the measured velocity V m of the corresponding joint i is greater than a threshold velocity Vseuiij(n) which is equal in this case to Vt.

19. A robotic arm device articulated according to any one of claims 4 to 18, wherein the stiffnesses Kri(n) are learned and tuned by the neural network, via the reinforcement learning mechanism, the learning rate 8 of the elongation 0ori(n) being faster than the learning rate 8' of the stiffness Kri(n), the neural network modifying the stiffnesses Kn of the virtual springs, with: Kri / E(n) = Kr_max_i. f(Sj W'ij(n). Ej(n)), with: dWij(n) = Ej(P(n)) . K ri / Ej(n) . R'i(n) R'i(n) = f((0i_cibie / Ej(n) - 0mi(n)) . a1) - f((0mi(n) - 0i_cibie / Ej(n)) . a1), a1 being chosen so that the reinforcement signal saturates for an angular difference greater than a given angular threshold, Wij(n+1) = W'ij(n) + 8' dWij(n) with 8' the learning rate between 0 and 1. f being a function of the activation of neurons having, by convention, for example, output values ​​between 0 and 1.

20. Articulated robotic arm device according to any one of the preceding claims, wherein the elongations are angular, or linear.

21. Articulated robotic arm device according to any one of the preceding claims, wherein the system is without a mechanical system for achieving impedance such as springs, and / or without sensors for measuring forces, and / or without a disengagement mechanism triggered by an over-torque.

22. Articulated robotic arm device according to any one of the preceding claims, wherein the robot is a collaborative robot or cobot.

23. Articulated robotic arm device according to any one of the preceding claims 3 to 22, wherein: -T1 is less than or equal to 10ms, advantageously equal to or less than 1ms -T2 is less than or equal to 50ms, advantageously equal to or less than 10ms.

Images