Method and system for controlling a robotic system

Abstract

Method for controlling a slave robotic system Method for controlling a slave robotic system by a master control system comprising at least one vibrator, in which: - at least one quantity F is determined as a function of at least one force and / or at least one torque exerted or experienced by the slave robotic system during its interaction with an environment in which it operates; - a vibration feedback signal is calculated from this quantity and a predefined control law, which can take at least three levels depending on the value of said quantity; - the vibrator is controlled as a function of this signal so as to generate at least three corresponding levels of vibration intensity felt by the user. Figure for the abstract: Fig. 1

Description

Title of the invention: Method and system for controlling a robotic system Technical field

[0001] The present invention relates to robotic teleoperation. Prior art

[0002] Robotic teleoperation consists of entrusting control of a robot, called a "slave", to a human operator, by means of a control system, called a "master", the latter being able to be a remote control arm or a motion capture tool such as a joystick, among other devices.

[0003] For a number of operations in hazardous (e.g. nuclear) or inaccessible areas, or in complex environments where automation is not cost-effective or is difficult to envisage, teleoperation is an effective solution.

[0004] Force feedback teleoperation allows the operator to evaluate the forces applied by the robot, thus enabling him to better manage potential collisions, or expected interaction forces (for assembly or finishing, for example).

[0005] US patent application 2017 / 0108929 and the Park et al. article “A Tele-operation Interface with a Motion Capture System and a Haptic Glove” disclose haptic glove motion capture systems for virtual reality applications with haptic feedback in the form of vibrations when the system detects that the user is coming into contact with a virtual object.

[0006] In the article Ju et al. “Teleoperation of Humanoid Baxter Robot Using Haptic Feedback”, a teleoperation system using a Baxter robot is proposed, which includes haptic feedback enabling so-called bilateral manipulation of the teleoperation system. This article presents a position-position control strategy with force feedback where the slave robot arm moves according to the master's movement. A workspace mapping method is proposed, facilitating the possibility of matching the workspace boundaries of the master and the slave. An algorithm ensures the accuracy and position transformation between master and slave. A haptic rendering algorithm is designed for haptic feedback interaction.

[0007] A reactive motion capture system that combines a haptic feedback motion capture system and a human-scale virtual environment is proposed in the article Choi et al. “A Development and Evaluation of Reactive Motion Capture System with Haptic Feedback”.

[0008] The article Pacchierotti et al. “Teleoperation of Steerable Flexible Needles by Combining Kinesthetic and Vibratory Feedback” presents a teleoperation system for steering flexible surgical needles, in which vibrations on a master remote control arm are used to indicate a needle orientation direction in the slave system's coordinate system. The proposed teleoperation system tracks the needle position using an ultrasound imaging system and calculates the ideal needle position and orientation to reach a given target. A haptic interface then provides the clinician with mixed kinesthetic and vibratory navigation cues to guide the needle to the calculated ideal orientation and position.

[0009] The Springer journal Girbés-Juan et al. “Combining haptics and inertial motion capture to enhance remote control of a dual-arm robot” discloses a bimanual teleoperation system with a force-feedback remote control arm.

[0010] US patent application 2006 / 0290662 discloses a haptic feedback system with different vibration patterns. A wide variety of actuator types can be used to provide synchronized vibration, including linear actuators, rotary actuators, rotating eccentric mass actuators, and oscillating mass actuators. A controller can send signals to one or more driver circuits to direct the operation of the actuators. The controller can provide direction and amplitude control, vibration control, and frequency control to direct the haptic experience. Parameters such as frequency, phase, amplitude, duration, and direction can be programmed or input as different patterns suitable for use in gaming, virtual reality, and real-world situations.

[0011] US Patent 8,190,292 relates to high frequency force feedback in a robotic teleoperation system where the master controller receives force or velocity information. The controller of the telerobotic system is characterized by combining high frequency information with low frequency position or velocity information. The controller is useful for teleoperations with or without delay between the communication channels of the master and slave devices. Statement of the invention

[0012] Existing systems leave the operator unaware of the intensity of the forces he applies with the robot on its environment, which creates a difficulty during the teleoperation of the latter.

[0013] There is therefore a need to facilitate the remote operation of robotic systems and the invention meets this objective by proposing, according to one of its aspects, a method for controlling a slave robotic system by a master control system. comprising at least one vibrator, in which:

[0014] - at least one quantity F is determined as a function of at least one force Fttans and / or of at least one Trot torque exercised or undergone by the slave robotic system during its interaction with an environment in which it evolves;

[0015] - we calculate from this quantity F and a predefined control law a vibrational return signal which can take at least three levels depending on the value of said quantity F;

[0016] - the vibrator is controlled according to this vibration feedback signal so as to generate vibrations with a corresponding level of intensity felt by the user.

[0017] The invention makes it possible to deliver force information to the user, namely the intensity of vibration felt, by using said quantity F to modulate the intensity of the vibrations generated, according to the predefined control law. The vibratory feeling of the user can take, depending on the amplitude of the quantity F, at least three levels perceived differently by the user, which facilitates the remote operation of the slave system. It is therefore not a binary feeling, vibrator activated or not, but a feeling which can vary in amplitude between minimum and maximum values ​​with at least one intermediate level.

[0018] The invention can make it possible to communicate to the operator a force or a contact concerning the slave robotic system in a few tens of milliseconds, whereas bilateral force feedback is generally constrained to higher response times.

[0019] Preferably, the vibration feedback signal is capable of taking a large number of values, in particular more than three values, for example 2n values, where n designates the number of bits on which this signal is coded. The vibrator can be controlled with as many different intensities as there are values ​​of the vibration feedback signal, but it is also possible to have control levels of the vibrator for respective ranges of values ​​of the vibration feedback signal; the number of different operating modes of the vibrator, corresponding to as many control levels, can thus be between 2n / 50 and 2n, for example. Each operating mode of the vibrator advantageously corresponds to a level of perceived intensity different from any other mode.

[0020] To vary the level of intensity felt, the vibratory amplitude of the vibrator, at constant or variable frequency, and / or the duration during which the vibrator is in operation per unit of time can be varied. For example, at least three different levels of intensity felt can correspond to a zero level (without vibration), a maximum level where the vibrator operates 100% of the time over a period of 1s and at least one intermediate level where the vibrator operates for example between 20% and 80% of the time over a 1s period, for example at 50%, alternating 250 ms operating periods and 250 ms rest periods.

[0021] Thus, in exemplary embodiments, the control of the vibrator is carried out by modulating its operating time during a given period of time, this modulation causing a modulation of the vibratory intensity felt by the user. For example, there is a PWM type control (modulation of the duty cycle).

[0022] Preferably, the control law includes a dead zone in which no vibration is triggered as long as said quantity F does not exceed a predefined triggering threshold. This limits the risk of untimely triggering of the vibrator.

[0023] The control law preferably comprises a saturation zone from which the image by this law of the quantity F no longer increases or increases less and less with the quantity F so as to reach a predefined ceiling even when said quantity F continues to increase. The saturation zone can be chosen as a function of a threshold of force or torque exerted or undergone by the robot beyond which the operator's knowledge of the amplitude of the force or torque is no longer as useful for teleoperation as lower values.

[0024] The control law may be a ReLu or sigmoid type function, among other possible functions, preferably having a relatively steep slope, which may be adjusted so as to give a maximum variation in vibration amplitude felt by the user for a given force or torque range.

[0025] Preferably, said quantity F is given by the weighted sum of the norms of the force components F^ and of the torque Trot: F = + fi ■ |7\J.

[0026] The robotic system may comprise an actuator, such as a gripper, and —__L_, L Lpincè clamp being a characteristic dimension of the clamp, in particular the length of the lever arm of the clamp, which may correspond substantially to the length of the branches of the clamp.

[0027] The characteristic dimension of the gripper may be of the same order of magnitude as the dimensions of the object being handled.

[0028] In one embodiment, P = 0, which does not prevent maintaining a feeling consistent with the efforts encountered by the slave system.

[0029] The maximum felt vibration intensity is preferably Vmax = C*Fmax, c being a scaling coefficient, the control law preferably being calibrated such that the maximum intensity is felt when said quantity F reaches a predefined value Fmax.

[0030] The master control system may include a user motion capture system, the vibrator then equipping, for example, a telephony device. control in the form of a joystick called a “controller”, independent, manipulated by the user, the movements of this joystick being detected for example optically and / or using an inertial unit or an accelerometer integrated into it.

[0031] Alternatively, the master control system comprises a master remote control arm or a haptic glove.

[0032] In one embodiment, in the event of detection of a coupling control signal originating from a user action on an interface, for example a button present on the aforementioned joystick or arm, the set position of the slave robotic system is reset to its current position.

[0033] Control system and teleoperation installation

[0034] The invention also relates, according to another of its aspects, to a control system to be connected to a command system comprising a system for capturing the movement of the user equipped with a vibrator, the control system being intended to control a slave robotic system, the control system being configured to implement the method according to the invention as defined above, and being arranged to: - determine at least one quantity F as a function of at least one force Ftrans and / or at least one torque Trot exerted or undergone by the slave robotic system during its interaction with the environment in which it operates; - calculate, from this quantity F and a predefined control law, a vibrational feedback signal, which can take at least three levels depending on the value of said quantity; - control the vibrator according to this signal, so as to generate vibrations with a corresponding level of intensity felt by the user.

[0035] The invention also relates, according to another of its aspects, to a teleoperation installation comprising the control system according to the invention, the control system comprising the vibrator, and the slave robotic system. Brief description of the drawings

[0036] The invention may be better understood by reading the detailed description which follows, of non-limiting examples of its implementation, and by examining the attached drawing, in which:

[0037] [Fig.l] [Fig.l] is a schematic view of an example of a teleoperation installation according to the invention; and

[0038] [Fig.2] [Fig.2] graphically illustrates examples of control laws. Detailed description

[0039] [Fig.l] schematically illustrates an example of a teleoperation installation 1 according to the invention.

[0040] This installation 1 comprises a control system 2 and a slave robotic system 3 operated remotely by the control system 2 via a control system 4. Robotic system

[0041] The slave robotic system 3 may be any robot, for example a six-axis robotic arm, or any other number of joints.

[0042] The slave robotic system 3 may be equipped with any actuator, effector or gripping mechanism. It comprises for example, as illustrated, a gripper 5 which can serve as a gripping member.

[0043] The robotic system 3 is equipped with force and / or torque sensors which make it possible to measure the force exerted or experienced by the robotic system on its environment. The robotic system 3 comprises, for example, sensors which make it possible to measure the clamping force of the gripper 5. It may also comprise torque sensors at the joints, which make it possible to know the forces exerted or experienced by the different segments of the arm. Control system

[0044] The control system 2 is used to control the robotic system 3 remotely via the control system 4.

[0045] It comprises a motion capture system 25 manipulated by the user.

[0046] This capture system 25 may comprise a controller, as illustrated.

[0047] The controller may integrate an inertial unit and / or an optical or radiofrequency detection system for detecting its position and movements when manipulated by the user. Suitable controllers, such as those marketed under the HTC VIVE or HP brand, exist on the market, which integrate one or more cameras for detecting movements of the controller within their environment; other systems exist which include one or more fixed cameras and luminous markers on the controller, which are detected by this or these cameras.

[0048] The control system 2 may also include a remote control arm, for example of the joystick or remote control arm type.

[0049] The invention is not limited to a particular control system and the latter may be presented for example in a different form, for example a glove worn by the user.

[0050] The motion capture system comprises a vibrator 20. This vibrator 20 comprises, for example, a motor and a weight forming an unbalance. The vibrator may also be piezoelectric or any other electromechanical device. Control system

[0051] The control system 4 comprises: - a go chain which allows you to send the command information to the robot: • a shaping module 200 which receives information relating to the movement and / or position of the capture system 25, • an operational module 201 receiving from the formatting module 200 processed data relating to the manipulation of the capture system, as well as coupling information Co, • a control module 202 receiving from the operational module 201 information relating to the actions to be carried out by the robot, including a position instruction Xcons. - a return chain which allows the control of the vibrator 20, and which includes: • a processing module 300, and • a conversion module 301.

[0052] These modules correspond to functionalities of the control system 4 which can be implemented by any suitable computer system, comprising for example one or more microcomputers and / or microcontrollers and / or specialized circuits, as well as all necessary interfaces.

[0053] Several modules can correspond to blocks of a program executed on the same hardware entity.

[0054] The control system 4 is shown in the figure as a separate block from the command system 2, but all or part of the control system may, if necessary, be integrated into the joystick or other capture system 25 used.

[0055] The module 200 receives from the capture system 25 information on the latter's position or speed, in this case its current position X^i^. This data undergoes calibration, in a manner known per se, in order to match given movements of the capture system 25 with the desired corresponding movements of the robot 3.

[0056] The module 201 receives from the capture system 25 a coupling information Co; this information can be a binary information, generated for example by pressing a coupling button. For example, if the information Co is true, which corresponds to the absence of pressing the coupling button for example, the calibrated position information X^^ is processed by the module 201 which emits the instruction Xcons. This instruction Xcons is transmitted to the robotic control module 202 which ensures that the robot 3 follows these instructions.

[0057] If Co is false, which corresponds for example to pressing the coupling button, the set position of the robot 3 is repositioned to the current position of the capture system 25, which can allow the contact forces to be released quickly and intuitively. Indeed, upon contact, forces are generated when the set position is displaced beyond the contact surface, the stiffness of the control (the 'spring' component) then inducing an interaction force. By resetting this position, the forces are then cancelled (at least, in the case of rigid materials). This functionality offers a quick possibility to reset a contact when the operator is no longer able to assess the direction of the interaction forces (which are not represented by the vibration feedback).

[0058] The force information transmitted by the sensors of the robotic system 3 to the module 300 relates to at least one force and / or torque Trot exerted or undergone by the robotic system 3, in particular a force torque on the gripper 5, during its interaction with the environment.

[0059] These data are processed by the module 300, in particular to filter them, because the force and / or joint torque sensors of the robot can emit noisy data.

[0060] The filtered signal is then converted into a scalar value by the conversion module 301.

[0061] This module 301 calculates in step 31 a quantity F which is a function of at least the force F^ and / or at least the torque Trot, then calculates in step 32 from this quantity F and a predefined control law a vibration feedback signal.

[0062] This vibration feedback signal is processed in step 33 to be transformed into a control signal capable of taking at least three levels depending on the value of said quantity F, so as to generate at least three corresponding levels of vibration intensity felt by the user, F being expressed by:

[0063] F = Ka-\F,„„]+p-\Tm,n.

[0064] It is preferable to fix a=l and to adapt [3 according to the dimensions of the clamp 5 (which conditions the torques felt during contacts). For example, we have ___!__, LL clamp clamp being the lever arm length of the clamp.

[0065] Then, the scalar value F is scaled by a coefficient c equal to vmax! Pmax with Vmax the maximum felt vibration intensity, and Fmax the maximum force standard estimated as being relevant to distinguish (beyond Fmax, saturation blocks the vibration at its maximum intensity). This coefficient c reflects in particular the sensitivity of the haptic feedback and the order of magnitude of the forces that we seek to express.

[0066] The control law can be an activation function, of the Rectified Linear Unit (ReLu) or sigmoid type, depending on the desired feeling.

[0067] Several examples of ReLu and sigmoid functions are given in [Fig.2].

[0068] The graphs represent on the ordinate the duration of haptic pulsation, which can correspond to the duration of activation of the vibrator, for example in ps as a function of the quantity F, for example expressed in N, the numerical values ​​indicated being purely indicative and may be different depending on the applications.

[0069] The first line of graphs illustrates a ReLu function, the second line a sigmoid, the third line a steep sigmoid, and the fourth line a flat sigmoid.

[0070] The first column represents each of these functions with a dead zone 50 and a saturation zone 60.

[0071] The second column represents the ReLu function without the saturation zone.

[0072] The third column represents the ReLu function without dead zone or saturation and the other sigmoid functions without the dead zone.

[0073] The dead zone 50 corresponds to a range of values ​​of the quantity F in which no vibration is triggered as long as said quantity F does not exceed a predefined trigger threshold Fmin.

[0074] The saturation zone 60 corresponds to a range of values ​​of the quantity F from which the duration of haptic pulsation no longer increases or increases less and less so as to remain below a predefined ceiling even when said quantity F continues to increase.

[0075] An overall gain of the control system can be adapted (keeping the saturation) to feel low forces and more subtle force variations. The gain can be increased and an offset from zero introduced in the activation function can be introduced to obtain a feeling of low variations for higher force values.

[0076] Beyond the gain modification, the operator can also learn to gauge equivalences between vibration intensity and real effort. In practice, this is quite intuitive; for example, it is sufficient to manipulate a known flexible object for a short time while visually observing its deformations.

[0077] The signal thus obtained at the output of the conversion module 301 is transmitted to the vibrator 20, giving the operator feedback on the forces applied to the clamp 5.

[0078] The vibrator 20 provides a non-binary felt intensity level v such that 0 < v < ^max.

[0079] For example, when the clamp 5 is not serving any object, the vibrator 20 remains inactive. When the clamp 5 applies the maximum tightening, the vibration felt is maximum. When the tightening torque is intermediate, the level of vibration felt by the user is intermediate.

[0080] The invention is not limited to the exemplary embodiments described above, and can be applied to the detection of collisions of any part of the slave robotic system by equipping the robotic system with joint torque sensors, as opposed to slave robots equipped with force sensors on the effector where only the forces applied by the effector are detected. 10

Claims

Claims

1. Non-surgical method for controlling a slave robotic system (3) by a master control system (2) comprising at least one vibrator (20), in which: - at least one quantity F is determined (31) as a function of at least one force Ftrans and at least one torque Trot exerted or undergone by the slave robotic system (3) during its interaction with an environment in which it evolves; - a vibration feedback signal is calculated (32) from this quantity F and a predefined control law, which signal can take at least three levels depending on the value of said quantity F; - the vibrator (20) is controlled (33) as a function of this vibration feedback signal, so as to generate vibrations with a corresponding level of intensity felt by the user.

2. Method according to the preceding claim, the control law comprising a dead zone in which no vibration is triggered as long as said quantity F does not exceed a predefined trigger threshold (Fmin).

3. Method according to one of the two preceding claims, the control law comprising a saturation zone from which the image by this law of the quantity F no longer increases or increases less and less with the quantity F so as to reach a predefined ceiling even when said quantity F continues to increase.

4. Method according to any one of the preceding claims, the control law being a ReLu or sigmoid type function.

5. Method according to any one of the preceding claims, the control of the vibrator (2) being carried out by modulating its operating time during a given period of time, this modulation causing a modulation of the vibratory intensity felt by the user.

6. Method according to any one of the preceding claims, said quantity F being given by the weighted sum of the norms of the force components F^ and of the torque Trot: F = \a- \FTrans \+ / F | T mt 11.

7. Method according to the preceding claim, the slave robotic system (3) comprising an actuator, such as a gripper (5), and — —1_, Lgripper Lptece being a characteristic dimension of the gripper.

8. A method according to any preceding claim, the maximum felt vibration intensity being = C*Fmax, c being a scaling coefficient, the control law preferably being calibrated such that the maximum intensity is felt when said quantity F reaches a predefined value Fmax.

9. A method according to any one of the preceding claims, the master control system (2) comprising a system (25) for capturing user movement.

10. Method according to any one of the preceding claims, in which in the event of detection of a coupling control signal originating from a user action on an interface, the set position of the slave robotic system (3) is reset to its current position.

11. Control system (4) to be connected to a control system (2) comprising a system (25) for capturing the movement of the user equipped with a vibrator (20), the control system (4) being intended to control a slave robotic system (3), the control system (4) being configured to implement the method according to any one of the preceding claims and being arranged to: - determine (31) at least one quantity F as a function of at least one force Ftrans and at least one torque Trot exerted or undergone by the slave robotic system (3) during its interaction with the environment in which it evolves; - calculate (32), from this quantity F and a predefined control law, a vibration feedback signal which can take at least three levels as a function of the value of said quantity;- control (33) the vibrator (20) according to this signal so as to generate vibrations with a corresponding level of intensity felt by the user.;

12. Installation (1) for remote operation comprising the control system (4) according to the preceding claim, the control system (2) comprising the vibrator (20) and the slave robotic system (3).

Images