Robot remote control method, device, electronic device, storage medium and system

Abstract

The present disclosure provides a robot remote control method, device, electronic device, storage medium and system. The robot remote control method of the embodiment of the present disclosure includes: obtaining a first voltage signal from a strain gauge and first posture data from an inertial measurement unit, the first voltage signal is a voltage signal generated by the strain gauge after sensing that the user's first finger performs an action when the user wears a smart glove, and the first posture data is posture data measured by the inertial measurement unit after the user's arm performs an action when the user wears the smart glove; controlling the first finger of the manipulator to perform a first action so that the angle of the first finger of the manipulator relative to the palm of the hand corresponds to the first voltage signal; and controlling the manipulator to perform a second action according to the first posture data. The present disclosure has the advantages of simple operation, high accuracy of manipulator control, and low hardware cost.

Description

Technical Field

[0001] The present disclosure relates to the field of robot technology, and in particular to a robot remote control method, device, electronic device, storage medium and system. Background Art

[0002] The combination of a robotic arm and a manipulator is usually called a robot. Robots are widely used in various fields such as warehousing and logistics, surgery, medical rehabilitation, processing and manufacturing, energy and mineral collection, etc. Therefore, the control method of the robot is very important.

[0003] At present, the remote control of robots is mainly completed through large devices such as computers and servers. This remote control method is not only complex and difficult to implement, but also has high cost and poor control accuracy. In view of this, it is necessary to provide a robot remote control solution that is simple to operate, easy to implement and low in cost. Summary of the invention

[0004] In view of this, the present disclosure provides a robot remote control method, device, electronic device, storage medium and system.

[0005] According to a first aspect of the present disclosure, a robot remote control method is provided, wherein the robot comprises a robot arm and a robot hand connected to each other; the method comprises:

[0006] Acquire a first voltage signal from a strain gauge and first posture data from an inertial measurement unit, wherein the strain gauge is installed in a first finger area of ​​a smart glove, the first voltage signal is a voltage signal generated by the strain gauge after sensing that a first finger of a user performs an action when the user wears the smart glove, the inertial measurement unit is installed on the smart glove, and the first posture data is posture data measured by the inertial measurement unit after an arm of a user performs an action when the user wears the smart glove;

[0007] Controlling a first finger of the manipulator to perform a first action so that the bending angle of the first finger of the manipulator relative to a palm corresponds to the first voltage signal;

[0008] The robotic arm is controlled to perform a second action according to the first posture data.

[0009] In some possible implementations of the first aspect of the present disclosure, controlling the first finger of the manipulator to perform a first action so that the angle at which the first finger of the manipulator bends relative to the palm corresponds to the first voltage signal includes:

[0010] Determining a first angle according to a pre-calibrated voltage-angle mapping relationship and the first voltage signal;

[0011] The first finger of the manipulator is controlled to perform a first action so that the angle of the first finger of the manipulator relative to the palm of the manipulator is the first angle.

[0012] In some possible implementations of the first aspect of the present disclosure, the method further includes: collecting a voltage signal generated by the strain gauge after the first finger of the user performs bending movements at multiple angles when the user wears the smart glove; and fitting the voltage-angle mapping relationship based on the voltage signals of the strain gauge corresponding to the multiple angles.

[0013] In some possible implementations of the first aspect of the present disclosure, the first finger includes one or more of the following: thumb, index finger, middle finger, ring finger, little finger; the first finger area is: the area between the intermecarpal joint and the metacarpophalangeal joint, or the area between the metacarpophalangeal joint and the interphalangeal joint; the first action is finger extension or finger flexion.

[0014] In some possible implementations of the first aspect of the present disclosure, the second action includes one or more of the following: wrist rotation, arm flexion and extension, and arm adduction and abduction.

[0015] In some possible implementations of the first aspect of the present disclosure, the first posture data includes a roll angle, a pitch angle and a heading angle; and controlling the robotic arm to perform the second action according to the first posture data includes: verifying whether the roll angle in the first posture data is within a predetermined roll angle valid range, whether the pitch angle is within a predetermined pitch angle valid range, and whether the heading angle is within a predetermined heading angle valid range; if the roll angle in the first posture data is within the roll angle valid range, the pitch angle is within the pitch angle valid range, and the heading angle is within the heading angle valid range, then controlling the robotic arm to perform the second action according to the first posture data; if the roll angle in the first posture data is not within the roll angle valid range, the pitch angle is not within the pitch angle valid range, or the heading angle is not within the heading angle valid range, then controlling the robotic arm not to perform any action.

[0016] In some possible implementations of the first aspect of the present disclosure, the effective range of the roll angle is determined according to the maximum roll angle and the minimum roll angle measured by the inertial measurement unit when the user wears the smart glove and performs a wrist rotation action; and / or, the effective range of the pitch angle is determined according to the maximum pitch angle and the minimum pitch angle measured by the inertial measurement unit when the user wears the smart glove and performs an arm flexion and extension action; and / or, the effective range of the heading angle is determined according to the maximum heading angle and the minimum heading angle measured by the inertial measurement unit when the user wears the smart glove and performs an arm abduction and adduction action.

[0017] In some possible implementations of the first aspect of the present disclosure, controlling the mechanical arm to perform a second action according to the first posture data includes one or more of the following:

[0018] Determine whether the roll angle in the first posture data is within a first predetermined range or a second predetermined range, and if the roll angle is within the first predetermined range, control the end of the robot arm to move in the positive direction of the X-axis of the robot arm so that the robot arm performs a left-hand wrist rotation action; if the roll angle is within the second predetermined range, control the end of the robot arm to move in the negative direction of the X-axis of the robot arm so that the robot arm performs a right-hand wrist rotation action;

[0019] Determine whether the pitch angle in the first posture data is within a third predetermined range or a fourth predetermined range, and if the pitch angle is within the third predetermined range, control the end of the robotic arm to move along the positive direction of the Y axis of the robotic arm so that the robotic arm performs an arm bending action, and if the pitch angle is within the fourth predetermined range, control the end of the robotic arm to move along the negative direction of the Y axis of the robotic arm so that the robotic arm performs an arm extending action;

[0020] Determine whether the heading angle in the first posture data is within the fifth predetermined range or the sixth predetermined range. If the heading angle is within the fifth predetermined range, control the end of the robotic arm to move in the positive direction of the Z axis of the robotic arm so that the robotic arm performs an arm retraction action. If the heading angle is within the sixth predetermined range, control the end of the robotic arm to move in the negative direction of the Z axis of the robotic arm so that the robotic arm performs an arm abduction action.

[0021] In some possible implementations of the first aspect of the present disclosure, the first predetermined range and the second predetermined range are determined based on the roll angle effective range and a first predetermined threshold; and / or, the third predetermined range and the second predetermined range are determined based on the pitch angle effective range and the second predetermined threshold; and / or, the fifth predetermined range and the sixth predetermined range are determined based on the heading angle effective range and the third predetermined threshold.

[0022] According to a second aspect of the present disclosure, there is provided a robot remote control device, comprising:

[0023] an acquisition unit, configured to acquire a first voltage signal from a strain gauge and first posture data from an inertial measurement unit, wherein the strain gauge is installed in a first finger region of a smart glove, the first voltage signal is a voltage signal generated by the strain gauge after sensing that a first finger of a user performs an action when the user wears the smart glove, the inertial measurement unit is installed on the smart glove, and the first posture data is posture data measured by the inertial measurement unit after an arm of a user performs an action when the user wears the smart glove;

[0024] a manipulator control unit, configured to control a first finger of the manipulator to perform a first action so that an angle at which the first finger of the manipulator bends relative to a palm corresponds to the first voltage signal;

[0025] A robot arm control unit is used to control the robot arm to perform a second action according to the first posture data.

[0026] According to a third aspect of the present disclosure, an electronic device is provided, comprising: one or more processors and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the above method.

[0027] According to a fourth aspect of the present disclosure, a computer-readable storage medium storing a program is provided, wherein the program includes instructions, which, when executed by one or more processors of a computing device, cause the computing device to execute the above method.

[0028] According to a fifth aspect of the present disclosure, a robot remote control system is provided, comprising: a smart glove and an electronic device, the electronic device being built-in or externally connected to the robot; the smart glove comprising a glove body and a strain gauge, an inertial measurement unit and a first communication module installed on the glove body, the strain gauge and the inertial measurement unit being electrically connected to the first communication module respectively, and the first communication module being capable of communicating with the electronic device; the electronic device comprising: a controller and a second communication module, the second communication module being electrically connected to the controller, the second communication module being capable of communicating with the first communication module, and the controller being used to execute the above method.

[0029] It can be seen from the above technical solutions that, through the embodiments of the present disclosure, the user wearing the smart gloves can control the corresponding fingers on the manipulator to make the same amplitude of movement by making finger movements; at the same time, the user wearing the smart gloves can control the manipulator to make various arm-related movements such as wrist rotation, arm flexion and extension, arm adduction and abduction, etc. The embodiments of the present disclosure are not only simple to operate, but also have high accuracy in manipulating the manipulator, and low processing complexity, and can be implemented using relatively low-cost hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 A schematic diagram of the structure of a robot remote control system provided by an embodiment of the present disclosure;

[0032] Figure 2 A physical schematic diagram of a robot remote control system according to an embodiment of the present disclosure;

[0033] Figure 3 A flowchart of a robot remote control method provided by an embodiment of the present disclosure;

[0034] Figure 4 Schematic diagram of the XYZ axis of the robot arm involved in the embodiment of the present disclosure;

[0035] Figure 5 A schematic diagram of the structure of a robot remote control device provided in an embodiment of the present disclosure;

[0036] Figure 6 A schematic structural block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0038] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. The singular forms "a", "said" and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0039] As used herein, the words "if," "if," and the like may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0040] To facilitate understanding, the robot remote control system provided in the embodiment of the present disclosure is first exemplarily described below.

[0041] Figure 1 FIG. 1 shows a schematic diagram of the structure of a robot remote control system provided by an embodiment of the present disclosure. Figure 1The robot remote control system provided by the embodiment of the present disclosure includes: a smart glove 110, a robot 120 and an electronic device 130. The smart glove 110 can communicate with the electronic device 130, and the electronic device 130 can be used to control the robot 120 to perform actions through the robot 120 remote control method described below.

[0042] See also Figure 1 As shown, the robot 120 may include a manipulator 121 and a manipulator arm 122, and the manipulator 121 is connected to the manipulator arm 122. Exemplarily, the upper end of the manipulator 121 may be movably connected to the end of the manipulator arm 122, so that the manipulator 121 and the manipulator arm 122 can better simulate the linkage movement of human arms and hands under the control of the electronic device 130.

[0043] See also Figure 1 The electronic device 130 may include a controller 131 and a second communication module 132, the second communication module 132 is electrically connected to the controller 131, the second communication module 132 can communicate with the first communication module 114 of the smart glove 110 to obtain the voltage signal from the strain gauge 112 and the posture data from the inertial measurement unit 113, and the controller 131 can be used to execute the robot 120 remote control method described below. In specific applications, the electronic device 130 can be implemented as a microcontroller 131 or other similar device that can be built into the robot 120. Alternatively, the electronic device 130 can be implemented as an external device such as a computer, a server, etc., which can control the robot 120 by connecting to the robot 120.

[0044] See also Figure 1 The smart glove 110 includes a glove body 111 and a strain gauge 112 installed on the glove body 111, an inertial measurement unit 113 (IMU, Inertial Measurement Unit) and a first communication module 114. The strain gauge 112 and the inertial measurement unit 113 are electrically connected to the first communication module 114 respectively, and the first communication module 114 can communicate with the second communication module 132 in the electronic device 130.

[0045] In some examples, the first communication module 114 and the second communication module 132 can be implemented as, but not limited to, a Bluetooth module, a short-range communication module, an Ethernet communication interface, or other similar modules to communicate wirelessly. In other examples, the first communication module 114 and the second communication module 132 can also be implemented as a communication data serial port or other electrical connection interface to communicate wiredly.

[0046] The strain gauge 112 may be a sensor element for measuring the deformation amount of a material when subjected to an external force. In a specific application, the strain gauge 112 may be arranged in the finger area of ​​the glove body 111 to convert the finger movement into a voltage signal. In one example, a strain gauge 112 may be arranged on one or more knuckles of each finger in the glove body 111, so that the strain gauge 112 may convert the flexion and extension movement of the finger into a voltage signal. In one example, strain gauges may be arranged only on a pre-selected portion of the fingers (for example, the four fingers except the thumb) to control the corresponding fingers of the manipulator 121 to perform actions.

[0047] The inertial measurement unit 113 can be arranged in the back area, palm area or wrist area of ​​the glove body 111, so that the inertial measurement unit 113 can measure the posture data of the smart glove 110 in real time when it moves with the user's arm movement.

[0048] Figure 2 A real picture of the robot remote control system provided by the embodiment of the present disclosure is shown. Figure 2 In the example, the electronic device 130 is implemented as a computer, which is electrically connected to the robot. A strain gauge is provided on each finger of the smart glove, and the strain gauge is fixed in the area between the intermetacarpal joint and the metacarpophalangeal joint of each finger of the smart glove.

[0049] The above-mentioned system architecture of the embodiment of the present disclosure is only an example. Those skilled in the art should understand that in specific applications, the robot remote control system including the smart glove 110 can also adopt any other applicable architecture, and the embodiment of the present disclosure does not limit this.

[0050] The robot remote control method provided by the embodiment of the present disclosure is described in detail below.

[0051] Figure 3 The flowchart of the robot remote control method provided by the embodiment of the present disclosure is shown. The robot remote control method provided by the embodiment of the present disclosure can be executed by an electronic device built into or connected to the robot. Figure 3 , the method of the embodiment of the present disclosure may include:

[0052] Step 301, obtaining a first voltage signal from a strain gauge and first attitude data from an inertial measurement unit;

[0053] The strain gauge is installed in the first finger area of ​​the smart glove, the first voltage signal is a voltage signal generated by the strain gauge after sensing that the first finger of the user performs an action when the user is wearing the smart glove, the inertial measurement unit is installed on the smart glove, and the first posture data is posture data measured by the inertial measurement unit after the user's arm performs an action when the user is wearing the smart glove;

[0054] Step 302, controlling a first finger of the manipulator to perform a first action so that the bending angle of the first finger of the manipulator relative to the palm corresponds to a first voltage signal;

[0055] Step 303: Control the robotic arm to perform a second action according to the first posture data.

[0056] In a specific application, step 302 and step 303 may be performed simultaneously or sequentially. The first voltage signal and the first posture data may be obtained simultaneously or sequentially in step 301 .

[0057] Through the embodiments of the present disclosure, users wearing smart gloves can control specific fingers of the robot to simulate the actions of the user's fingers. Wearing smart gloves can control the robot arm to perform different actions by changing the hand posture. The operation is simple and easy to implement. The hardware and software development costs are low. The robot hand can be controlled to perform various actions such as releasing, grasping, and making a fist. The robot arm can be operated to perform various actions such as wrist rotation, arm flexion and extension, and arm adduction and abduction. It is suitable for complex robot operations in various fields such as warehousing and logistics, surgical operations, medical rehabilitation, processing and manufacturing, energy and mineral collection.

[0058] The specific implementation of the robot remote control method provided by the embodiment of the present disclosure is described in detail below.

[0059] In step 301, when the user wears the smart glove and the first finger of the user performs an action, the strain gauge senses the deformation of the first finger of the smart glove and generates a first voltage signal, and sends the first voltage signal to the electronic device through the first communication module.

[0060] The first voltage signal refers to the voltage signal generated by the strain gauge on the first finger. The first finger refers to one or more fingers of the user wearing the smart glove who is currently making an action. The first finger may include but is not limited to one or more of the thumb, index finger, middle finger, ring finger, and little finger. The strain gauge may be set in the area between the metacarpal joint and the metacarpophalangeal joint of the first finger or in the area between the metacarpophalangeal joint and the interphalangeal joint. When the first finger of the smart glove makes a flexion and extension movement as the first finger of the user, the strain gauge on the first finger is deformed, and the deformation is sensed to generate a voltage signal.

[0061] For example, if the user wearing the smart gloves is currently making a fist movement, which involves bending five fingers, then the first finger includes five fingers, and the first voltage signal in step 301 may include a voltage signal generated by a strain gauge on each of the five fingers.

[0062] In step 301, when the user wears the smart glove, the hand wearing the smart glove is driven by the movement of the user's arm. At this time, the inertial measurement unit can measure the posture data of the hand wearing the smart glove in real time and send the posture data to the electronic device through the first communication module to control the mechanical arm to perform the action. In step 301, the first posture data refers to the posture data of the hand wearing the smart glove measured by the inertial measurement unit at the current moment, that is, the posture data of the smart glove at the current moment, which is consistent with the posture data of the arm connected to the hand wearing the smart glove. In some examples, the first posture data may include roll angle, pitch angle and heading angle.

[0063] For example, when a user wears smart gloves, if the arm connected to the hand wearing the smart gloves at the current moment simultaneously performs wrist rotation and arm flexion and extension movements, in step 301, the first posture data can represent the posture of the smart glove (i.e., the end of the arm) after the wrist rotation and arm flexion and extension movements are performed.

[0064] In step 302, the first angle can be determined according to the pre-calibrated voltage-angle mapping relationship and the first voltage signal, and the first finger on the manipulator is controlled to perform a first action so that the angle of the first finger of the manipulator relative to the palm of the manipulator is the first angle.

[0065] The first action may be a finger flexion and extension action. Specifically, if the angle of the first finger of the manipulator relative to the palm of the manipulator is less than the first angle before the first action is performed, the first finger of the manipulator may be controlled to bend so that the angle of the first finger relative to the palm of the manipulator reaches the first angle. If the angle of the first finger of the manipulator relative to the palm of the manipulator is greater than the first angle before the first action is performed, the first finger of the manipulator may be controlled to extend so that the angle of the first finger relative to the palm of the manipulator is reduced to the first angle.

[0066] For example, if the user wearing the smart gloves is currently making a fisting action, which involves bending of the five fingers, then the first finger includes the five fingers. In step 302, the angle value corresponding to the voltage signal of each finger in the first voltage signal can be queried from the voltage angle mapping relationship, thereby obtaining the first angle of each finger, and then controlling the five fingers on the manipulator to perform a bending action so that the angle of each finger on the manipulator relative to the palm of the manipulator is its first angle. It can be seen that through the embodiments of the present disclosure, each finger of the manipulator can be controlled to simulate the action of the corresponding finger of the user, so that the user can control the manipulator to make complex hand actions such as grasping and releasing by wearing smart gloves.

[0067] The voltage-angle mapping relationship refers to the mapping relationship between a voltage signal and an angle. The angle refers to the angle of the finger relative to the palm, which can usually be between 0° and 90°.

[0068] In a specific application, the voltage-angle mapping relationship may be calibrated for each finger, or different fingers may share a pre-calibrated voltage-angle mapping relationship.

[0069] In some implementations, before step 302, the method may further include: collecting voltage signals generated by the strain gauge after the first finger of the user performs bending actions at multiple angles when the user wears the smart glove; fitting the voltage-angle mapping relationship according to the voltage signals of the strain gauge corresponding to the multiple angles. The voltage-angle mapping relationship may be expressed in the form of a mapping table, a functional relationship representing a voltage-angle curve, etc. By querying the voltage-angle mapping relationship, the angle corresponding to a certain voltage signal may be determined, and the voltage signal corresponding to a certain angle may also be determined.

[0070] Exemplarily, the calibration process of the voltage-angle mapping relationship can be as follows: after the user wears the smart gloves, the five fingers perform maximum bending and stretching movements at the same time, and the maximum angle value of each finger (the angle of the finger relative to the palm is the largest when bent to the maximum extent) and the minimum angle value (for example, the angle of the finger relative to the palm is the smallest when fully stretched) are measured, and the strain gauge voltage value of each finger at the maximum angle value and the strain gauge voltage value of each finger at the minimum angle value are collected, and the voltage angle curve of each finger is fitted or a voltage angle mapping table for each finger is formed.

[0071] Given that different people's fingers have different bending degrees, the voltage-angle mapping relationship can be calibrated for different users and saved in the memory of the electronic device. When a user A wears the smart glove to remotely control the robot, the robot can be controlled by reading the voltage-angle mapping relationship of user A in the memory.

[0072] In step 303, the second action may include but is not limited to one or more of the following: wrist rotation, arm flexion and extension, arm adduction and abduction, etc.

[0073] In step 303, it can be determined whether the first posture data is valid, and if the first posture data is valid, the robot arm is controlled to perform the second action according to the first posture data. In some examples, whether the first posture data can be determined according to a preset posture valid range.

[0074] In some embodiments, the valid range of the first attitude data may include: a valid range of a roll angle, a valid range of a pitch angle, and a valid range of a heading angle. If the roll angle in the first attitude data is within the valid range of the roll angle, the pitch angle in the first attitude data is within the valid range of the pitch angle, and the heading angle in the first attitude data is within the valid range of the heading angle, it indicates that the first attitude data is valid. If the roll angle in the first attitude data is not within the valid range of the roll angle, the pitch angle in the first attitude data is not within the valid range of the pitch angle, or the heading angle in the first attitude data is not within the valid range of the heading angle, the first attitude data is invalid.

[0075] Specifically, step 303 may include: verifying whether the roll angle in the first posture data is within a predetermined effective range of the roll angle, whether the pitch angle is within a predetermined effective range of the pitch angle, and whether the heading angle is within a predetermined effective range of the heading angle; if the roll angle in the first posture data is within the effective range of the roll angle, the pitch angle is within the effective range of the pitch angle, and the heading angle is within the effective range of the heading angle, then controlling the robotic arm to perform the second action according to the first posture data; if the roll angle in the first posture data is not within the effective range of the roll angle, the pitch angle is not within the effective range of the pitch angle, or the heading angle is not within the effective range of the heading angle, then controlling the robotic arm not to perform any action.

[0076] In specific applications, the effective range of the gesture can be set by the user himself or according to the amplitude of the user's arm movement.

[0077] In some examples, the effective range of the roll angle can be determined based on the maximum roll angle and the minimum roll angle measured by the inertial measurement unit when the user wears the smart gloves and performs the wrist rotation action. Specifically, when the user wears the smart gloves, the maximum roll angle a2 measured by the inertial measurement unit when the user's wrist is rotated to the left to the maximum extent is collected, and the minimum roll angle a1 measured by the inertial measurement unit when the user's wrist is rotated to the right to the maximum extent is collected, and then the effective range of the roll angle can be determined to be the interval [a1, a2].

[0078] In some examples, the effective range of the pitch angle can be determined based on the maximum pitch angle and the minimum pitch angle measured by the inertial measurement unit when the user performs arm flexion and extension after wearing the smart gloves. Specifically, when the user wears the smart gloves, the minimum pitch angle b1 measured by the inertial measurement unit when the user's arm is extended to the maximum extent is collected, and the maximum pitch angle b2 measured by the inertial measurement unit when the user's arm is bent to the maximum extent is collected, and the effective range of the pitch angle can be determined to be the interval [b1, b2].

[0079] In some examples, the effective range of the heading angle is determined based on the maximum heading angle and the minimum heading angle measured by the inertial measurement unit when the user wears the smart gloves and performs the arm abduction and adduction action. Specifically, when the user wears the smart gloves, the minimum heading angle c1 measured by the inertial measurement unit when the user's arm is adducted to the maximum extent is collected, and the maximum heading angle c2 measured by the inertial measurement unit when the user's arm is abducted to the maximum extent is collected, and the effective range of the heading angle can be determined to be the interval [c1, c2].

[0080] Further, step 303 may include one or more of the following:

[0081] 1) Determine whether the roll angle in the first posture data is within the first predetermined range or the second predetermined range. If the roll angle is within the first predetermined range, control the end of the robot arm to move in the positive direction of the X-axis of the robot arm so that the robot arm performs a left-hand rotation of the wrist. If the roll angle is within the second predetermined range, control the end of the robot arm to move in the negative direction of the X-axis of the robot arm so that the robot arm performs a right-hand rotation of the wrist.

[0082] 2) determining whether the pitch angle in the first posture data is within a third predetermined range or a fourth predetermined range, and if the pitch angle is within the third predetermined range, controlling the end of the robotic arm to move along the positive direction of the Y axis of the robotic arm so that the robotic arm performs an arm bending action, and if the pitch angle is within the fourth predetermined range, controlling the end of the robotic arm to move along the negative direction of the Y axis of the robotic arm so that the robotic arm performs an arm extending action; wherein,

[0083] 3) Determine whether the heading angle in the first posture data is within the fifth predetermined range or the sixth predetermined range. If the heading angle is within the fifth predetermined range, control the end of the robotic arm to move in the positive direction along the Z axis of the robotic arm so that the robotic arm performs an arm retraction action. If the heading angle is within the sixth predetermined range, control the end of the robotic arm to move in the negative direction along the Z axis of the robotic arm so that the robotic arm performs an arm abduction action.

[0084] Figure 4 A schematic diagram of the XYZ axis of the robot arm is shown. The X axis is a coordinate axis parallel to the cross section of the robot arm, the Y axis is parallel to the center axis of the robot arm, and the Z axis is a coordinate axis perpendicular to the X axis and the Y axis.

[0085] The first predetermined range and the second predetermined range can be determined according to the roll angle effective range and the first predetermined threshold. For example, if the roll angle effective range is the interval [a1, a2], the first predetermined threshold is set to a0, the first predetermined range can be [a1, a1+a0], and the second predetermined range can be [a2-a0, a2].

[0086] The third predetermined range and the fourth predetermined range can be determined according to the effective range of the pitch angle and the second predetermined threshold. For example, if the effective range of the pitch angle is the interval [b1, b2], the second predetermined threshold is set to b0, the third predetermined range can be [b1, b1+b0], and the fourth predetermined range can be [b2-b0, b2].

[0087] The fifth predetermined range and the sixth predetermined range can be determined according to the heading angle valid range and the third predetermined threshold. For example, if the heading angle valid range is the interval [c1, c2], the second predetermined threshold is set to c0, the fifth predetermined range can be [c1, c1+c0], and the sixth predetermined range can be [c2-c0, c2].

[0088] In a specific application, the values ​​of the first predetermined threshold a0, the second predetermined threshold b0 and the third predetermined threshold c0 may be the same or different. The specific values ​​of the first predetermined threshold a0, the second predetermined threshold b0 and the third predetermined threshold c0 may be flexibly set according to various factors such as application scenarios and different users.

[0089] In some embodiments, the first predetermined threshold, the second predetermined threshold and the third predetermined threshold The first predetermined threshold can be set as follows: when the user wears the smart gloves, the user can place the arm in a default position (the default position is within the maximum range that the user's arm can make a first action), and the inertial measurement unit on the smart gloves measures the posture data (i.e., roll angle, pitch angle and heading angle) at the default position and transmits it to the electronic device through the first communication module, and the electronic device uses the roll angle in the posture data as the first predetermined threshold, the pitch angle as the second predetermined threshold and the heading angle as the third predetermined threshold.

[0090] As can be seen from the above, through the embodiments of the present disclosure, the user wearing the smart gloves can control the corresponding fingers on the manipulator to make the same amplitude of movement by making finger movements. At the same time, the user wearing the smart gloves can control the manipulator to make various movements such as wrist rotation, arm flexion and extension, and arm adduction and abduction. The embodiments of the present disclosure are not only simple to operate, but also have high accuracy in manipulating the manipulator, and low processing complexity, and can be implemented using relatively low-cost hardware.

[0091] Figure 5 FIG. 1 is a schematic diagram showing the structure of a robot remote control device provided by an embodiment of the present disclosure. Figure 5 , the robot remote control device 500 may include:

[0092] An acquisition unit 501 is used to acquire a first voltage signal from a strain gauge and first posture data from an inertial measurement unit, the strain gauge is installed in a first finger area of ​​a smart glove, the first voltage signal is a voltage signal generated by the strain gauge after sensing that a first finger of a user performs an action when the user wears the smart glove, the inertial measurement unit is installed on the smart glove, and the first posture data is posture data measured by the inertial measurement unit after an arm of a user performs an action when the user wears the smart glove;

[0093] A manipulator control unit 502, configured to control a first finger of the manipulator to perform a first action so that an angle at which the first finger of the manipulator is bent relative to a palm corresponds to a first voltage signal;

[0094] The robot arm control unit 503 is used to control the robot arm to perform a second action according to the first posture data.

[0095] Furthermore, the manipulator control unit 502 is used to determine a first angle according to a pre-calibrated voltage-angle mapping relationship and a first voltage signal, and control a first finger on the manipulator to perform a first action so that the angle of the first finger of the manipulator relative to the palm of the manipulator is the first angle.

[0096] Furthermore, the robot remote control device 500 may also include: a calibration unit 504, which is used to collect the voltage signal generated by the strain gauge after the user's first finger performs bending movements at multiple angles when the user wears the smart gloves; and, based on the voltage signals of the strain gauges corresponding to the multiple angles, fit the voltage-angle mapping relationship.

[0097] Furthermore, the robot remote control device 500 may also include: a verification unit 505, used to verify whether the roll angle in the first posture data is within the predetermined roll angle valid range, whether the pitch angle is within the predetermined pitch angle valid range, and whether the heading angle is within the predetermined heading angle valid range; if the roll angle in the first posture data is within the roll angle valid range, the pitch angle is within the pitch angle valid range, and the heading angle is within the heading angle valid range, then the robotic arm control unit 503 is notified to control the robotic arm to perform the second action according to the first posture data; if the roll angle in the first posture data is not within the roll angle valid range, the pitch angle is not within the pitch angle valid range, or the heading angle is not within the heading angle valid range, then the robotic arm control unit 503 is not notified so that the robotic arm does not perform any action.

[0098] Furthermore, the robot remote control device 500 may also include: an effective range determination unit 506, which is used to determine the effective range of the roll angle according to the maximum roll angle and the minimum roll angle measured by the inertial measurement unit when the user wears the smart gloves and performs the wrist rotation action; and / or, it is used to determine the effective range of the pitch angle according to the maximum pitch angle and the minimum pitch angle measured by the inertial measurement unit when the user wears the smart gloves and performs the arm flexion and extension action; and / or, it is used to determine the effective range of the heading angle according to the maximum heading angle and the minimum heading angle measured by the inertial measurement unit when the user wears the smart gloves and performs the arm abduction and adduction action.

[0099] Furthermore, the robot arm control unit 503 can be specifically used for:

[0100] Determine whether the roll angle in the first posture data is within a first predetermined range or a second predetermined range, if the roll angle is within the first predetermined range, control the end of the robot arm to move in the positive direction of the X-axis of the robot arm so that the robot arm performs a left-hand wrist rotation action, if the roll angle is within the second predetermined range, control the end of the robot arm to move in the negative direction of the X-axis of the robot arm so that the robot arm performs a right-hand wrist rotation action; and,

[0101] Determine whether the pitch angle in the first posture data is within a third predetermined range or a fourth predetermined range, if the pitch angle is within the third predetermined range, control the end of the robotic arm to move along the positive direction of the Y axis of the robotic arm so that the robotic arm performs an arm bending action, if the pitch angle is within the fourth predetermined range, control the end of the robotic arm to move along the negative direction of the Y axis of the robotic arm so that the robotic arm performs an arm extending action; and,

[0102] Determine whether the heading angle in the first posture data is within the fifth predetermined range or the sixth predetermined range. If the heading angle is within the fifth predetermined range, control the end of the robotic arm to move in the positive direction along the Z-axis of the robotic arm so that the robotic arm performs an arm retraction action. If the heading angle is within the sixth predetermined range, control the end of the robotic arm to move in the negative direction along the Z-axis of the robotic arm so that the robotic arm performs an arm abduction action.

[0103] Furthermore, the robotic arm control unit 503 can also be used to: determine the first predetermined range and the second predetermined range based on the effective range of the roll angle and the first predetermined threshold; and, determine the third predetermined range and the second predetermined range based on the effective range of the pitch angle and the second predetermined threshold; and, determine the fifth predetermined range and the sixth predetermined range based on the effective range of the heading angle and the third predetermined threshold.

[0104] For other technical details about the robot remote control device 500, please refer to the section on the robot remote control method above, which will not be repeated here.

[0105] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. The program includes instructions, and when the instructions are executed by one or more processors of a computing device, the steps of the aforementioned robot remote control method are executed.

[0106] Figure 6 FIG. 1 is a schematic diagram showing the structure of an electronic device provided by an embodiment of the present disclosure. Figure 6 The electronic device 600 may include: one or more processors 601, and also includes a memory 602 storing one or more programs, which are executed by the one or more processors 601 to implement the method flow shown in the above embodiments of the present disclosure and / or program units corresponding to each unit in the device.

[0107] The various components are interconnected using different buses and can be mounted on a common motherboard or in other ways as needed. The processor 601 can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the user interface on an external input / output device (such as a display device coupled to the interface). In other embodiments, if desired, multiple processors and / or multiple buses can be used with multiple memories and multiple memories.

[0108] The processor 601 may include one or more single-core processors or multi-core processors. The processor 601 may include any combination of general-purpose processors or dedicated processors (such as image processors, application processors, baseband processors, etc.).

[0109] The memory 602 is a computer-readable storage medium provided by the present disclosure, which can be used to store non-transient software programs, non-transient computer executable programs and units, such as the following in the embodiments of the present disclosure: Figure 3 The processor 601 executes the non-transient software programs, instructions and units stored in the memory 602 to perform the above method embodiments. Figure 3 The programs, instructions and units corresponding to the robot remote control method shown.

[0110] The electronic device 600 may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0111] The input device 603 can receive input digital or character information, and generate signal input related to user settings and function control, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 605 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0112] The above-mentioned programs (also referred to as software, software applications, or codes) include machine instructions for programmable processors, and these computer programs may be implemented using object-oriented programming languages, assembly or machine languages.

[0113] With the development of time and technology, the meaning of medium is becoming more and more extensive, and the dissemination path of computer programs is no longer limited to tangible media, but can also be downloaded directly from the Internet. Any combination of one or more computer-readable storage media can be used. Computer-readable storage media can be used, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, device or device.

[0114] The technical solution provided by the present disclosure is described in detail above. The principles and implementation methods of the present disclosure are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present disclosure. At the same time, for those skilled in the art, according to the idea of ​​the present disclosure, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present disclosure.

[0115] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A robot remote control method, characterized in that: The robot comprises a mechanical arm and a mechanical hand connected to each other; the method comprises: Acquire a first voltage signal from a strain gauge and first posture data from an inertial measurement unit, wherein the strain gauge is installed in a first finger area of ​​a smart glove, the first voltage signal is a voltage signal generated by the strain gauge after sensing that a first finger of a user performs an action when the user wears the smart glove, the inertial measurement unit is installed on the smart glove, and the first posture data is posture data measured by the inertial measurement unit after an arm of a user performs an action when the user wears the smart glove; Controlling a first finger of the manipulator to perform a first action so that the bending angle of the first finger of the manipulator relative to a palm corresponds to the first voltage signal; Controlling the robotic arm to perform a second action according to the first posture data; The first posture data includes a roll angle, a pitch angle, and a heading angle; and controlling the robotic arm to perform a second action according to the first posture data includes one or more of the following: Determine whether the roll angle in the first posture data is within a first predetermined range or a second predetermined range, and if the roll angle is within the first predetermined range, control the end of the robot arm to move in the positive direction of the X-axis of the robot arm so that the robot arm performs a left-hand wrist rotation action; if the roll angle is within the second predetermined range, control the end of the robot arm to move in the negative direction of the X-axis of the robot arm so that the robot arm performs a right-hand wrist rotation action; Determine whether the pitch angle in the first posture data is within a third predetermined range or a fourth predetermined range, and if the pitch angle is within the third predetermined range, control the end of the robotic arm to move along the positive direction of the Y axis of the robotic arm so that the robotic arm performs an arm bending action, and if the pitch angle is within the fourth predetermined range, control the end of the robotic arm to move along the negative direction of the Y axis of the robotic arm so that the robotic arm performs an arm extending action; Determine whether the heading angle in the first posture data is within a fifth predetermined range or a sixth predetermined range, and if the heading angle is within the fifth predetermined range, control the end of the robotic arm to move in the positive direction of the Z axis of the robotic arm so that the robotic arm performs an arm adduction action; if the heading angle is within the sixth predetermined range, control the end of the robotic arm to move in the negative direction of the Z axis of the robotic arm so that the robotic arm performs an arm abduction action; Wherein, the first predetermined range and the second predetermined range are determined according to a predetermined roll angle effective range and a first predetermined threshold; and / or, the third predetermined range and the fourth predetermined range are determined according to a predetermined pitch angle effective range and a second predetermined threshold; and / or, the fifth predetermined range and the sixth predetermined range are determined according to a predetermined heading angle effective range and a third predetermined threshold.

2. The method according to claim 1, characterized in that The controlling the first finger of the manipulator to perform a first action so that the bending angle of the first finger of the manipulator relative to the palm corresponds to the first voltage signal, comprises: Determining a first angle according to a pre-calibrated voltage-angle mapping relationship and the first voltage signal; The first finger of the manipulator is controlled to perform a first action so that the angle of the first finger of the manipulator relative to the palm of the manipulator is the first angle.

3. The method according to claim 1, characterized in that The controlling the robotic arm to perform a second action according to the first posture data comprises: Verifying whether the roll angle in the first posture data is within the roll angle valid range, whether the pitch angle is within the pitch angle valid range, and whether the heading angle is within the heading angle valid range; If the roll angle in the first posture data is within the roll angle valid range, the pitch angle is within the pitch angle valid range, and the heading angle is within the heading angle valid range, controlling the robotic arm to perform the second action according to the first posture data; If the roll angle in the first posture data is not within the effective range of the roll angle, the pitch angle is not within the effective range of the pitch angle, or the heading angle is not within the effective range of the heading angle, the robotic arm is controlled not to perform any action.

4. The method according to claim 3, characterized in that The effective range of the roll angle is determined according to the maximum roll angle and the minimum roll angle measured by the inertial measurement unit when the user wears the smart glove and performs a wrist rotation action; and / or, The effective range of the pitch angle is determined according to the maximum pitch angle and the minimum pitch angle measured by the inertial measurement unit when the user performs arm flexion and extension after wearing the smart glove; and / or, The effective range of the heading angle is determined according to the maximum heading angle and the minimum heading angle measured by the inertial measurement unit when the user wears the smart gloves and performs arm abduction and adduction actions.

5. A robot remote control device, characterized in that: include: an acquisition unit, configured to acquire a first voltage signal from a strain gauge and first posture data from an inertial measurement unit, wherein the strain gauge is installed in a first finger region of a smart glove, the first voltage signal is a voltage signal generated by the strain gauge after sensing that a first finger of a user performs an action when the user wears the smart glove, the inertial measurement unit is installed on the smart glove, and the first posture data is posture data measured by the inertial measurement unit after an arm of a user performs an action when the user wears the smart glove; a manipulator control unit, configured to control a first finger of the manipulator to perform a first action so that an angle at which the first finger of the manipulator bends relative to a palm corresponds to the first voltage signal; a robotic arm control unit, configured to control the robotic arm to perform a second action according to the first posture data; The first posture data includes a roll angle, a pitch angle, and a heading angle; and controlling the robotic arm to perform a second action according to the first posture data includes one or more of the following: Determine whether the roll angle in the first posture data is within a first predetermined range or a second predetermined range, and if the roll angle is within the first predetermined range, control the end of the robot arm to move in the positive direction of the X-axis of the robot arm so that the robot arm performs a left-hand wrist rotation action; if the roll angle is within the second predetermined range, control the end of the robot arm to move in the negative direction of the X-axis of the robot arm so that the robot arm performs a right-hand wrist rotation action; Determine whether the pitch angle in the first posture data is within a third predetermined range or a fourth predetermined range, and if the pitch angle is within the third predetermined range, control the end of the robotic arm to move along the positive direction of the Y axis of the robotic arm so that the robotic arm performs an arm bending action, and if the pitch angle is within the fourth predetermined range, control the end of the robotic arm to move along the negative direction of the Y axis of the robotic arm so that the robotic arm performs an arm extending action; Determine whether the heading angle in the first posture data is within a fifth predetermined range or a sixth predetermined range, and if the heading angle is within the fifth predetermined range, control the end of the robotic arm to move in the positive direction of the Z axis of the robotic arm so that the robotic arm performs an arm adduction action; if the heading angle is within the sixth predetermined range, control the end of the robotic arm to move in the negative direction of the Z axis of the robotic arm so that the robotic arm performs an arm abduction action; Wherein, the first predetermined range and the second predetermined range are determined according to a predetermined roll angle effective range and a first predetermined threshold; and / or, the third predetermined range and the fourth predetermined range are determined according to a predetermined pitch angle effective range and a second predetermined threshold; and / or, the fifth predetermined range and the sixth predetermined range are determined according to a predetermined heading angle effective range and a third predetermined threshold.

6. An electronic device, characterized in that: include: One or more processors and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a program, wherein the program comprises instructions, and when the instructions are executed by one or more processors of a computing device, the computing device is caused to execute the method according to any one of claims 1 to 4.

8. A robot remote control system, comprising: Smart gloves and electronic devices, the electronic devices are built-in or externally connected to the robot; The smart glove comprises a glove body, a strain gauge mounted on the glove body, an inertial measurement unit and a first communication module, wherein the strain gauge and the inertial measurement unit are electrically connected to the first communication module respectively, and the first communication module can communicate with the electronic device; The electronic device comprises: a controller and a second communication module, the second communication module is electrically connected to the controller, the second communication module can communicate with the first communication module, and the controller is used to execute the method according to any one of claims 1 to 4.

Images