Force tactile feedback teleoperation system and method for remotely piloting a robotic manipulator

Abstract

Embodiments of the present disclosure disclose a force tactile feedback teleoperation system and a method for remotely controlling a robot. A specific embodiment of the system comprises: a force tactile feedback glove, a remotely controlled robot, a control device, wherein: the force tactile feedback glove is configured to perform the following capturing steps: capturing human hand action information; the control device is configured to perform the following mapping steps: receiving the human hand action information; generating robot mapping motion information; controlling the remotely controlled robot to make a motion corresponding to the robot mapping motion information; sending the robot mapping motion information to the remotely controlled robot; the remotely controlled robot is configured to perform the following feedback information collecting steps: collecting motion feedback information; collecting tactile information when in contact with an object during the motion; sending the motion feedback information and the tactile information to the control device; the control device is further configured to perform force tactile reproduction on the human hand. The embodiment improves user experience.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a force tactile feedback teleoperation system and a method for remotely controlling a robot. BACKGROUND

[0002] With the continuous progress of artificial intelligence technology, robots have been widely used in various aspects of life. The force tactile feedback teleoperation system is a system for controlling the movement of a robot (for example, controlling the robot to make a specified action). At present, when controlling the movement of a robot, the commonly used way is to control the movement trajectory and action of the robot through a pre-written program.

[0003] However, when the movement of the robot is controlled in the above manner, the following technical problems often exist:

[0004] First, by controlling the movement trajectory and action of the robot through a pre-written program, the action of the robot is usually fixed, and the flexibility of the movement of the robot is poor. At the same time, by controlling the movement trajectory and action of the robot through a pre-written program, the user cannot intuitively feel the interaction between the movement and operation process of the robot and the space environment, and the user experience is poor.

[0005] Continuing, in the process of using the technical solution to solve the above technical problem one, the following technical problem often exists: in the process of contacting the robot with various objects in the space environment, it is usually necessary to feed back the tactile sensation and feedback force of the robot when contacting various objects in the space environment to the hand of the person controlling the movement of the robot. The conventional solution for feeding back the tactile sensation and feedback force of the robot when contacting various objects in the space environment to the hand of the person controlling the movement of the robot is generally to install a rope binding fingers and palms in the glove controlling the movement of the robot, and to realize that the hand feels the feedback force by tensioning the rope. There are still the following problems:

[0006] Second, by installing a rope binding fingers and palms in the glove controlling the movement of the robot, and by tensioning the rope to realize that the hand feels the feedback force, there may be a non-linear relationship between the tensioning degree of the rope and the size of the feedback force received by the robot during the movement of the robot, resulting in poor authenticity of the feedback force received by the hand of the person during the movement of the robot. The installation of the rope binding fingers and palms in the glove cannot feed back the tactile sensation of the robot contacting the space objects during the movement of the robot to the hand, and at the same time, the use of the rope binding fingers and palms may bring a sense of restraint and discomfort to the user, resulting in poor user experience.

[0007] The above information disclosed in this BACKGROUND section is only for the purpose of enhancing the understanding of the background of the present inventive concepts, and therefore, it can contain information that does not form the prior art known to those of ordinary skill in the art in the country. SUMMARY

[0008] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the DETAILED DESCRIPTION. This Summary is not intended to identify key features or essential features of the claimed technology, nor is it intended to be used to limit the scope of the claimed technology.

[0009] Some embodiments of the present disclosure propose a force tactile feedback teleoperation system and a method for remotely controlling a robot to solve one or more of the technical problems mentioned in the background section.

[0010] In a first aspect, some embodiments of the present disclosure provide a force tactile feedback teleoperation system, comprising: a force tactile feedback glove, a remotely controlled robot, and a control device, wherein: the force tactile feedback glove is configured to perform the following capturing steps: capturing human hand action information; transmitting the human hand action information to the control device; the control device is configured to perform the following mapping steps: receiving the human hand action information transmitted by the force tactile feedback glove; generating robot mapping motion information based on the human hand action information; controlling the remotely controlled robot to make a motion corresponding to the robot mapping motion information, wherein the robot mapping motion information is each robot joint mapping motion information or each robot fingertip spatial coordinate information; sending the robot mapping motion information to the remotely controlled robot; the remotely controlled robot is configured to perform the following feedback information collection steps: in response to determining that the robot mapping motion information is each robot joint mapping motion information, collecting each current robot joint motion information of the robot joint where the preset robot angle sensor is located as motion feedback information through at least one preset robot angle sensor; in response to determining that the robot mapping motion information is each robot fingertip spatial coordinate information, collecting each force information of the robot fingertip where the force sensor is located as motion feedback information through at least one force sensor on the robot fingertip; collecting tactile information when contacting an object during the motion through at least one tactile feedback information collector; sending the motion feedback information and the tactile information to the control device; the control device is further configured to perform force tactile reproduction operation on the human hand through the force tactile feedback glove based on the motion feedback information and the tactile information.

[0011] In a second aspect, some embodiments of the present disclosure provide a method for remotely controlling a robot, comprising: receiving human hand action information transmitted by the force tactile feedback glove; generating robot mapping motion information based on the human hand action information; controlling the remotely controlled robot to make a motion corresponding to the robot mapping motion information.

[0012] The above various embodiments of the present disclosure have the following beneficial effects: the force tactile feedback teleoperation system of some embodiments of the present disclosure improves the flexibility of the motion of the robot and the user experience. Specifically, the reason why the flexibility of the motion of the robot and the user experience are poor is that the motion trajectory and action of the robot are controlled by a pre-written program, the action of the robot is usually fixed, the flexibility of the motion of the robot is poor, and the user cannot intuitively feel the interaction with the spatial environment during the motion and operation of the robot, and the user experience is poor. Based on this, the force tactile feedback teleoperation system of some embodiments of the present disclosure includes a force tactile feedback glove, a remote control robot, and a control device, wherein: first, the force tactile feedback glove is configured to perform the following capturing steps: capturing human hand action information; transmitting the human hand action information to the control device. Thus, the human hand action information for controlling the motion of the remote control robot can be captured by the force tactile feedback glove. Then, the control device is configured to perform the following mapping steps: receiving the human hand action information transmitted by the force tactile feedback glove; generating robot mapping motion information based on the human hand action information; controlling the remote control robot to make a motion corresponding to the robot mapping motion information, wherein the robot mapping motion information is each robot joint mapping motion information or each robot fingertip spatial coordinate information; sending the robot mapping motion information to the remote control robot. Thus, the control device can generate robot mapping motion information and control the motion of the remote control robot based on the robot mapping motion information. The remote control robot is controlled by the force tactile feedback glove and the control device to make a motion consistent with the human hand action information, improving the flexibility of the motion of the robot. Then, the remote control robot is configured to perform the following feedback information collection steps: in response to determining that the robot mapping motion information is each robot joint mapping motion information, collecting each current robot joint motion information of the robot joint where the preset robot angle sensor is located as motion feedback information through at least one preset robot angle sensor; in response to determining that the robot mapping motion information is each robot fingertip spatial coordinate information, collecting each force information of the robot fingertip where the force sensor is located as motion feedback information through at least one force sensor on the robot fingertip; collecting tactile information when contacting an object during the motion through at least one tactile feedback information collector; sending the motion feedback information and the tactile information to the control device. Thus, the motion feedback information and the tactile information of the interaction with the spatial environment during the motion of the remote control robot can be collected through at least one preset robot angle sensor or at least one force sensor. Then, the control device is further configured to perform force tactile reproduction operation on the human hand through the force tactile feedback glove based on the motion feedback information and the tactile information.Therefore, the motion feedback information and the tactile information interacting with the space environment during the motion of the remote control manipulator can be fed back to the human hand through the force tactile feedback glove by controlling the device, so that the user can intuitively feel the interaction between the motion of the remote control manipulator and the space environment, and the experience of the user is improved. BRIEF DESCRIPTION OF DRAWINGS

[0013] The above and other features, advantages and aspects of embodiments of the present disclosure will become more apparent by describing in detail some embodiments thereof with reference to the attached drawings in which:

[0014] Figure 1 is an architecture diagram of an exemplary system of the force tactile feedback teleoperation system according to the present disclosure;

[0015] Figure 2 is a flowchart of some embodiments of the remote control manipulator method according to the present disclosure;

[0016] Figure 3 is an internal test diagram of some embodiments of the force tactile feedback teleoperation system according to the present disclosure;

[0017] Figure 4 is an internal product diagram of some embodiments of the force tactile feedback glove according to the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for the sake of description, only the parts related to the present application are shown in the drawings. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0020] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0022] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0023] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Figure 1 An exemplary system architecture 100 of a force-haptic feedback teleoperation system to which some embodiments of the present disclosure may be applied is shown.

[0025] like Figure 1 As shown, the system architecture 100 may include: a force-feedback glove 102, a remote-controlled robotic arm 101, and a control device 103. The force-feedback glove 102 and the control device 103 can be connected via a communication link. The remote-controlled robotic arm 101 and the control device 103 can also be connected via a communication link. The communication link can be a physical medium or method for transmitting data, such as a wired or wireless communication link or fiber optic cable. The control device can be a computing device.

[0026] Figure 4 These are internal product diagrams of some embodiments of the force-haptic feedback glove 102 according to this disclosure.

[0027] In some embodiments, the force-feedback glove 102 described above can be configured to perform the following capture steps:

[0028] The first step is to capture hand movement information. The force feedback glove 102 includes at least one angle sensor (e.g., four angle sensors are installed on the thumb, and three angle sensors are installed on each of the remaining fingers). A fingertip spatial position sensor can also be installed at a preset position on the fingertip of each finger. A wrist spatial position sensor can also be installed in the wrist area of ​​the force feedback glove. The fingertip spatial position sensor and the wrist spatial position sensor can be, but are not limited to, one of the following: an ultrasonic tracking sensor or a GPS positioning sensor.

[0029] The second step is to transmit the aforementioned hand movement information to the aforementioned control device 103.

[0030] In some optional implementations of certain embodiments, the force-haptic feedback glove 102 described above can obtain hand movement information through the following steps:

[0031] In the first step, the joint motion information of the hand joint where each of the above-mentioned angle sensors is located is collected by each of the above-mentioned angle sensors. The joint motion information corresponds to the angle sensor identifier of the angle sensor. The joint motion information can be the angle of the hand joint at a time point. For example, the joint motion information can be "the angle of the hand joint 1 at the time point 5 seconds is 85 degrees". The hand joint 1 can be a hand joint identifier.

[0032] In the second step, the collected joint motion information is determined as the human hand motion information. Each of the at least one preset mechanical hand angle sensor corresponds to one of the above-mentioned angle sensors. Each of the current mechanical hand joint motion information corresponds to one of the above-mentioned joint motion information.

[0033] In some optional implementations of some embodiments, the force tactile feedback glove 102 can capture the human hand motion information by the following steps, including:

[0034] In the first step, the human hand wrist space point position information is collected by the wrist space position collector. The human hand wrist space point position information includes wrist space point coordinates. The human hand wrist space point position information can represent the spatial coordinates of a preset point in the human hand wrist. The wrist space point coordinates can be the spatial coordinates of the preset point.

[0035] In the second step, the fingertip space position information of the fingertip where each of the above-mentioned fingertip space position collectors is located is collected by each of the above-mentioned fingertip space position collectors, and each of the fingertip space position information is obtained. Each of the fingertip space position information includes fingertip space point coordinates. The fingertip space position information can represent the spatial position of a preset point in the fingertip of the force tactile feedback glove. The fingertip space point coordinates can represent the spatial coordinates of the preset point in the fingertip.

[0036] In the third step, the above-mentioned fingertip space position information and the above-mentioned human hand wrist space point position information are determined as the human hand motion information.

[0037] In some embodiments, the control device 103 can be configured to perform the following mapping steps:

[0038] In the first step, the human hand motion information transmitted by the force tactile feedback glove is received.

[0039] In the second step, the mechanical hand mapping motion information is generated based on the human hand motion information.

[0040] In the third step, the remote control manipulator robot is controlled to make a movement corresponding to the robot mapping movement information. The robot mapping movement information can be each robot joint mapping movement information or each robot fingertip spatial coordinate information. Each robot joint mapping movement information can represent an angle of a robot joint in space. For example, the robot joint mapping movement information can be "robot joint 1, angle 90 degrees". In practice, in response to determining that the robot mapping movement information is each robot joint mapping movement information, each robot joint in the remote control manipulator robot is controlled to make an angle represented by the corresponding robot joint mapping movement information. In response to determining that the robot mapping movement information is each robot fingertip spatial coordinate information, the remote control manipulator robot is controlled to make a movement corresponding to the robot mapping movement information by inverse kinematics technology.

[0041] In the fourth step, the robot mapping movement information is sent to the remote control manipulator robot.

[0042] In some optional implementations of some embodiments, the control device 103 can generate robot mapping movement information based on the human hand movement information by the following steps, including:

[0043] In the first step, preset human hand movement limit range information corresponding to the force tactile feedback glove and preset movement range information of the remote control manipulator robot are obtained. The preset human hand movement limit range information includes each joint movement range information of each joint, and the preset movement range information includes each robot joint movement range information of each remote control manipulator robot joint. Each joint movement range information in the preset human hand movement limit range information corresponds to each robot joint movement range information in the preset movement range information one by one, and each joint movement range information in the preset human hand movement limit range information corresponds to each joint movement information one by one. In practice, the control device 103 can obtain the preset human hand movement limit range information of the force tactile feedback glove and the preset movement range information of the remote control manipulator robot from a preset file. Each joint movement range information in the preset human hand movement limit range information can represent a bending degree range of a human hand joint. Each robot joint movement range information in the preset movement range information can represent a bending degree range of a robot joint. For example, the joint movement range information can be "0 degrees to 60 degrees". The robot joint movement range information can be "0 degrees to 60 degrees".

[0044] Secondly, for each joint motion information included in the human hand motion information, the following steps are performed:

[0045] Firstly, for each joint motion range information in the joint motion range information, the following steps are performed:

[0046] Secondly, based on the joint motion information and the target joint motion range information, joint range of motion ratio information is generated. In practice, the control device 103 can determine the absolute value of the difference between the upper limit value and the lower limit value corresponding to the target joint motion range information as a first value. Then, the control device 103 can take the joint angle corresponding to the joint motion information as a second value. After that, the control device 103 can determine the ratio of the second value to the first value as the joint range of motion ratio information. As an example, the target joint motion range information can be “0 degrees to 60 degrees”. The first value can be 60 degrees. The joint motion information can be “the angle of the hand joint corresponding to time point 5 seconds is 30 degrees”. The first value can be 30 degrees. The joint range of motion ratio information can be one half (i.e., 30 degrees / 60 degrees).

[0047] Thirdly, for each mechanical joint motion range information in the mechanical joint motion range information, the following steps are performed:

[0048] Fourthly, based on the joint range of motion ratio information and the target mechanical joint motion range information, mechanical joint mapping motion information is generated. In practice, the control device 103 can determine the absolute value of the difference between the upper limit value and the lower limit value of the target mechanical joint motion range information as a motion range span value. After that, the control device 103 can determine the product of the value corresponding to the joint range of motion ratio information and the motion range span value as a target value. After that, the control device 103 can determine the sum of the lower limit value of the target mechanical joint motion range information and the target value as the mechanical joint mapping motion information.

[0049] Thirdly, the generated each mechanical joint mapping motion information is determined as the mechanical hand mapping motion information. Wherein each current mechanical hand joint motion information in the current mechanical hand joint motion information corresponds to one mechanical joint mapping motion information in the mechanical joint mapping motion information.

[0050] In some optional implementations of some embodiments, the control device 103 can generate the mechanical hand mapping motion information based on the human hand motion information by the following steps:

[0051] Firstly, the wrist space point coordinate included in the human hand action information is determined as a reference wrist space point coordinate.

[0052] Secondly, the space coordinate of a preset point in the wrist of the robot hand is collected as a robot hand wrist space point coordinate. In practice, the control device 103 can collect the robot hand wrist space point coordinate through the wrist space position collector installed at the preset point in the wrist of the robot hand.

[0053] Thirdly, for each fingertip space position information included in the human hand action information, the following steps are performed:

[0054] Firstly, the relative space coordinate information between the reference wrist space point coordinate and the fingertip space position information is generated based on the reference wrist space point coordinate and the fingertip space position information. In practice, the control device 103 can determine the difference between the horizontal coordinate value of the reference wrist space point coordinate and the horizontal coordinate value included in the fingertip space position information as a relative horizontal coordinate difference. Then, the difference between the vertical coordinate value of the reference wrist space point coordinate and the vertical coordinate value included in the fingertip space position information can be determined as a relative vertical coordinate difference. Next, the difference between the vertical coordinate value of the reference wrist space point coordinate and the vertical coordinate value included in the fingertip space position information can be determined as a relative vertical coordinate difference. Finally, the control device 103 can determine the relative horizontal coordinate difference, the relative vertical coordinate difference, and the relative vertical coordinate difference as the relative space coordinate information between the reference wrist space point coordinate and the fingertip space position information.

[0055] Secondly, the robot fingertip space coordinate information is generated based on the robot hand wrist space point coordinate and the relative space coordinate information. In practice, the control device 103 can determine the sum of the horizontal coordinate value included in the robot hand wrist space point coordinate and the relative horizontal coordinate difference included in the relative space coordinate information as a robot fingertip space horizontal coordinate. Then, the sum of the vertical coordinate value included in the robot hand wrist space point coordinate and the relative vertical coordinate difference included in the relative space coordinate information can be determined as a robot fingertip space vertical coordinate. Next, the sum of the vertical coordinate value included in the robot hand wrist space point coordinate and the relative vertical coordinate difference included in the relative space coordinate information can be determined as a robot fingertip space vertical coordinate. Finally, the control device 103 can determine the robot fingertip space horizontal coordinate, the robot fingertip space vertical coordinate, and the robot fingertip space vertical coordinate as the robot fingertip space coordinate information.

[0056] Fourthly, each generated robot fingertip space coordinate information is determined as robot mapping motion information.

[0057] In some embodiments, the remote control robot 101 can be configured to perform the following steps:

[0058] In the first step, in response to determining that the robot mapping motion information is the individual robot joint mapping motion information, the individual current robot joint motion information of the robot joint where the preset robot angle sensor is located is collected by the preset robot angle sensor as the motion feedback information. Each of the individual current robot joint motion information can be the angle representing the bending or tilting of the robot joint where the preset robot angle sensor is located. For example, the current robot joint motion information can be "the angle of the robot joint 1 corresponding to the robot joint at time point 6 seconds is 85 degrees". The at least one preset robot angle sensor can be at least one angle sensor installed in the remote control robot (for example, four angle sensors are installed on the robot thumb of the remote control robot 101, and three angle sensors are installed on each of the remaining robot fingers). The robot joint 1 can be the identification of the robot joint.

[0059] In the second step, in response to determining that the robot mapping motion information is the individual robot fingertip spatial coordinate information, the individual force information of the robot fingertip where the at least one force sensor is located is collected by the at least one force sensor as the motion feedback information. The at least one robot fingertip and the at least one finger of the force tactile feedback glove correspond one-to-one to the at least one human fingertip.

[0060] In the third step, the tactile information when the remote control robot 101 contacts with the object in the motion process is collected by the at least one tactile feedback information collector. The tactile feedback information collector can be a tactile sensor for collecting the tactile information when the remote control robot 101 contacts with the object in the space.

[0061] In the fourth step, the motion feedback information and the tactile information are sent to the control device.

[0062] In some embodiments, the control device 103 can be further configured to perform force tactile reproduction operation on the human hand through the force tactile feedback glove based on the motion feedback information and the tactile information. Each fingertip of the force tactile feedback glove is provided with a tactile feedback device, each finger of the force tactile feedback glove is provided with a force feedback actuator, each force feedback actuator of the at least one force feedback actuator included in the force tactile feedback glove corresponds to each force sensor of the at least one force sensor, the force feedback actuator is composed of a steering wheel, a winch, a tendon, the tactile information includes at least one tactile sub-information collected by the at least one tactile feedback information collector, and each tactile feedback information collector of the at least one tactile feedback information collector corresponds to each tactile feedback device of the at least one tactile feedback device of the force tactile feedback glove. The tactile feedback device can be a device for expressing and delivering tactile sensation such as vibration, electric stimulation, ultrasound, array, etc. For example, the tactile feedback device can be, but is not limited to, one of the following: ERM or LRA vibration motor. The force feedback actuator can be a device for applying feedback force to the human hand. Each tactile sub-information of the at least one tactile sub-information can be information collected by one tactile feedback device of the at least one tactile feedback device.

[0063] In some optional implementations of some embodiments, the control device 103 can be further configured to perform force tactile reproduction operation on the human hand through the force tactile feedback glove based on the motion feedback information and the tactile information, including:

[0064] In the first step, in response to determining that the motion feedback information is the respective mechanical joint mapping motion information, the following first force feedback reproduction operation is performed based on the motion feedback information:

[0065] In the first sub-step, for each current mechanical hand joint motion information included in the motion feedback information, the following steps are performed:

[0066] In the first sub-step, for each current mechanical hand joint motion information included in the motion feedback information, the following steps are performed:

[0067] Sub-step two: Based on the current manipulator joint motion information and the target mechanical joint mapping motion information, torque force information is generated. This torque force information corresponds to an angle sensor, which in turn corresponds to a force feedback actuator. In practice, the control device 103 can determine the angle of the mechanical joint corresponding to the current manipulator joint motion information as the first target angle. Then, the control device 103 can determine the angle represented by the target mechanical joint mapping motion information as the second target angle. Next, the control device 103 can determine the absolute value of the difference between the second angle and the first angle as the target angle difference. Then, the control device 103 can query a preset angle difference and torque relationship table to obtain the torque corresponding to the target angle difference. Next, the control device 103 can determine the length of the mechanical joint where the preset manipulator angle sensor that collected the current manipulator joint motion information is located. Finally, the product of the mechanical joint length and the torque is determined as the torque force information. The preset angle difference and torque relationship table can be a preset table representing a one-to-one correspondence between angle differences and torques. As an example, the target angle difference can be 5 degrees. The torque corresponding to the aforementioned 5 degrees can be 3 Nm. The length corresponding to the length of the mechanical joint can be 0.5 m. The torque force information can be "3 Nm × 0.5 m = 1.5 N".

[0068] In sub-step three, the force feedback actuator in the force feedback glove, corresponding to the torque force information, is controlled to apply the feedback force corresponding to the torque force information to the human hand. In practice, the control device 103 can determine the movement direction of the hand joint corresponding to the mechanical joint through the gesture tracking sensor (e.g., fiber optic sensor) in the force feedback glove. Then, the force feedback actuator can apply the feedback force corresponding to the movement direction and torque force information to the hand joint corresponding to the mechanical joint in the human hand. The mechanical hand joints correspond one-to-one with the hand joints of the human hand corresponding to the force feedback glove.

[0069] The second step, in response to determining that the motion feedback information is the force information, is to perform the following second force feedback reproduction operation based on the motion feedback information:

[0070] The first sub-step involves performing the following steps for each force information included in the aforementioned force feedback information:

[0071] Sub-step one: Determine the force sensor for collecting the above force information.

[0072] Sub-step two: Determine the force feedback actuator that corresponds to the force sensor among the above-mentioned at least one force feedback actuator as the target force feedback actuator.

[0073] Sub-step three, control the target force feedback actuator to act the feedback force corresponding to the force information on the human hand. In practice, the control device 103 can determine the motion direction of the human finger tip through the gesture tracking sensor (for example, optical fiber sensor) in the force tactile feedback glove. Wherein, the human finger tip can be the human finger tip corresponding to the mechanical finger tip where the force sensor collects the force information. Then, the control device 103 can act the feedback force corresponding to the motion direction and the force information on the human finger tip through the force feedback actuator. Wherein, each of the at least one force feedback actuator corresponds to one of the at least one force sensor.

[0074] Third step, for each of the at least one tactile sub-information included in the tactile information, the following steps are performed:

[0075] First sub-step, determine the tactile feedback information collector collecting the tactile sub-information;

[0076] Second sub-step, determine the target tactile feedback device corresponding to the tactile feedback information collector among the at least one tactile feedback device in the force tactile feedback glove;

[0077] Third sub-step, control the target tactile feedback device to act the tactile corresponding to the tactile sub-information on the preset area of the finger tip where the tactile feedback device is located.

[0078] The technical solution and related content are an invention point of an embodiment of the present disclosure, which solves the second technical problem mentioned in the background. By installing a rope that binds fingers and palms in a glove for controlling the movement of a robot hand, the rope is tightened to make the human hand feel a feedback force. However, the degree of tightening of the rope and the size of the feedback force received by the robot hand during movement may have a nonlinear relationship, resulting in poor authenticity of the feedback force received by the human hand during movement of the robot hand. The rope installed in the glove that binds the fingers and palms cannot feedback the tactile sensation of the robot hand contacting the spatial object during movement to the human hand. At the same time, the use of the rope to bind the fingers and palms may bring a sense of restraint and discomfort to the user, resulting in poor user experience. Factors that lead to poor user experience often include: by installing a rope that binds fingers and palms in a glove for controlling the movement of a robot hand, the rope is tightened to make the human hand feel a feedback force. However, the degree of tightening of the rope and the size of the feedback force received by the robot hand during movement may have a nonlinear relationship, resulting in poor authenticity of the feedback force received by the human hand during movement of the robot hand. The rope installed in the glove that binds the fingers and palms cannot feedback the tactile sensation of the robot hand contacting the spatial object during movement to the human hand. At the same time, the use of the rope to bind the fingers and palms may bring a sense of restraint and discomfort to the user, resulting in poor user experience. If the above factors are solved, the effect of improving user experience can be achieved. To achieve this effect, the first step is to determine the movement feedback information as each robot joint mapping movement information. Based on the movement feedback information, the following first force feedback reproduction operation is performed: in the first sub-step, for each current robot joint movement information included in the movement feedback information, the following steps are performed: in the first sub-step, the mechanical joint mapping movement information corresponding to the current robot joint movement information is determined as the target mechanical joint mapping movement information. In the second sub-step, based on the current robot joint movement information and the target mechanical joint mapping movement information, torque force information is generated, wherein the torque force information corresponds to an angle sensor corresponding to a force feedback actuator. In the third sub-step, the force feedback actuator corresponding to the torque force information in the force tactile feedback glove is controlled to act on the human hand with the feedback force corresponding to the torque force information. Thus, based on the difference between each current robot joint movement information (current actual movement position) included in the movement feedback information and the corresponding mechanical joint mapping movement information (target movement position), the size of the feedback force actually received by the current robot joint during movement, that is, the torque force information, can be determined. The force feedback actuator included in the force tactile feedback glove acts on the human hand with the actual feedback force, improves the authenticity of the feedback force, reduces the sense of restraint and discomfort to the user, and improves the user experience.Second step, in response to determining that the motion feedback information is each force information, based on the motion feedback information, the following second force feedback reproduction operation is performed: first sub-step, for each force information in the force information included in the force feedback information, the following steps are performed: sub-step one, determine the force sensor collecting the force information. Sub-step two, determine the force feedback actuator corresponding to the force sensor in the at least one force feedback actuator as the target force feedback actuator. Sub-step three, control the target force feedback actuator to act the feedback force corresponding to the force information on the human hand. Thus, in the case of motion feedback information being each force information, the actual feedback force received can be acted on the human hand through the force feedback actuator included in the force tactile feedback glove, the authenticity of the feedback force is improved, the bondage and discomfort brought to the user are reduced, and the user experience is improved. Third step, for each tactile sub-information in the at least one tactile sub-information included in the tactile information, the following steps are performed: first sub-step, determine the tactile feedback information collector collecting the tactile sub-information; second sub-step, determine the tactile feedback device corresponding to the tactile feedback information collector in the at least one tactile feedback device in the force tactile feedback glove as the target tactile feedback device; third sub-step, control the target tactile feedback device to act the tactile corresponding to the tactile sub-information on the fingertip preset area where the tactile feedback device is located. Thus, the tactile information corresponding to the tactile information collected by the force tactile feedback glove when the remote control manipulator contacts the object during the movement can be acted on the human hand through the tactile feedback device included in the force tactile feedback glove, and the user experience is improved.

[0079] The above various embodiments of the present disclosure have the following beneficial effects: the force tactile feedback teleoperation system of some embodiments of the present disclosure improves the flexibility of the motion of the robot and the user experience. Specifically, the reason why the flexibility of the motion of the robot and the user experience are poor is that the motion trajectory and action of the robot are controlled by a pre-written program, the action of the robot is usually fixed, the flexibility of the motion of the robot is poor, and the user cannot intuitively feel the interaction with the spatial environment during the motion and operation of the robot, and the user experience is poor. Based on this, the force tactile feedback teleoperation system of some embodiments of the present disclosure includes a force tactile feedback glove, a remote control robot, and a control device, wherein: first, the force tactile feedback glove is configured to perform the following capturing steps: capturing human hand action information; transmitting the human hand action information to the control device. Thus, the human hand action information for controlling the motion of the remote control robot can be captured by the force tactile feedback glove. Then, the control device is configured to perform the following mapping steps: receiving the human hand action information transmitted by the force tactile feedback glove; generating robot mapping motion information based on the human hand action information; controlling the remote control robot to make a motion corresponding to the robot mapping motion information, wherein the robot mapping motion information is each robot joint mapping motion information or each robot fingertip spatial coordinate information; sending the robot mapping motion information to the remote control robot. Thus, the control device can generate robot mapping motion information and control the motion of the remote control robot based on the robot mapping motion information. The remote control robot is controlled by the force tactile feedback glove and the control device to make a motion consistent with the human hand action information, improving the flexibility of the motion of the robot. Then, the remote control robot is configured to perform the following feedback information collection steps: in response to determining that the robot mapping motion information is each robot joint mapping motion information, collecting each current robot joint motion information of the robot joint where the preset robot angle sensor is located as motion feedback information through at least one preset robot angle sensor; in response to determining that the robot mapping motion information is each robot fingertip spatial coordinate information, collecting each force information of the robot fingertip where the force sensor is located as motion feedback information through at least one force sensor on the robot fingertip; collecting tactile information when contacting an object during the motion through at least one tactile feedback information collector; sending the motion feedback information and the tactile information to the control device. Thus, the motion feedback information and the tactile information of the interaction with the spatial environment during the motion of the remote control robot can be collected through at least one preset robot angle sensor or at least one force sensor. Then, the control device is further configured to perform force tactile reproduction operation on the human hand through the force tactile feedback glove based on the motion feedback information and the tactile information.Thus, the motion feedback information and the tactile information of the interaction with the space environment in the motion process of the remote control manipulator can be transmitted to the human hand through the force tactile feedback glove by the control device, so that the user can intuitively feel the motion and the operation process of the manipulator and the interaction with the space environment, and the user experience is improved.

[0080] Figure 2 Flow 200 of some embodiments of a remote control manipulator method according to the present disclosure, which includes the control device of the force tactile feedback teleoperation system described above, is shown. The remote control manipulator method includes the following steps:

[0081] Step 201, receiving the human hand action information transmitted by the force tactile feedback glove.

[0082] In some embodiments, the execution subject of the remote control manipulator (for example, the control device included in the force tactile feedback teleoperation system) can receive the human hand action information transmitted by the force tactile feedback glove.

[0083] Step 202, generating manipulator mapping motion information based on the human hand action information.

[0084] In some embodiments, the execution subject can generate the manipulator mapping motion information based on the human hand action information.

[0085] Step 203, controlling the remote control manipulator to make a motion corresponding to the manipulator mapping motion information.

[0086] In some embodiments, the execution subject can control the remote control manipulator to make a motion corresponding to the manipulator mapping motion information.

[0087] The above various embodiments of the present disclosure have the following beneficial effects: through the remote control robot arm method of the control device applied to the force tactile feedback teleoperation system of some embodiments of the present disclosure, the flexibility of the robot arm movement is improved. Specifically, the reason for poor flexibility of the robot arm movement is that the movement trajectory and action of the robot arm are controlled by a pre-written program, and the action of the robot arm is usually fixed, and the flexibility of the robot arm movement is poor. Based on this, the remote control robot arm method of the control device applied to the force tactile feedback teleoperation system of some embodiments of the present disclosure includes: first, receiving the human hand action information transmitted by the force tactile feedback glove. Thus, the human hand action information representing the real movement of the human hand can be collected by the force tactile feedback glove. Then, based on the human hand action information, the robot arm mapping movement information is generated. Thus, the robot arm mapping movement information for controlling the movement of the remote control robot arm can be generated according to the human hand action information. Then, the remote control robot arm is controlled to make a movement corresponding to the robot arm mapping movement information. Thus, the remote control robot arm can be controlled to make a movement corresponding to the robot arm mapping movement information. Also because the human hand action information representing the real movement of the human hand is collected, and the robot arm mapping movement information for controlling the movement of the robot arm is generated based on the human hand action information, the robot arm can follow the movement of the human hand and make a corresponding movement, and the flexibility of the robot arm movement is improved.

[0088] Figure 3 is an internal test diagram of some embodiments of the force tactile feedback teleoperation system according to the present disclosure, Figure 3 The test remote control robot arm 301 and the test force tactile feedback glove 302 in the test are shown.

[0089] The above description is only some preferred embodiments of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the application involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combinations of technical features, and should also cover other technical solutions formed by any combination of technical features or their equivalent features without departing from the inventive concept. For example, the technical solutions formed by replacing the features with the technical features disclosed in the embodiments of the present disclosure (but not limited to) having similar functions.

Claims

1. A force tactile feedback teleoperation system, characterized by, The force tactile feedback teleoperation system comprises: a force tactile feedback glove, a remote control manipulator, and a control device, wherein: The force tactile feedback glove is configured to perform the following capturing steps: Capturing human hand action information, wherein the force tactile feedback glove comprises respective angle sensors, and the force tactile feedback glove is further configured to capture human hand action information by the following steps, comprising: collecting joint motion information of a hand joint where each of the respective angle sensors is located by each of the respective angle sensors, wherein the joint motion information corresponds to an angle sensor identifier of the angle sensor; determining the collected respective joint motion information as human hand action information, wherein each of at least one preset manipulator angle sensor corresponds to one of the respective angle sensors, and each of respective current manipulator joint motion information corresponds to one of the respective joint motion information; Transmitting the human hand action information to the control device; The control device is configured to perform the following mapping steps: Receiving human hand action information transmitted by the force tactile feedback glove; Based on the human hand action information, generating manipulator mapping motion information, wherein the control device is further configured to generate manipulator mapping motion information based on the human hand action information by the following steps, comprising: obtaining preset human hand motion limit range information corresponding to the force tactile feedback glove and preset motion range information of the remote control manipulator, wherein the preset human hand motion limit range information comprises respective joint motion range information of respective joints, the preset motion range information comprises respective manipulator joint motion range information of respective remote control manipulator joints, each of the respective joint motion range information corresponds one-to-one to each of the respective manipulator joint motion range information, and each of the respective joint motion range information corresponds one-to-one to each of the respective joint motion information; for each of the respective joint motion information included in the human hand action information, performing the following steps: determining joint motion range information corresponding to the joint motion information in the respective joint motion range information as target joint motion range information; based on the joint motion information and the target joint motion range information, generating joint mobility ratio information; determining manipulator joint motion range information corresponding to the target joint motion range information in the respective manipulator joint motion range information as target manipulator joint motion range information; based on the joint mobility ratio information and the target manipulator joint motion range information, generating manipulator joint mapping motion information; determining the generated respective manipulator joint mapping motion information as manipulator mapping motion information, wherein each of the respective current manipulator joint motion information corresponds to one of the respective manipulator joint mapping motion information. controlling the remote manipulator robot to make a movement corresponding to the manipulator mapping movement information, wherein the manipulator mapping movement information comprises each manipulator joint mapping movement information and each manipulator fingertip spatial coordinate information; sending the manipulator mapping movement information to the remote manipulator robot; the remote manipulator robot is configured to perform the following feedback information collection steps: in response to determining that the manipulator mapping movement information comprises each manipulator joint mapping movement information, collecting, by at least one preset manipulator angle sensor, each current manipulator joint movement information of a manipulator joint where the preset manipulator angle sensor is located as movement feedback information; in response to determining that the manipulator mapping movement information comprises each manipulator fingertip spatial coordinate information, collecting, by at least one force sensor on the at least one manipulator fingertip, each force information of a manipulator fingertip where the force sensor is located as movement feedback information; collecting, by at least one haptic feedback information collector, haptic information when the object is contacted during the movement; sending the movement feedback information and the haptic information to the control device; the control device is further configured to perform force and haptic reproduction operations on the human hand by the force and haptic feedback glove based on the movement feedback information and the haptic information.

2. The force tactile feedback teleoperation system of claim 1, wherein, the force and haptic feedback glove comprises each fingertip spatial position collector and wrist spatial position collector, and the force and haptic feedback glove is further configured to capture human hand action information by the following steps: collecting, by the wrist spatial position collector, human hand wrist spatial point position information, wherein the human hand wrist spatial point position information comprises wrist spatial point coordinates; collecting, by each fingertip spatial position collector of the each fingertip spatial position collector, fingertip spatial position information of a fingertip where the fingertip spatial position collector is located, to obtain each fingertip spatial position information, wherein each fingertip spatial position information of the each fingertip spatial position information comprises fingertip spatial point coordinates; determining the each fingertip spatial position information and the human hand wrist spatial point position information as human hand action information.

3. The force tactile feedback teleoperation system of claim 2, wherein, the control device is further configured to generate manipulator mapping movement information based on the human hand action information by the following steps: determining wrist spatial point coordinates comprised in the human hand action information as reference wrist spatial point coordinates; collecting spatial coordinates of a preset point in a manipulator wrist as manipulator wrist spatial point coordinates; for each fingertip spatial position information comprised in the human hand action information, performing the following steps: generating relative spatial coordinate information between the reference wrist spatial point coordinates and the fingertip spatial position information based on the reference wrist spatial point coordinates and the fingertip spatial position information; generating manipulator fingertip spatial coordinate information based on the manipulator wrist spatial point coordinates and the relative spatial coordinate information; determining each generated manipulator fingertip spatial coordinate information as manipulator mapping movement information.

4. A remote manipulator robot method applied to a control device included in a force and haptic feedback teleoperation system according to any one of claims 1-3, the method comprising: receive human hand action information transmitted by the force tactile feedback glove; generate robot mapping motion information based on the human hand action information; control the remote control robot to make motion corresponding to the robot mapping motion information.

Images