Remote control system, remote control method for robot, and remote control program

By combining display devices and tactile devices in the remote control system, tactile feedback is provided, which solves the problem that users find it difficult to accurately judge the robot status, improves judgment accuracy and reduces time lag.

CN119031997APending Publication Date: 2024-11-26KAWASAKI JUKOGYO KK
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202380033851.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-15
Filing Date
2023-04-14
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In remote control robot systems, it is difficult for users to accurately judge the status of the robot, especially because the image quality of the display device affects the confirmation accuracy, which may lead to time lag.

Method used

Combined with the display device, tactile devices and control devices, the robot is operated through the user's input, displaying the robot image and providing tactile feedback when the robot touches an object, ensuring that the user can accurately judge the status of the robot.

Benefits of technology

Through haptic feedback, users can instantly understand the contact between the robot and the object, reducing dependence on the image quality of the display device, improving judgment accuracy and reducing time lag.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119031997A_ABST
    Figure CN119031997A_ABST
Patent Text Reader

Abstract

The invention provides a remote control system. A remote control system (100) includes a robot (1), an operation device (4) that accepts an input from a user, a display device (52) that presents an image of the robot (1) to the user, a haptic device (8) that imparts a stimulus to a haptic sense of the user, and a control device (6) that controls the operation device (4), the display device (52), the haptic device (8), and the control device (6). The control device (6) controls the robot (1) in accordance with an input to the operation device (4). The control device (6) operates the tactile device (8) when the robot (1) is in contact with another object.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a remote control system, a remote control method of a robot and a remote control program. Background Art

[0002] Until now, the technology of remotely controlling a robot via an operating device is well known. For example, Patent Document 1 discloses a remote control system including a robot and an operating device, wherein the robot and the operating device are arranged at locations physically separated from each other. In the remote control system, an image of the robot is displayed on a display device. A user operates the operating device while viewing the display device, thereby remotely operating the robot.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2010-89211 Summary of the invention

[0004] In the above-mentioned remote control system, the user confirms the status of the robot via the display device. Therefore, the accuracy with which the user can confirm the status of the robot depends on the image quality of the image displayed on the display device. When the image quality is improved in order to improve the confirmation accuracy, there is a possibility that a delay will occur in the image displayed on the display device, resulting in a time lag in confirming the status of the robot.

[0005] In view of the above, an object of the technology of the present invention is to enable a user to accurately judge the status of a robot during remote control of the robot.

[0006] The remote control system of the present invention includes a robot, an operating device, a display device, a tactile device and a control device, wherein the operating device accepts input from a user, the display device presents an image of the robot to the user, the tactile device imparts stimulation to the user's tactile sense, the control device controls the robot based on the input to the operating device, and the control device activates the tactile device when the robot contacts other objects.

[0007] Furthermore, the remote control system of the robot of the present invention includes a robot, an operating device, a display device, a tactile device and a control device, wherein the operating device accepts input from a user, the display device presents an image of the robot to the user, the tactile device provides stimulation to the user's tactile sense, and the control device enables the robot to perform a task including multiple processes based on the input to the operating device, and the control device activates the tactile device when at least one of the multiple processes is completed.

[0008] The remote control method of a robot of the present invention is a remote control method of a robot for controlling the robot via an operating device. The remote control method includes the steps of accepting input from a user via the operating device, operating the robot according to the input to the operating device, presenting an image of the operating robot to the user via a display device, and operating a tactile device that provides tactile stimulation to the user when the robot contacts another object.

[0009] The remote control program of the present invention is a remote control program for enabling a computer to realize a function of controlling a remote control system, wherein the remote control system includes a robot, an operating device, and a tactile device, wherein the operating device accepts input from a user, and the tactile device imparts stimulation to the user's sense of touch. The remote control program enables the computer to realize a function of accepting input from a user via the operating device, a function of causing the robot to move according to the input to the operating device, and a function of causing the tactile device that imparts stimulation to the user's sense of touch to move when the robot contacts another object.

[0010] (Effects of the Invention)

[0011] According to the remote control system, the user can accurately judge the status of the robot.

[0012] According to the remote control method of the robot, the user can accurately judge the status of the robot.

[0013] According to the remote control program, the user can accurately judge the status of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram showing the structure of the remote control system according to the embodiment.

[0015] Figure 2 2 is a schematic front view showing the hand 2 .

[0016] Figure 3 2 is a diagram showing a schematic hardware configuration of the control device 6 .

[0017] Figure 4 This is a functional block diagram of the control device 6.

[0018] Figure 5 It is a flowchart of remote control.

[0019] Figure 6 This is a flowchart of remote control according to a modification. DETAILED DESCRIPTION

[0020] Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. Figure 1 1 is a schematic diagram showing a configuration of a remote control system 100 according to an embodiment.

[0021] The remote control system 100 includes a robot 1, an operating device 4, a display device 52, a tactile device 8, and a control device 6. The operating device 4 receives input from a user, the display device 52 displays an image of the robot 1 to the user, the tactile device 8 provides stimulation to the user's sense of touch, and the control device 6 controls the robot 1 according to the input to the operating device 4. The operating device 4, the display device 52, and the tactile device 8 are arranged at a first site S1 where the user is located. The robot 1 is arranged at a second site S2 different from the first site S1. The control device 6 implements remote control between the operating device 4 and the robot 1. The operating device 4 functions as a master device, and the robot 1 functions as a slave device.

[0022] The remote control system 100 may further include a camera 51 for photographing the robot 1. The camera 51 is disposed at the second station S2. The display device 52 displays an image photographed by the camera 51.

[0023] According to the remote control system 100, the user remotely operates the robot 1 via the operating device 4 to make the robot 1 perform a predetermined operation. At this time, the camera 51 captures the image of the robot 1, and the display device 52 displays the captured image of the camera 51. The user operates the operating device 4 while confirming the image of the robot 1 displayed on the display device 52. In addition, the user is given tactile stimulation from the tactile device 8 at an appropriate time. The user can know that a predetermined event has occurred during the remote operation by receiving tactile stimulation from the tactile device 8.

[0024] It should be noted that, in the present invention, the work performed by the robot 1 does not include teaching work and teaching confirmation and correction work. Therefore, the operating device 4 does not include a teaching pendant.

[0025] [robot]

[0026] The robot 1 is an industrial robot and includes a robot arm 11 and a hand 2 . The hand 2 is provided with fingers 22 for opening and closing and is connected to the robot arm 11 .

[0027] The robot arm 11 changes the position and posture of the hand 2. The robot arm 11 is a vertical multi-joint robot arm. The robot arm 11 has a plurality of links 12, joints 13, and servo motors 14 (see Figure 3 ) and encoder 31 (refer to Figure 3), the joint 13 connects a plurality of connecting rods 12, the servo motor 14 rotationally drives the plurality of joints 13, and the encoder 31 detects the rotation position of the servo motor 14. For example, the connecting rod 12 at one end of the robot arm 11 is rotatably connected to the base 10 around a rotation axis extending in a vertical direction via the joint 13. It should be noted that the robot arm 11 can also be a horizontal multi-joint robot arm, a parallel link robot arm, a rectangular coordinate robot arm, a polar coordinate robot arm, etc.

[0028] The hand 2 is the end effector of the robot arm 11. The hand 2 is connected to the front end of the robot arm 11. Specifically, the hand 2 is connected to the end of the link 12 on the opposite side of the link 12 connected to the base 10 among the plurality of links 12. The hand 2 can be in various postures through the movement of the robot arm 11.

[0029] Figure 2 is a schematic front view showing the hand 2. The hand 2 has a hand body 21 and two fingers 22, and the two fingers 22 are provided on the hand body 21. The hand body 21 is rotatably connected to the connecting rod 12 of the robot arm 11 around a specified rotation axis A. The two fingers 22 are arranged in a specified opening and closing direction X. Specifically, the two fingers 22 are symmetrically arranged with respect to a center axis Z that is approximately orthogonal to the opening and closing direction X. The center axis Z and the rotation axis A are arranged approximately in a straight line. The two fingers 22 perform an opening and closing action in the opening and closing direction X. The fingers 22 extend in a direction intersecting the opening and closing direction X. Specifically, the fingers 22 extend approximately parallel to the center axis Z. The hand 2 also has a servo motor 23 and an encoder 32, wherein the servo motor 23 causes the two fingers 22 to perform an opening and closing action, and the encoder 32 detects the rotation position of the servo motor 23.

[0030] The remote control system 100 further includes a sensor 3 for detecting a force acting on the robot 1. The sensor 3 includes a contact force sensor 33 for detecting a force acting on the finger 22 from the surface on which the workpiece W is placed. Figure 2As shown, the contact force sensor 33 is provided in the hand 2. Specifically, the contact force sensor 33 is provided in the connection portion of the hand 2 with the robot arm 11. The contact force sensor 33 detects the force in the direction of the central axis Z of at least two fingers 22, that is, the direction of the rotation axis A of the hand 2 with respect to the connecting rod 12. Specifically, the contact force sensor 33 detects the force in the three-axis directions orthogonal to the direction of the central axis Z of the finger 22 and the torque around the three axes. When holding the workpiece W, the hand 2 usually moves in the direction of the central axis Z and contacts the surface on which the workpiece W is placed. In other words, the contact force sensor 33 detects the contact force acting on the finger 22 from the surface on which the workpiece W is placed. It should be noted that the contact force sensor 33 can also detect only the force in the single-axis, double-axis or three-axis directions. The contact force sensor 33 is an example of a first sensor.

[0031] In addition, the encoder 31 and the encoder 32 are included in the sensor 3. As will be described in detail later, when the deviation between the current rotation position of the servo motor 14 detected by the encoder 31 and the target rotation position is large, the robot arm 11 contacts other objects, that is, external force acts on the robot 1. Similarly, when the deviation between the current rotation position of the servo motor 23 detected by the encoder 32 and the target rotation position is large, the finger 22 contacts other objects, that is, external force acts on the robot 1. That is, the encoder 31 and the encoder 32 also function as sensors for detecting the force acting on the robot 1. Here, the encoder 32 is equivalent to the second sensor that detects the force acting on the finger 22 in the opening and closing direction X from the workpiece W to the finger 22.

[0032] [Operation device]

[0033] like Figure 1 As shown, the operating device 4 has an input unit 41 for the user to perform input operations. The input unit 41 includes a direction key 41a, a first input button 41b, a second input button 41c, a third input button 41d, and a fourth input button 41e. The operating device 4 receives input for remotely operating the robot 1 and outputs the input information, i.e., operation information, to the control device 6.

[0034] The operating device 4 enables various operation inputs through a combination of the direction key 41a, the first input button 41b, the second input button 41c, the third input button 41d, and the fourth input button 41e. For example, when the robot arm 11 is moved, the user can input the movement of the hand 2 in the direction corresponding to the direction key 41a, forward, backward, left, and right, by operating the direction key 41a. At this time, the user can input the upward and downward movement of the hand 2 through the second input button 41c and the fourth input button 41e. When the finger 22 is moved, the user can input the closing action of the finger 22 by operating the first input button 41b, and input the opening action of the finger 22 by operating the third input button 41d. In addition, it is also possible to switch options with the direction key 41a, and input a specified command through the first input button 41b as a decision button or the third input button 41d as a cancel button. For example, the user can input a desired command by selecting a command from the options of commands displayed on the display device 52 using the direction key 41a and deciding the command using the first input button 41b.

[0035] [Camera device]

[0036] The camera 51 is disposed at the second station S2. Specifically, the camera 51 is mounted on the front end of the robot arm 11. The camera 51 captures an image of the robot 1, specifically, the hand 2. The camera 51 outputs the captured image to the control device 6.

[0037] [Display device]

[0038] The display device 52 is arranged at a position where the user can see while operating the operating device 4. The display device 52 displays the image captured by the camera 51 and sent from the control device 6. The display device 52 assists the user who operates the operating device 4 by displaying the captured image of the robot 1.

[0039] [Haptic Devices]

[0040] The haptic device 8 includes, for example, a voice coil motor 81. The haptic device 8 is worn on, for example, a wrist of a user. The haptic device 8 generates vibrations by the operation of the voice coil motor 81. The haptic device 8 provides tactile stimulation to the user through the vibration of the voice coil motor 81.

[0041] The tactile device 8 can change the vibration mode. The vibration mode includes vibration, frequency, and repetition cycle or rhythm. For example, the tactile device 8 can adjust the amplitude, that is, the size of the vibration, by adjusting the current applied to the voice coil motor 81. The tactile device 8 can adjust the size of the vibration in three stages. The tactile device 8 can selectively generate a first vibration with a smaller vibration, a second vibration greater than the first vibration, and a third vibration greater than the second vibration. The first vibration, the second vibration, and the third vibration can be either continuous vibration or intermittent vibration. The vibration modes of the first vibration, the second vibration, and the third vibration can be unified or different from each other. It should be noted that the tactile device 8 can also adjust the frequency of the vibration in addition to the amplitude, or can also adjust the frequency of the vibration instead of the amplitude. In addition, the tactile device 8 can generate an alarm vibration in addition to the first vibration, the second vibration, and the third vibration. For example, the amplitude of the alarm vibration is greater than the first vibration, the second vibration, and the third vibration, and the vibration mode of the alarm vibration is different from the first vibration, the second vibration, and the third vibration.

[0042] [Control device]

[0043] The robot 1, the sensor 3, the operating device 4, the camera 51, the display device 52, and the haptic device 8 are connected to the control device 6. The control device 6 operates the robot 1 according to the operation information from the operating device 4. Specifically, the control device 6 causes the robot 1 to perform a work including a plurality of processes according to the input to the operating device 4. For example, the control device 6 causes the robot 1 to perform a picking operation of the workpiece W. In this example, the control device 6 does not perform force control and position control of the operating device 4 based on the state of the robot 1. The control device 6 performs the so-called one-way control of controlling the robot 1 by one-way passage according to the operation information of the operating device 4.

[0044] In parallel with the control of the robot 1, the control device 6 transmits the captured image from the camera 51 to the display device 52. In addition, the control device 6 controls the tactile device 8 in response to the status of the robot 1. Specifically, the control device 6 notifies the user of the contact status of the robot 1 with other objects through the tactile device 8. In this way, the control device 6 performs the control of the robot 1, the presentation of the captured image, and the control of the tactile device 8 in parallel.

[0045] Figure 3 2 is a diagram showing a schematic hardware configuration of the control device 6. The control device 6 includes a main control device 60 and a robot control device 7. The main control device 60 and the robot control device 7 transmit and receive information, commands, data, and the like.

[0046] The main control device 60 receives a signal from the operating device 4 and outputs a command to the robot control device 7. The robot control device 7 applies a current to the servo motor 14 or the servo motor 23 according to the command. At this time, the robot control device 7 receives a signal from the encoder 31 or the encoder 32, and performs feedback control on the servo motor 14 or the servo motor 23. In addition, the robot control device 7 receives signals from the encoder 31, the encoder 32, and the contact force sensor 33 and an image signal from the camera device 51, and outputs the received signals to the main control device 60. The main control device 60 outputs the image signal from the camera device 51 to the display device 52. The main control device 60 controls the tactile device 8 according to the signals from the encoder 32 and the contact force sensor 33.

[0047] The robot control device 7 includes a control unit 71 , a storage unit 72 , and a memory 73 . The main control device 60 includes a control unit 61 , a storage unit 62 , and a memory 63 .

[0048] The control unit 71 controls the entire robot control device 7. The control unit 61 controls the entire main control device 60. The control unit 71 and the control unit 61 perform various calculation processes respectively. For example, each of the control unit 71 and the control unit 61 is formed by a processor such as a CPU (central processing unit). Each of the control unit 71 and the control unit 61 can also be formed by an MCU (microcontroller unit), an MPU (microprocessor unit), an FPGA (field programmable gate array), a PLC (programmable logic controller), a system LSI, etc.

[0049] The storage unit 72 stores programs and various data executed by the control unit 71. The storage unit 62 stores programs and various data executed by the control unit 61. Specifically, the storage unit 62 stores a remote control program 62a. Each of the storage unit 72 and the storage unit 62 is formed of a nonvolatile memory, an HDD (hard disk drive) or an SSD (solid state drive) or the like.

[0050] The memory 73 and the memory 63 each temporarily store data etc. For example, each of the memory 73 and the memory 63 is formed of a volatile memory.

[0051] Figure 4 6 is a functional block diagram of the control device 6. The control unit 61 realizes various functions by reading the remote control program 62a from the storage unit 62 into the memory 63 and developing it. Specifically, the control unit 61 functions as an input processing unit 64, a command position generating unit 65, a command angle converting unit 66, an image processing unit 67, and a tactile control unit 68. The control unit 71 realizes various functions by reading the program from the storage unit 72 into the memory 73 and developing it. Specifically, the control unit 71 functions as a sensor processing unit 74, an action control unit 75, and a camera control unit 76.

[0052] The input processing unit 64 switches between the first mode for moving the robot arm 11 and the second mode for moving the finger 22 according to the operation information from the operation device 4. For example, the user inputs the mode switching command from the operation device 4 to the input processing unit 64 by operating the direction key 41a, the first input button 41b, or the second input button 41c.

[0053] The input processing unit 64 generates a command speed of the motion target in the current mode based on the operation information from the operation device 4 .

[0054] Specifically, when in the first mode, the user inputs the moving direction of the hand 2 by operating the direction key 41a or the like. The input processing unit 64 generates the command speed of the hand 2 based on the moving direction input from the operating device 4. The command speed is a so-called speed vector, which is defined by the moving direction and the speed. Here, the speed is made constant. However, in addition to the moving direction of the hand 2, the speed of the hand 2 can also be input by operating the direction key 41a or the like. In that case, the input processing unit 64 generates the command speed of the hand 2 based on the moving direction and the speed input from the operating device 4.

[0055] When in the second mode, the user inputs the moving direction of the finger 22, that is, the opening direction or the closing direction, by operating the direction key 41a, etc. The input processing unit 64 generates the command speed of the finger 22 based on the moving direction input from the operating device 4. The command speed is the so-called speed vector, which is defined by the moving direction and the speed. Here, the speed is made constant. However, in addition to the moving direction of the finger 22, the speed of the finger 22 can also be input by operating the direction key 41a, etc. In that case, the input processing unit 64 generates the command speed of the finger 22 based on the moving direction and the speed input from the operating device 4.

[0056] The command position generating unit 65 converts the command speed generated by the input processing unit 64 into a command position. The command position generating unit 65 converts the command position of the hand 2 in the first mode and converts the command position of the finger 22 in the second mode. Here, the command position of the hand 2 includes not only the position of the hand 2 but also the posture of the hand 2. It should be noted that even if the finger 22 performs an opening and closing action, the posture of the finger 22 relative to the hand body 21 does not change, so the command position of the finger 22 does not include the posture of the finger 22.

[0057] The command angle conversion unit 66 converts the command position converted by the command position generation unit 65 into the command angle position of the servo motor. When the command angle conversion unit 66 is in the first mode, the angle position of each servo motor 14 corresponding to the command position of the hand 2 is obtained as the command angle position. When the command angle conversion unit 66 is in the second mode, the angle position of the servo motor 23 corresponding to the command position of the finger 22 is obtained as the command angle position. The command angle conversion unit 66 outputs the obtained command angle position to the robot control device 7.

[0058] The sensor processing unit 74 receives detection signals, i.e., detection results, from the encoder 31, the encoder 32, and the contact force sensor 33. The sensor processing unit 74 outputs the detection signals from the encoder 31 and the encoder 32 to the motion control unit 75. Furthermore, the sensor processing unit 74 outputs the detection signals from the encoder 31, the encoder 32, and the contact force sensor 33 to the main control device 60.

[0059] The motion control unit 75 receives the command angle position from the main control device 60 and moves the robot 1 according to the command angle position. Specifically, when the motion control unit 75 receives the command angle position of the servo motor 14, it obtains the current command value for realizing the command angle position. The motion control unit 75 applies a current corresponding to the current command value to the servo motor 14 to move the robot arm 11. At this time, the motion control unit 75 performs feedback control on the current applied to the servo motor 14 based on the detection signal of the encoder 31 from the sensor processing unit 74. In this way, the hand 2 moves to the command position. Alternatively, when the motion control unit 75 receives the command angle position of the servo motor 23, it obtains the current command value for realizing the command angle position. The motion control unit 75 applies a current corresponding to the current command value to the servo motor 23 to move the finger 22. At this time, the motion control unit 75 performs feedback control on the current applied to the servo motor 23 based on the detection signal of the encoder 32 from the sensor processing unit 74. In this way, the finger 22 moves to the command position.

[0060] The image capture control unit 76 outputs a command to the image capture device 51 to cause the image capture device 51 to perform image capture. The image capture control unit 76 receives a captured image from the image capture device 51 and transmits the captured image to the main control device 60.

[0061] The image processing unit 67 transmits the captured image received from the imaging device 51, that is, the image signal, to the display device 52. At this time, the image processing unit 67 may perform predetermined image processing on the captured image.

[0062] The haptic control unit 68 controls the haptic device 8. Specifically, the haptic control unit 68 determines whether the robot 1 contacts another object, and when contact occurs, the haptic device 8 vibrates. The haptic control unit 68 determines the occurrence of contact based on the detection results of the sensor 3 including the encoder 31, the encoder 32, and the contact force sensor 33.

[0063] The contact force sensor 33 detects the contact force received by the hand 2 from other objects. Since the contact force sensor 33 detects at least the force in the direction of the central axis Z of the finger 22, it detects the contact force when the finger 22 contacts other objects in the direction of the central axis Z. When the contact force detected by the contact force sensor 33 is greater than a predetermined first threshold value, the tactile control unit 68 determines that the finger 22 contacts other objects. The first threshold value is a value of the contact force that can determine the degree of contact between the hand 2 and other objects, and is a relatively small value.

[0064] The tactile control unit 68 obtains the deviation between the current angular position of the servo motor 14 corresponding to the robot arm 11 and the command angular position based on the detection result of the encoder 31. The deviation of the angular position of the servo motor 14 corresponds to the contact force received by the robot arm 11 from the other object. The tactile control unit 68 determines that the robot arm 11 is in contact with the other object when the deviation of the angular position is greater than a predetermined second threshold value. The second threshold value is a value equivalent to the deviation of the contact force to the extent that it can be determined that the robot arm 11 is in contact with the other object.

[0065] The tactile control unit 68 calculates the deviation between the current angular position of the servo motor 23 of the hand 2 and the command angular position based on the detection result of the encoder 32. The deviation of the angular position of the servo motor 23 corresponds to the contact force received by the finger 22 from other objects in the opening and closing direction X. When the deviation of the angular position is greater than a specified third threshold, the tactile control unit 68 determines that the finger 22 is in contact with other objects. The third threshold is a value equivalent to the deviation of the gripping force of the workpiece W held by the finger 22. For example, when the finger 22 is in contact with the workpiece W while the finger 22 is gripping the workpiece W, the finger 22 cannot close further, and the deviation of the servo motor 23 increases, exceeding the third threshold. At this time, the tactile control unit 68 determines that the finger 22 is in contact with other objects.

[0066] The tactile control unit 68 recognizes the contact detected by each of the encoder 31, the encoder 32, and the contact force sensor 33 as a single contact. That is, the tactile control unit 68 determines the type of contact of the robot 1 according to which of the encoder 31, the encoder 32, and the contact force sensor 33 detects the contact. For example, when the encoder 31 and the contact force sensor 33 each detect a contact, the tactile control unit 68 determines that two different types of contacts have occurred.

[0067] When the robot 1 makes contact with another object, the tactile control unit 68 operates the tactile device 8. The tactile control unit 68 operates the tactile device 8 for each contact of different categories. Specifically, the tactile control unit 68 changes the operation mode of the tactile device 8 in response to the number of categories of contact that are generated simultaneously. When one category of contact is generated, the tactile control unit 68 outputs a command to generate a first vibration to the tactile device 8. When two categories of contact are generated, the tactile control unit 68 outputs a command to generate a second vibration to the tactile device 8. When three categories of contact are generated, the tactile control unit 68 outputs a command to generate a third vibration to the tactile device 8. In either case, the tactile control unit 68 makes a judgment based only on the number of categories of contact, regardless of the category of the contact, and outputs the corresponding command.

[0068] In addition, the tactile control unit 68 determines the magnitude of the contact force of the robot 1 and changes the action mode of the tactile device 8 in response to the magnitude of the contact force. Specifically, the tactile device 8 monitors whether the robot 1 contacts another object with a large contact force to the extent that the action of the robot 1 is forcibly stopped (hereinafter referred to as "strong contact"). Here, strong contact is contact with a contact force larger than the contact force when the contact is determined to have occurred based on the detection results of each sensor. When the contact force detected by the contact force sensor 33 is greater than the first upper limit value greater than the first threshold, the tactile control unit 68 determines that the strong contact of the finger 22 has occurred. When the deviation of the angular position of the servo motor 14 is greater than the second upper limit value greater than the second threshold, the tactile control unit 68 determines that the strong contact of the robot arm 11 has occurred. When the deviation of the angular position of the servo motor 23 is greater than the third upper limit value greater than the third threshold, the tactile control unit 68 determines that the strong contact of the finger 22 has occurred in the opening and closing direction X. The first upper limit value, the second upper limit value, and the third upper limit value are respectively values ​​of a large contact force equivalent to the extent that the action of the robot 1 is forcibly stopped.

[0069] When the strong contact of the robot 1 occurs, the haptic control unit 68 outputs a command to generate an alarm vibration to the haptic device 8. The vibration pattern of the alarm vibration is different from the first vibration, the second vibration, and the third vibration.

[0070] [Pick up operation]

[0071] The operation of the remote control system 100 configured as above will be described by taking a picking operation as an example of a specific operation performed by the robot 1 . Figure 5: is a flowchart of remote control. The picking operation is an operation in which the robot 1 picks up the workpiece W placed in the container C from a predetermined starting state and transfers it to the tray T. The picking operation includes a first moving process, a gripping process, a second moving process, and a releasing process. In the first moving process, the hand 2 is moved to the picking position. In the gripping process, the hand 2 grips the workpiece W. In the second moving process, the hand 2 gripping the workpiece W is moved to the transfer position. In the releasing process, the grip of the workpiece W by the hand 2 is released.

[0072] In the remote control system 100, the image of the robot 1 captured by the camera 51 is displayed on the display device 52. The user operates the operating device 4 while observing the image of the robot 1 displayed on the display device 52. The display of the image of the robot 1 by the display device 52 is equivalent to presenting the image of the robot 1 in action to the user via the display device 52.

[0073] First, in step S101, the user inputs an operation to the operating device 4. When the user operates the operating device 4, the operation information is input from the operating device 4 to the control unit 61. The input processing unit 64 of the control unit 61 receives the operation information from the operating device 4. Step S101 corresponds to receiving the input from the user via the operating device 4.

[0074] In step S102, the control unit 61 causes the robot 1 to perform an action in response to the operation information. In detail, the input processing unit 64 generates a command speed in response to the operation information from the operating device 4. The command position generating unit 65 converts the command speed into a position command. The command angle conversion unit 66 converts the position command into a command angle position, and outputs the converted command angle position to the motion control unit 75. The motion control unit 75 outputs the current used to achieve the command angle position to the corresponding servo motor 14 or servo motor 23. The motion control unit 75 performs feedback control on the applied current based on the detection signal of the encoder 31 or the encoder 32. In this way, the robot 1 performs an action in response to the operation of the user's operating device 4. Step S102 is equivalent to causing the robot 1 to act based on the input to the operating device 4.

[0075] Next, in step S103, the control unit 61 determines whether the robot 1 contacts other objects. The control unit 61 determines whether the robot 1 contacts other objects based on the detection results of the encoder 31, the encoder 32, and the contact force sensor 33. Specifically, when the contact force detected by the contact force sensor 33 is greater than a first threshold, when the deviation of the angular position of the servo motor 14 obtained based on the detection result of the encoder 31 is greater than a second threshold, or when the deviation of the angular position of the servo motor 23 obtained based on the detection result of the encoder 32 is greater than a third threshold, the tactile control unit 68 determines that contact has occurred.

[0076] If no contact is made, the control unit 61 returns to step S101 and repeats the process from step S101. By repeating the process from step S101 at a predetermined control cycle, the robot 1 is continuously operated in response to the operation of the operating device 4 by the user.

[0077] During such an operation of the robot 1, when the robot 1 makes contact with another object, the control unit 61 proceeds from step S103 to step S104. In step S104, the control unit 61 determines whether the contact is a strong contact. Specifically, when the contact force, the deviation of the angular position of the servo motor 14, or the deviation of the angular position of the servo motor 23 used in the determination of step S103 by the tactile control unit 68 is above the corresponding upper limit value, the tactile control unit 68 determines that a strong contact has occurred.

[0078] When a strong contact is generated, the tactile control unit 68 outputs a command to generate an alarm vibration to the tactile device 8 in step S105. The tactile device 8 generates an alarm vibration. The user can know that the robot 1 has contacted another object with a large contact force through the alarm vibration of the tactile device 8. In this way, the user can stop the movement of the robot 1 or move the robot 1 in a manner to eliminate the contact of the robot 1. It should be noted that the control unit 61 can also force the robot 1 to stop in addition to generating the alarm vibration.

[0079] On the other hand, when the contact is not a strong contact, the tactile control unit 68 determines the number N of categories of contacts generated simultaneously (hereinafter referred to as "contact number N") in step S106. When the contact number N is 1, the tactile control unit 68 outputs a command to generate a first vibration to the tactile device 8 in step S107. When the contact number N is 2, the tactile control unit 68 outputs a command to generate a second vibration to the tactile device 8 in step S108. When the contact number N is 3, the tactile control unit 68 outputs a command to generate a third vibration to the tactile device 8 in step S109. The tactile device 8 generates vibrations in response to the command. Steps S105, S107, S108, and S109 are equivalent to operating the tactile device 8 when the robot 1 contacts other objects, and the tactile device 8 imparts tactile stimulation to the user.

[0080] In this way, when the robot 1 makes contact with another object during the operation of the robot 1, the tactile device 8 vibrates to notify the user of the occurrence of the contact. The user can know the occurrence of the contact through the vibration of the tactile device 8. In addition, the user can judge what kind of contact has occurred from the image of the robot 1 displayed on the display device 52. For example, sometimes a job includes a process that is accompanied by contact when it is completed. When the user is informed of the contact at a timing when the process can be predicted to be roughly completed based on the operation of the operating device 4 and the image of the display device 52, the user can judge that the process is completed and move to the next process. Alternatively, when the user is informed of the contact by the tactile device 8 even if the timing of the contact cannot be predicted based on the operation of the operating device 4 and the image of the display device 52, the user can know that an unexpected contact has occurred in a part that cannot be confirmed by the display device 52. And, when the vibration of the tactile device 8 at that time is an alarm vibration, it can be known that the unexpected contact is a contact accompanied by a large contact force. In this way, the user can make the robot 1 take an evasive action.

[0081] In more detail, in the example of the picking operation, the user first causes the robot 1 to perform the first moving process. Specifically, the user operates the operating device 4 and sets it to the first mode. The user moves the hand 2 toward the holding position by operating the direction key 41a and the second input button 41c or the fourth input button 41e as needed while observing the display device 52. The holding position is the position of the workpiece W in the container C. Steps S101, S102, and S103 are repeated, and the hand 2 gradually moves from the starting position to the holding position. Basically, the user determines the arrival of the hand 2 at the holding position based on the image of the display device 52. Assuming that the finger 22 accidentally contacts the bottom surface of the container C in the first moving process, the contact force sensor 33 detects the contact force (step S103). Since the number of contacts N at this time is 1, the tactile controller 68 causes the tactile device 8 to generate a first vibration (step S107). When the user senses the first vibration of the tactile device 8, the user can judge that the finger 22 is in contact with the bottom surface of the container C, etc., from the operation content of the operating device 4 and the image of the robot 1 on the display device 52. When the finger 22 contacts the bottom surface of the container C, etc., since the hand 2 roughly reaches the gripping position, the user eliminates the contact of the finger 22 and stops the movement of the robot 1.

[0082] Depending on the image quality of the display device 52 or the viewing angle of the camera device 51, it may not be possible to confirm the contact of the finger 22 with the bottom surface of the container C from the image of the display device 52. Even in that case, the user can recognize that the hand 2 is close to the container C from the image of the display device 52, and can know that the robot 1 has made contact from the vibration of the tactile device 8. In this way, the user can eliminate the accidental contact of the finger 22 and accurately judge that the finger 22 has roughly reached the gripping position.

[0083] When the first moving process is completed, the user causes the robot 1 to perform the gripping process. The user operates the operating device 4 to switch from the first mode to the second mode. For example, the user causes the finger 22 to perform a closing action by continuously pressing the first input button 41b. Steps S101, S102, and S103 are repeated, and the finger 22 approaches the workpiece W. When the finger 22 finally contacts the workpiece W and the contact force reaches the specified gripping force, the tactile control unit 68 detects the contact of the finger 22 based on the detection result of the encoder 32 (step S103). The finger 22 contacts the workpiece W, so that the number of contacts N becomes 1. The tactile control unit 68 causes the tactile device 8 to generate a first vibration (step S107). The user can know that the finger 22 has contacted the workpiece W by sensing the vibration of the tactile device 8. Since the user can judge the contact of the finger 22 from the operation content of the operating device 4 and the image of the robot 1 on the display device 52, when the user senses the first vibration of the tactile device 8, the completion of the gripping process is judged, and the movement of the finger 22 is stopped. Therefore, the contact at this time hardly reaches strong contact.

[0084] Depending on the image quality of the display device 52 or the viewing angle of the camera device 51, it may not be possible to confirm that the finger 22 is holding the workpiece W from the image of the display device 52. Even in that case, the user can recognize that the finger 22 is approaching the workpiece W from the image of the display device 52, and can know that the robot 1 has made contact from the vibration of the tactile device 8. In this way, the user can accurately determine the completion of the holding process through the vibration of the tactile device 8.

[0085] When the holding process is completed, the user causes the robot 1 to perform the second moving process. The user operates the operating device 4 to switch from the second mode to the first mode. The user moves the hand 2 toward the transfer position by operating the direction key 41a and the second input button 41c or the fourth input button 41e as needed while observing the display device 52. Steps S101, S102, and S103 are repeated, and the hand 2 gradually moves from the holding position to the transfer position. Since the detection of the contact of the finger 22 based on the encoder 32 continues while the hand 2 holds the workpiece W, the tactile device 8 continues to generate the first vibration.

[0086] When the hand 2 reaches the transfer position, the workpiece W is placed on the bottom surface of the tray T. In this way, the contact force sensor 33 detects the contact force received by the finger 22 from the bottom surface of the tray T via the workpiece W (step S103). The number of contacts N increases from 1 to 2, and the tactile control unit 68 causes the tactile device 8 to generate a second vibration (step S108). Since the user can judge the contact of the workpiece W from the operation content of the operating device 4 and the image of the robot 1 on the display device 52, when the user senses the second vibration of the tactile device 8, the second movement process is judged to be completed, and the movement of the robot 1 is stopped. Therefore, the contact at this time hardly reaches a strong contact.

[0087] Depending on the image quality of the display device 52 or the viewing angle of the camera device 51, it may not be possible to confirm the contact of the workpiece W with the bottom surface of the pallet T from the image of the display device 52. Even in that case, the user can recognize that the workpiece W is close to the bottom surface of the pallet T from the image of the display device 52, and can know that the robot 1 has made contact from the vibration of the tactile device 8. In this way, the user can accurately determine the completion of the second moving process through the vibration of the tactile device 8.

[0088] When the second moving process is completed, the user causes the robot 1 to perform the release process. The user operates the operating device 4 to switch from the first mode to the second mode. For example, the user causes the finger 22 to perform the opening action by continuously pressing the third input button 41d. Steps S101, S102, and S103 are repeated, and the finger 22 is separated from the workpiece W. When the finger 22 is separated from the workpiece W, the contact force based on the encoder 32 becomes zero. In addition, the contact force acting on the finger 22 from the bottom surface of the tray T through the workpiece W also becomes zero. The tactile control unit 68 stops the vibration of the tactile device 8. Since the vibration of the tactile device 8 stops, the user can know that the contact of the finger 22 has been eliminated. Since the user can judge the elimination of the contact of the finger 22 from the operation content of the operating device 4 and the image of the robot 1 on the display device 52, when the user senses that the vibration of the tactile device 8 has stopped, he judges that the release process is completed and stops the movement of the finger 22.

[0089] Depending on the image quality of the display device 52 or the viewing angle of the camera device 51, when the fingers are not widely opened, it may not be possible to confirm the release of the grip of the workpiece W from the image of the display device 52. Even in that case, the user can know that the contact of the robot 1 with other objects has been eliminated by the reduction or elimination of the vibration of the tactile device 8. In this way, the user can accurately judge the completion of the release process by the reduction or elimination of the vibration of the tactile device 8.

[0090] When the release process is completed, the picking operation is completed. It should be noted that the picking operation may be completed by returning the robot 1 to the starting state after the release process.

[0091] In addition, during such a picking operation, when an unexpected contact of the robot 1 occurs, the tactile device 8 can generate an alarm vibration to notify the user of the occurrence of the contact of the robot 1 and urge the user to take evasive action. That is, when an unexpected contact occurs, since the user does not immediately stop the action of the robot 1, a large contact force may act on the robot 1. In particular, when a part of the robot 1 that is not displayed on the display device 52 contacts other objects, since the user cannot know the occurrence of the contact from the display device 52, a large contact force is likely to be generated. When the tactile control unit 68 detects a contact with a large contact force, it determines that a strong contact has occurred in step S104, and causes the tactile device 8 to generate an alarm vibration in step S105. In this way, the user can know the occurrence of the contact of the robot 1 that cannot be detected by the display device 52, and can eliminate the contact of the robot 1 at an early stage.

[0092] In this way, the remote control system 100 notifies the user of the occurrence of contact of the robot 1 through the tactile sense of the user through the tactile device 8. The user determines the cause of the contact based on the operation of the operating device 4, the image of the robot 1 on the display device 52, and the vibration mode. If the contact is related to the completion of the process, the user moves to the next process. When the contact is unexpected, the user stops the action of the robot 1 or causes the robot 1 to perform an action to eliminate the contact. In other words, the user can accurately determine the status of the robot 1.

[0093] It should be noted that, as long as the picking operation is performed appropriately, the number of contacts N will not become 3. However, depending on the operation of the robot 1, the number of contacts N may become 3. At this time, the haptic device 8 generates the third vibration.

[0094] Furthermore, in the above description, the operating device 4 continuously inputs the continuous motion, i.e., the motion trajectory, of the robot 1. That is, the robot 1 performs motion while the input from the operating device 4 continues, and stops when the input from the operating device 4 stops. However, the operation input from the operating device 4 is not limited to this. For example, a command may be input from the operating device 4, and the control device 6 may autonomously control the robot 1 so that the robot 1 performs the motion in response to the command.

[0095] For example, in the picking operation, the operating device 4 inputs multiple commands corresponding to multiple processes from the user. The multiple commands are an execution command for the first moving process, an execution command for the holding process, an execution command for the second moving process, and an execution command for the release process. For example, the execution commands for each process may be assigned to the first input button 41b, the second input button 41c, the third input button 41d, and the fourth input button 41e of the operating device 4. Alternatively, the commands for each process may be displayed as options on the display device 52, a command may be selected by the direction key 41a, and a command may be determined by the first input button 41b, so that the desired command may be input from the operating device 4 to the control device 6.

[0096] The control device 6 causes the robot 1 to execute a process corresponding to the command input to the operating device 4. Specifically, the input processing unit 64 determines the process to be executed based on the operation information from the operating device 4. The command position generation unit 65 generates a trajectory of the robot 1 for realizing the action corresponding to the determined process. The trajectory of the robot 1 can be generated based on pre-set teaching information, or the trajectory generated in this way can be corrected and generated by the captured image captured by the camera device 51. The command position generation unit 65 sequentially outputs the command positions for realizing the generated trajectory of the robot 1 to the command angle conversion unit 66. The control content of the command angle conversion unit 66 and the motion control unit 75 is the same as when the continuous action is input.

[0097] Specifically describing the picking operation performed by command input, first, the user inputs an execution command for the first moving process via the operating device 4. The control unit 61 causes the robot 1 to operate so that the hand 2 moves from the starting position to the holding position. After inputting the execution command for the first moving process, the user waits for the completion of the first moving process. When the hand 2 reaches the holding position, the finger 22 contacts the bottom surface of the container C. The contact force sensor 33 detects the contact force, and the tactile device 8 generates a first vibration. Even if the user finds it difficult to determine the completion of the first moving process using only the image of the robot 1 on the display device 52, the user can still accurately know the completion of the first moving process through the vibration of the tactile device 8.

[0098] When the user learns that the first moving process is completed, the user inputs an execution command for the gripping process via the operating device 4. The control unit 61 causes the finger 22 to perform a closing action. After inputting the execution command for the gripping process, the user waits for the gripping process to be completed. When the finger 22 contacts the workpiece W and grips the workpiece W with a specified gripping force, the vibration of the tactile device 8 switches from the first vibration to the second vibration. It is difficult for the user to determine whether the finger 22 has gripped the workpiece W with a specified gripping force using only the image of the robot 1 on the display device 52. The user can accurately know that the gripping of the workpiece W by the finger 22 has been completed through the vibration of the tactile device 8.

[0099] When the user learns that the gripping process is completed, the user inputs an execution command for the second moving process via the operating device 4. The control unit 61 operates the robot 1 to move the hand 2 from the container C to the tray T. After inputting the execution command for the second moving process, the user waits for the completion of the second moving process. In the second moving process, since the finger 22 leaves the bottom surface of the container C, the contact number N decreases from 2 to 1, and the vibration of the tactile device 8 switches from the second vibration to the first vibration. When the hand 2 reaches the transfer position of the tray T, the finger 22 contacts the bottom surface of the tray T. Since the contact number N increases from 1 to 2, the vibration of the tactile device 8 switches from the first vibration to the second vibration. Even if it is difficult for the user to judge the completion of the second moving process using only the image of the robot 1 on the display device 52, the user can still accurately know the completion of the second moving process through the vibration of the tactile device 8.

[0100] When the user learns that the second moving process is completed, the user inputs an execution command for the release process via the operating device 4. The control unit 61 causes the finger 22 to perform an opening action. After inputting the execution command for the release process, the user waits for the completion of the release process. When the finger 22 leaves the workpiece W, the vibration of the tactile device 8 switches from the second vibration to the first vibration. It is difficult for the user to determine that the finger 22 has left the workpiece W using only the image of the robot 1 on the display device 52. The user can accurately learn that the release of the grip of the workpiece W by the finger 22 has been completed through the vibration of the tactile device 8.

[0101] When the picking-up operation is repeated, the user inputs the execution command of the first moving process again via the operating device 4. On the other hand, when the user inputs the end command via the operating device 4, the picking-up operation ends.

[0102] In this way, even when the picking operation is performed according to the command input, the user can know the completion of each process through the vibration of the haptic device 8. Therefore, the user does not need to wait unnecessarily for the completion of the process, and can smoothly transfer to the next process.

[0103] As described above, in the first aspect of the technology of the present invention, the remote control system 100 includes a robot 1, an operating device 4, a display device 52, a tactile device 8, and a control device 6. The operating device 4 accepts input from a user, the display device 52 presents an image of the robot 1 to the user, the tactile device 8 provides stimulation to the user's sense of touch, and the control device 6 controls the robot 1 according to the input to the operating device 4. The control device 6 activates the tactile device 8 when the robot 1 contacts another object.

[0104] According to this structure, when the robot 1 is remotely controlled via the operating device 4, the user can accurately know the contact between the robot 1 and other objects, which is difficult to judge only by the image of the robot 1 on the display device 52, through the action of the tactile device 8. In addition, since it is not necessary to improve the image quality of the display device 52 in order to improve the accuracy of judging the contact of the robot 1, the delay of the image displayed on the display device is also reduced. In this way, the user can accurately judge the status of the robot 1.

[0105] Furthermore, in the second aspect of the technology of the present invention, in the remote control system 100 involved in the first aspect, the control device 6 causes the robot 1 to perform a task including multiple processes based on the input to the operating device 4, and causes the tactile device 8 to operate when at least one of the multiple processes is completed and the robot 1 contacts other objects.

[0106] According to this structure, the user can accurately know the completion of the process. That is, even when the contact of the robot 1 cannot be correctly determined only by the image of the robot 1 on the display device 52, the tactile device 8 notifies the user of the occurrence of the contact of the robot 1. The user can accurately determine that the contact is the contact of the robot 1 when the process is completed by considering the operation of the operating device 4 performed by the user and the image of the display device 52 together.

[0107] Furthermore, in a third aspect of the present invention, in the remote control system 100 according to the second aspect, the operating device 4 receives input of a plurality of commands corresponding to a plurality of processes, and the control device 6 causes the robot to execute the processes corresponding to the commands input to the operating device 4 .

[0108] According to this structure, a command is inputted via the operating device 4, and the control device 6 causes the robot 1 to execute a process corresponding to the inputted command. Among the multiple processes, the process that is accompanied by the contact of the robot 1 when the process is completed is a process in which the contact of the robot 1 when the process is completed is notified to the user through the action of the tactile device 8. That is, the user can input a command via the operating device 4, wait for the completion of the process, and accurately know the completion of the process through the action of the tactile device 8. As a result, the user does not need to wait unnecessarily for the completion of the process, and can smoothly transfer to the next process.

[0109] Furthermore, in a fourth aspect of the present invention, in the remote control system 100 according to the third aspect, the control device 6 changes the operation mode of the haptic device 8 in response to the number of types of contacts occurring simultaneously.

[0110] According to this structure, the control device 6 can inform the user of the number of types of contacts that occur simultaneously through the vibration of the tactile device 8. Even when contacts have occurred and the tactile device 8 is in operation, the user can be informed that new types of contacts have occurred or that part of the contacts that have occurred have been eliminated.

[0111] Furthermore, in a fifth aspect of the present invention, in the remote control system 100 according to any one of the first to third aspects, the control device 6 determines the type of contact of the robot 1 and operates the haptic device 8 for each contact of different types.

[0112] According to this configuration, even if a plurality of types of contacts occur simultaneously, the control device 6 can distinguish each type of contact.

[0113] Furthermore, in a sixth aspect of the technology of the present invention, in the remote control system 100 involved in any one of the first to fifth aspects, the control device 6 determines the magnitude of the contact force of the robot 1 and changes the action mode of the tactile device 8 in response to the magnitude of the contact force.

[0114] According to this structure, the action mode of the tactile device 8 is changed in response to the magnitude of the contact force. The user can know the magnitude of the contact force in addition to the occurrence of the contact.

[0115] In a seventh aspect of the technology of the present invention, the remote control system 100 involved in any one of the first to sixth aspects further includes a sensor 3 for detecting a force acting on the robot 1. The control device 6 determines the contact of the robot 1 with other objects based on the detection result of the sensor 3.

[0116] According to this configuration, the control device 6 determines the contact of the robot 1 based on the detection result of the sensor 3 .

[0117] Furthermore, in the eighth aspect of the technology of the present invention, in the remote control system 100 involved in the seventh aspect, the robot 1 has a robot arm 11 and a hand 2, the hand 2 is provided with fingers 22 for opening and closing actions and is connected to the robot arm 11, the control device 6 enables the robot 1 to perform the picking operation of the workpiece W, the sensor 3 includes a contact force sensor 33 (first sensor) and an encoder 32 (second sensor), the contact force sensor 33 detects the force acting on the finger 22 from the surface on which the workpiece W is placed, and the encoder 32 detects the force acting on the finger 22 in the opening and closing direction from the workpiece W to the finger 22.

[0118] According to this structure, during the picking operation, the finger 22 of the hand 2 contacts the surface on which the workpiece W is placed, or the finger 22 holds the workpiece W. That is, during the picking operation, the finger 22 contacts several times, and the contact of the finger 22 occurs at the timing when a part of the process is completed. That is, during the picking operation, by notifying the user of the contact of the finger 22 through the tactile device 8, the user can be notified of the completion of the process.

[0119] If the viewpoint is changed, then in the ninth aspect of the technology of the present invention, the remote control system 100 includes a robot 1, an operating device 4, a display device 52, a tactile device 8 and a control device 6, the operating device 4 accepts input from a user, the display device 52 presents an image of the robot 1 to the user, the tactile device 8 provides stimulation to the user's tactile sense, the control device 6 enables the robot 1 to perform a task including multiple processes based on the input to the operating device 4, and the control device 6 operates the tactile device 8 when at least one of the multiple processes is completed.

[0120] According to this structure, when the robot 1 is remotely controlled by the operating device 4, the user can accurately know the completion of the process that is difficult to judge only by the image of the robot 1 on the display device 52 through the action of the tactile device 8. In addition, since it is not necessary to improve the image quality of the display device 52 in order to improve the judgment accuracy of the completion of the process of the robot 1, the delay of the image displayed on the display device is also reduced. In this way, the user can accurately judge the status of the robot 1.

[0121] At this time, the control device 6 operates the tactile device 8 according to whether the process is completed. In addition, the control device 6 determines the completion of the process according to whether the contact of the robot 1 is generated. In the above-mentioned picking operation, the contact state of the robot 1 changes when the process is completed. Therefore, operating the tactile device 8 according to the contact of the robot 1 is equivalent to operating the tactile device 8 according to the completion of the process. However, there are also operations in which the contact state of the robot 1 does not necessarily change when the process is completed. In that case, the control device 6 can also operate the tactile device 8 when the process is completed even if the contact of the robot 1 is not generated.

[0122] Figure 6 The flowchart of the remote control at that time is shown. Figure 6 In the flowchart, from Figure 5 In the flowchart of FIG. 1 , steps S103, S106, S107, S108, and S109 are omitted, and steps S206 and S207 are added instead. Specifically, the control unit 61 does not determine whether the robot 1 contacts other objects during the operation of the robot 1, but determines whether strong contact occurs in step S104. The processing of step S104 is the same as that of step S106. Figure 5As in the flowchart of , when strong contact occurs, the tactile control unit 68 outputs a command to generate an alarm vibration to the tactile device 8 in step S105. Instead of monitoring the contact of the robot 1, the control unit 61 determines whether the process is completed in step S206. As long as there is no strong contact and the process is not completed, the control unit 61 repeats steps S101 and S102 to operate the robot 1 according to the input to the operating device 4. The control unit 61 can determine the completion of the process in various ways. One of the methods is that the control unit 61 determines the completion of the process by whether the contact of the robot 1 occurs. Among the multiple processes, there are processes that are accompanied by the contact of the robot 1 when they are completed. The control unit 61 can determine the completion of such a process based on the contact of the robot 1. For the process that cannot be determined to be completed based on the contact of the robot 1, the control unit 61 can also determine the completion based on an external signal such as an input from a user via the operating device 4. And when the process is completed, the control unit 61 vibrates the tactile device 8 in step S207. At this time, as described above, the control unit 61 may also change the vibration mode of the tactile device 8 in response to the number of contacts N.

[0123] In the tenth aspect of the technology in the present invention, the remote control method of a robot that controls the robot 1 via the operating device 4 includes the steps of accepting input from a user via the operating device 4, causing the robot 1 to move according to the input to the operating device 4, presenting an image of the moving robot 1 to the user via the display device 52, and causing the tactile device 8 to move to impart tactile stimulation to the user when the robot 1 contacts other objects.

[0124] According to this structure, when the robot 1 is remotely controlled via the operating device 4, the user can accurately know the contact between the robot 1 and other objects, which is difficult to determine only by the image of the robot 1 on the display device 52, through the action of the tactile device 8. In addition, since it is not necessary to improve the image quality of the display device 52 in order to improve the accuracy of determining the contact of the robot 1, the delay of the image displayed on the display device is also reduced. In this way, the user can accurately determine the status of the robot 1.

[0125] Furthermore, in the eleventh aspect of the technology in the present invention, the remote control program 62a is a program for causing the control unit 61 (computer) to realize a function of controlling the remote control system 100, the remote control system 100 including the robot 1, the operating device 4, and the tactile device 8, the operating device 4 accepting input from the user, and the tactile device 8 imparting stimulation to the user's sense of touch. The remote control program 62a causes the control unit 61 to realize a function of accepting input from the user via the operating device 4, a function of causing the robot 1 to operate according to the input to the operating device 4, and a function of causing the tactile device 8 to operate to impart stimulation to the user's sense of touch when the robot 1 contacts another object.

[0126] According to this configuration, when the robot 1 is remotely controlled via the operating device 4 , the user can accurately know the contact between the robot 1 and other objects through the operation of the tactile device 8 . In this way, the user can accurately judge the status of the robot 1 .

[0127] (Other embodiments)

[0128] As described above, the embodiments are described as examples of the technology invented in this application. However, the technology of the present invention is not limited thereto, and can also be applied to embodiments that have been appropriately changed, replaced, added, omitted, etc. Furthermore, the various constituent elements described in the embodiments can also be combined as new embodiments. Furthermore, the constituent elements recorded in the drawings and detailed descriptions include not only the constituent elements necessary to solve the problem, but also constituent elements that are not necessary to solve the problem in order to exemplify the technology. Therefore, it should not be immediately assumed that those non-essential constituent elements are necessary just because they are recorded in the drawings and detailed descriptions.

[0129] The structure of the remote control system 100 is just an example and is not limited thereto. For example, the operation performed by the robot 1 is not limited to the picking operation. The operation performed by the robot 1 may also be a processing operation. Furthermore, the end effector of the robot arm 11 is not limited to the hand 2. The end effector may be changed in response to the operation performed by the robot 1.

[0130] Furthermore, the robot 1 is not limited to an industrial robot, and for example, the robot 1 may be a medical robot.

[0131] The sensor 3 for detecting the force acting on the robot 1 is not limited to the encoder 31, the encoder 32, and the contact force sensor 33. Force sensors other than these may also be provided. Furthermore, the sensor 3 only needs to include at least one of the encoder 31, the encoder 32, and the contact force sensor 33, and may not include a part of the encoder 31, the encoder 32, and the contact force sensor 33. Furthermore, instead of the encoder 31 or the encoder 32, a current sensor for detecting the current of the servo motor 14, a torque sensor for detecting the torque of the servo motor 14, a current sensor for detecting the current of the servo motor 23, or a torque sensor for detecting the torque of the servo motor 23 may be used. The force acting on the robot 1 can be detected based on the deviation between the current value and the target value of the current or the torque.

[0132] The imaging device 51 is not limited to the device mounted on the robot arm 11. The imaging device 51 may be fixedly arranged at a position where the robot 1 can be imaged.

[0133] The above structure is an example of the operating device 4. The operating device 4 may also have an operating unit such as an operating lever. The operating device 4 may also have an operating unit such as a touch panel. The input through the operating device 4 may include the moving speed of the robot 1 in addition to the moving direction of the robot 1. For example, when the operating device 4 has an operating unit such as an operating lever, the moving speed may be input by tilting the operating lever.

[0134] The tactile device 8 is not limited to the above structure. The tactile device 8 can be worn by the user in any manner as long as it can provide tactile stimulation to the user. The tactile device 8 can also be built into the operating device 4. In addition, the vibration actuator of the tactile device 8 is not limited to the voice coil motor 81. The vibration actuator of the tactile device 8 can also be an eccentric rotating mass method or a piezoelectric method.

[0135] The tactile device 8 may also have a plurality of vibration actuators. The combination of the plurality of vibration actuators may also be changed in response to various contacts of the robot 1, thereby notifying the user of the type of contact of the robot 1. For example, the type of contact of the robot 1 may be distinguishably notified to the user using the technology of phantom tactile sensation.

[0136] The vibration performed by the tactile device 8 is not limited to the first vibration, the second vibration, the third vibration, and the alarm vibration. The contact that may be generated in the robot 1 may change in response to the structure of the robot 1 and the operation of the robot 1. The vibration of the tactile device 8 may be set in a manner corresponding to the contact of the robot 1 imagined in response to the structure of the robot 1 and the operation of the robot 1. For example, the vibration performed by the tactile device 8 may be changed in two stages or four stages or more.

[0137] The above block diagram is an example, and multiple blocks may be implemented as one block, one block may be divided into multiple blocks, or part of the functions may be transferred to other blocks.

[0138] The technology of the present invention may be a non-transitory computer-readable recording medium recording the remote control program 62a. Furthermore, the remote control program 62a may be a program that can be distributed via a transmission medium such as the Internet.

[0139] The flowchart is just an example. The steps in the flowchart may be appropriately changed, replaced, added, omitted, etc. Furthermore, the order of the steps in the flowchart may be changed, or serial processing may be performed in parallel.

[0140] The functions implemented by the components described in this specification may also be implemented in a circuit or processing circuitry that includes a general-purpose processor, a special-purpose processor, an integrated circuit, an ASIC (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), a conventional circuit, and / or a combination thereof that is programmed to implement the described functions. The processor includes transistors and other circuits and is considered a circuit or processing circuit. The processor may also be a programmable processor that executes a program stored in a memory.

[0141] In this specification, a circuit, unit or device is hardware programmed or executed to implement the described functions. The hardware may also be various hardware invented in this specification or various hardware well-known as programmed or executed hardware to implement the described functions.

[0142] When the hardware is a processor of a type regarded as a circuit, the circuit, device or unit is a combination of hardware and software used to constitute the hardware and / or processor.

Claims

1. A remote control system, characterized in that: The remote control system includes a robot, an operating device, a display device, a tactile device, and a control device, wherein the operating device receives input from a user, the display device presents an image of the robot to the user, the tactile device provides stimulation to the user's sense of touch, and the control device controls the robot according to the input to the operating device. The control device activates the tactile device when the robot contacts other objects.

2. The remote control system according to claim 1, characterized in that: The control device causes the robot to perform a task including a plurality of steps based on input to the operating device. When the robot contacts another object when at least one of the plurality of processes is completed, the haptic device is activated.

3. The remote control system according to claim 2, characterized in that: The operating device receives input of a plurality of commands corresponding to the plurality of processes, The control device causes the robot to execute a process corresponding to a command input to the operation device.

4. The remote control system according to claim 3, characterized in that: The control device changes the operation mode of the tactile device in response to the number of types of contacts generated simultaneously.

5. The remote control system according to claim 1, characterized in that: The control device determines the type of contact of the robot and operates the tactile device for each contact of different types.

6. The remote control system according to claim 1, characterized in that: The control device determines the magnitude of the contact force of the robot and changes the action mode of the tactile device in response to the magnitude of the contact force.

7. The remote control system according to claim 1, characterized in that: The remote control system further comprises a sensor, the sensor detecting a force acting on the robot, The control device determines the contact of the robot with other objects based on the detection result of the sensor.

8. The remote control system according to claim 7, characterized in that: The robot comprises a robot arm and a hand, wherein the hand is provided with fingers for opening and closing and is connected to the robot arm. The control device enables the robot to perform a workpiece picking operation. The sensor includes a first sensor that detects a force acting on the finger from a surface on which the workpiece is placed, and a second sensor that detects a force acting on the finger from the workpiece in an opening and closing direction of the finger.

9. A remote control system, characterized in that: The remote control system includes a robot, an operating device, a display device, a tactile device, and a control device, wherein the operating device receives input from a user, the display device presents an image of the robot to the user, the tactile device provides stimulation to the user's sense of touch, and the control device causes the robot to perform a task including a plurality of steps according to the input to the operating device. The control device operates the haptic device when at least one of the plurality of steps is completed.

10. A remote control method for a robot, wherein the robot is controlled via an operating device, and wherein: The remote control method includes the steps of accepting input from a user via the operating device, causing the robot to move according to the input to the operating device, presenting an image of the moving robot to the user via a display device, and causing a tactile device to move to impart tactile stimulation to the user when the robot contacts another object.

11. A remote control program for enabling a computer to realize a function of controlling a remote control system, wherein the remote control system comprises a robot, an operating device and a tactile device, wherein the operating device receives input from a user, and the tactile device imparts tactile stimulation to the user, wherein: The remote control program enables the computer to realize functions of accepting input from the user via the operating device, operating the robot according to the input to the operating device, and operating a tactile device that provides tactile stimulation to the user when the robot contacts another object.

Citation Information

Patent Citations

  • Robot control device and method

    JP2010269418A

  • Contact detector, hand, robot, and robot system

    JP2018192599A

  • Hand and robot

    JP2020131378A

  • Workpiece contact state estimating device and workpiece contact state estimation method

    US20180071915A1

  • User interface for a teleoperated robot

    US20180154518A1