ROBOT LEARNING DEVICE AND ROBOT SYSTEM

Abstract

Robot teaching device (30) that teaches a robot, wherein the robot teaching device (30) an image acquisition unit (34) that acquires distance images representing distance information of a subject to be photographed or two-dimensional images of the subject to be photographed as moving images; a trajectory detection unit (311) that detects the movement trajectory of an operator's hand captured as a recording object in the moving images using the moving images; a detection unit (312) which detects whether the detected trajectory represents a straight-line trajectory or a rotational trajectory; and a command generation unit (313) that generates a command that makes a specific movable portion of the robot perform a translational movement based on the movement trajectory when it is detected that the movement trajectory is a rectilinear movement, and makes the specific movable portion of the robot perform a rotational movement based on the movement trajectory when it is detected that the movement trajectory is a rotational movement.

Description

[0001] The present invention relates to a robot teaching apparatus and a robot system.

[0002] When performing various tasks using a robot system that uses an industrial robot, it is necessary to train the robot to perform the intended operation. Industrial robot training devices generally include operation buttons that enable so-called step-by-step operation and an operation screen.

[0003] As another example of a teaching device, International Publication WO 2017 / 130389 A1 describes a “robot teaching device configured to include an image input device that acquires images depicting a worker’s fingers and a work object; a finger movement detection unit that acquires the movement of the worker’s fingers from the images acquired by the image input device; and a work content presuming unit that presumes the content of the worker’s work on the work object from the movement of the fingers detected by the finger movement detection unit, wherein a control program creation unit creates a control program for the robot that reflects the work content presumed by the work content presuming unit” (paragraph 0006).

[0004] Furthermore, EP 1 435 737 A1 discloses an augmented reality system comprising a camera movably arranged to capture an image at a local location, a registration unit that generates graphics and registers the generated graphics with the image from the camera to provide a composite augmented reality image, and a display device.

[0005] WO 2011 / 065035 A1 discloses a method for creating teaching data for a robot and a teaching system for the robot. A monocular camera and a stereo camera capture a teaching image including a teacher's wrist and hand.

[0006] Furthermore, US 2015 / 0 314 442 A1 discloses a system for generating instructions for operating a robot to perform work on a workpiece. The system includes data from one or both hands of a gesture performed with one or both hands, and a computing device with program code configured to process the robot scene data and the gesture data to generate an instruction for operating the robot.

[0007] US 2014 / 0 022 171 A1 describes a system and method for implementing a remote-controlled user interface with object tracking.

[0008] JP 2018 - 015 863 A describes a robot system that can generate teaching data quickly and easily.

[0009] JP 2018 010 538 A discloses an image recognition device that can accurately identify rotational movements in gesture manipulations using a simple system.

[0010] When teaching an industrial robot using a teaching device, teaching methods that involve operating the stepping buttons of the teaching device or performing touch or slide operations on the screen of the teaching device are often performed. However, when teaching by operating the stepping buttons, the degree of freedom for teaching is limited by the number of stepping buttons, while when teaching by touch or slide operations on the screen, the degree of freedom for teaching is limited to a two-dimensional plane. Therefore, a robot teaching device that has an even higher degree of freedom and can teach the robot through intuitive operation, as well as a robot system, are desired.

[0011] The invention is therefore based on the object of providing a robot teaching device that has a high degree of freedom and teaches the robot through intuitive operation. Furthermore, the invention is based on the object of providing a robot system.

[0012] According to the invention, the above-mentioned object is achieved with regard to the robot teaching device by the subject matter of claim 1. With regard to the robot system, the above-mentioned object is achieved by the subject matter of claim 5.

[0013] Specifically, the object is achieved by a robot teaching device that teaches a robot, the robot teaching device comprising an image acquisition unit that acquires distance images representing distance information of a subject to be photographed or two-dimensional images of the subject to be photographed as moving images; a trajectory detection unit that detects, using the moving images, the movement trajectory of an operator's hand, which is photographed as a subject to be photographed, in the moving images; a recognition unit that recognizes whether the detected movement trajectory represents a rectilinear movement trajectory or a rotational movement trajectory;and a command generation unit that generates a command that makes a specific movable portion of the robot perform a translational motion based on the trajectory when it is detected that the trajectory is a rectilinear motion, and makes the specific movable portion of the robot perform a rotational motion based on the trajectory when it is detected that the trajectory is a rotational motion;

[0014] According to a subsidiary aspect, the invention relates to a robot system comprising a robot; a robot control device that controls the robot; and the robot teaching device described above, wherein the robot control device controls the robot based on the command outputted from the command generation unit of the robot control device.

[0015] The objects, features and advantages of the present invention will become more apparent from the following explanation of an embodiment taken in conjunction with the accompanying drawings. is Fig. 1 is a view showing the entire structure of a robot system having a robot teaching apparatus according to an embodiment, is Fig. 2 is a block diagram showing the functions of the robot teaching device, is Fig. 3 is a view showing a configuration example of the front side of the robot teaching device when the robot teaching device is configured as a teaching control panel, is Fig. 4 is a schematic view showing a state in which a gesture input expressing a linear movement is made, is Fig. 5 is a schematic view showing a state in which a gesture input expressing a rotational movement is made, is Fig. 6 is a flowchart showing gesture teaching processing, is Fig. 7 is a flowchart illustrating palm orientation detection processing, is Fig. 8 is a schematic view showing a state in which the position of the fingertips has been detected by the palm orientation detection processing, and is Fig. 9 is a schematic view showing the arrangement relationship of a captured image, a lens, a reference plane, and the hand when three-dimensional position information of the hand is obtained using a two-dimensional image.

[0016] An embodiment of the present disclosure will be explained below with reference to the accompanying drawings. Corresponding structural elements are designated by common reference numerals throughout the figures. The scale of these drawings is arbitrarily changed. The embodiments shown in the drawings represent an example of the implementation of the present invention, but the present invention is not limited to the illustrated embodiments.

[0017] Fig. 1 is a view illustrating the entire structure of a robot system 100 including a robot teaching device 30 according to an embodiment. Fig. 2 is a block diagram illustrating the functions of the robot teaching device 30. As in Fig. 1, the robot system 100 includes a robot 10, a robot controller 20 that controls the robot 10, and the robot teaching device 30 for teaching the robot 10. The robot 10 is, for example, a vertically articulated robot, but a robot of other types may also be used. The robot controller 20 controls the operation of the robot 10 by controlling motors of each axis of the robot 10. The robot controller 20 may also have a general computer configuration including a CPU, a ROM, a RAM, a storage device, a display unit, an operation unit, a communication unit, etc.

[0018] The robot teaching device 30 is, for example, a teaching operation panel or a tablet terminal type teaching device and has a structure as a general computer with a CPU 31, a memory 32 such as a ROM, a RAM, or the like, a communication interface 33, etc. as hardware components. The robot teaching device 30 is connected to the robot control device 20 via the communication interface 33. In addition, the robot teaching device 30 includes an operation unit 35 having various operation buttons as a user interface for performing the teaching operation, and a display unit 36 ​​(see Fig. 3). The robot teaching device 30 in the present embodiment further includes an image acquisition unit 34 that captures a hand of an operator performing a gesture input.

[0019] The image acquisition unit 34 is a range image camera that acquires range images representing distance information depending on the spatial position of the subject as moving images, or a camera that acquires two-dimensional images of the subject as moving images. As an example, the image acquisition unit 34 may be configured as a range image camera including a light source, a projector that projects patterned light, and two cameras arranged across the projector on both sides of the projector. The image acquisition unit 34 captures the subject onto which the patterned light is projected from the two cameras arranged at different positions, and acquires three-dimensional position information of the subject by the stereo method.Or, the image acquisition unit 34 as a range camera may be a ToF (Time of Flight) camera that includes a light source and image pickup elements and measures the time until light hits and returns from the object to be photographed. Or, the image acquisition unit 34 as a range camera may be a stereo camera configured to detect the position of the object to be photographed by image recognition in each of images captured by two cameras and to acquire three-dimensional position information of the object through the stereo method. Other types of devices may also be used as the range camera. In the configuration example shown in FIG. Fig. 1 to 3, the image acquisition unit 34 is implemented as a device built into the robot teaching device 30, but the image acquisition unit 34 may also be implemented as an external device connected to the robot teaching device 30 in a wired or wireless manner.

[0020] Fig. 3 is a view showing a configuration example of the front side of the robot teaching device 30 when the robot teaching device 30 is configured as a teaching control panel. As shown in Fig. 3, the image acquisition unit 34 is arranged at the front of the robot teaching device 30, on which the operation unit 35 and the display unit 36 ​​are mounted, and captures the object to be captured from the front. With this configuration, the operator can hold the robot teaching device 30 with one hand and perform gesture input by moving the other hand. As will be explained in detail below, a certain movable portion of the robot 10 is used to perform a translational movement ( Fig. 4) when the movement of the operator's hand is a rectilinear movement, and the specific movable section of the robot 10 is caused to perform a rotary movement ( Fig. 5) when the movement of the operator's hand is a rotary movement. As an example, the specific movable portion is assumed to be the tip end 11 of the arm of the robot 10.

[0021] As in Fig. 2, the robot teaching device 30 includes, in addition to the image acquisition device 34, a trajectory detection unit 311, a recognition unit 312, and a command generation unit 313. These functions are implemented, for example, by the CPU 31 executing software stored in the memory 32. The trajectory detection unit 311 detects the trajectory of the operator's hand, which is depicted as a recognition object in the moving images, using the moving images acquired by the image acquisition unit 34. The recognition unit 312 detects whether the detected trajectory of the hand represents a rectilinear movement or a rotational movement. The command generation unit 313 generates a command that causes the tip end 11 of the arm of the robot 10 to perform a translational movement based on the trajectory if it is detected that the trajectory represents a rectilinear movement (see Fig. 4), and issues a command that makes the tip end 11 of the arm of the robot 10 perform a rotational movement based on the trajectory (see Fig. 5), if it has been recognized that the movement path represents a rotational movement (see Fig. 5). The robot control device 20 controls the robot 10 according to the command from the robot teaching device 30.

[0022] Fig. Figure 6 is a flowchart illustrating the processing of teaching by gestures (hereinafter referred to as gesture teaching processing). The gesture teaching processing of Fig. 6 is started, for example, when a specific operation is performed on the robot teaching device 30 via the operation unit 35. The gesture teaching processing is executed under the control of the CPU 31 of the robot teaching device 30. When the gesture teaching processing is started, an image is captured by the image acquisition device 34. When performing the gesture input, it is assumed that the operator performs the operation, taking into account that the image captured by the image acquisition device 34 only captures the area of ​​one of the operator's hands, and other locations such as the face are not imaged.

[0023] First, the robot teaching device 30 (the trajectory recognition unit 311) performs hand recognition in each frame of the moving images acquired by the image acquisition device 34 (step S11). The hand recognition in frame units will be explained both for the case where the image acquisition device 34 is a range camera and for the case where it is a two-dimensional camera. Step S11 for a distance imaging camera:

[0024] In the images of each frame captured by the range camera, the distance to the subject is represented, for example, as a shading. The operator's hand can be viewed in the range images as pixels concentrated in a certain distance range as a cluster (that is, a collection of pixels whose distance hardly changes between adjacent images). Therefore, by marking the pixels located in the certain distance range in the range images, it is possible to recognize the cluster of marked pixels as a hand. By performing image recognition according to this method under the condition that the operator moves the hand at a position within a certain specified distance from the image acquisition device 34 (for example, approximately 15 cm), the recognition accuracy can be increased. Step S11 for a two-dimensional camera:

[0025] When the image acquiring device 34 is a camera that acquires moving images from two-dimensional images, the trajectory recognition unit 11 may perform recognition by any of the methods described below. (A1): The hand is detected by grouping skin-colored pixels in the images. In this case, the skin-colored area can be accurately detected by converting the color information of the images from RGB, for example, to HSV (chroma, saturation, and brightness) and extracting the skin-colored area. (A2): Image recognition is performed using the deep learning method. As an example, a convolutional neural network is pre-trained on multiple hand images and then performs hand recognition in the images of each frame. In this case, learning can also be performed under the condition that the operator performs the gesture input with an open hand, for example, by preparing images of numerous people's open hands as training data. (A3): Hand recognition is performed through pattern matching. In this case, pattern matching can also be performed under the condition that the operator performs the gesture input with a specific hand posture (e.g., the open hand posture).

[0026] The detection of the palm orientation at step S11 will be discussed later.

[0027] When the hand is detected for individual frames in this way, the trajectory detection unit 311 tracks the movement of the hand between frames, thereby obtaining the hand's trajectory (step S12). In step S12, the trajectory detection unit 311 determines the center of gravity of the detected hand for each frame. Here, various methods can be used to determine the geometric center of gravity of the hand based on the two-dimensional position information of the hand detected in step S11. The trajectory detection unit 311 obtains the hand's trajectory by connecting the determined centers of gravity of the hand in each frame. The acquisition of the hand's trajectory will be explained both for the case where the image acquisition device 34 is a range camera and for the case where it is a two-dimensional camera. S12 for a distance camera:

[0028] Since the pixels of the area of ​​the hand detected by step S11 have distance information when using distance images, it becomes possible to obtain the three-dimensional position of the center of gravity of the hand. Therefore, in this case, the trajectory detection unit 311 obtains three-dimensional position information of the hand's movement trajectory by connecting the centers of gravity of the hand detected for each frame. S12 for a two-dimensional camera:

[0029] By connecting the center of gravity of the hand detected in the two-dimensional images of each frame, two-dimensional information of the hand's trajectory can be obtained. To obtain three-dimensional information of the hand's trajectory using a two-dimensional camera, gesture input is performed under conditions such as the following as an example. (r1): The operator holds the robot teaching device 30 at a known position in a robot coordinate system in a position where the optical axis of a lens 34b of the image acquiring device 34 substantially coincides with the X-axis of the robot coordinate system. (r2): The operator ensures that a reference plane S perpendicular to the optical axis 0 at a position at a certain distance (for example, about 15 cm) from the robot teaching device 30 (that is, a plane at which the position in the X-axis direction with respect to the image acquiring device 34 becomes the same) as a plane that forms the reference for the arrangement of the hand at the start of the gesture input.

[0030] Fig. 9 shows the positional relationship between the image pickup surface 34a of the image pickup elements of the image acquisition device 34, the lens 34b and the reference plane S 0under the conditions (r1) and (r2) described above. From the conditions (r1) and (r2) the respective positions P 0 , P 1 the image recording surface 34a and the reference plane S 0 in the X-axis direction of the robot coordinate system. The size (image magnification) of the image of the hand on the image pickup surface 34a when the hand is at the position of the reference plane S in this arrangement relationship 0 is stored. When the operator's hand moves in the X-axis direction with respect to the reference plane in conjunction with the gesture input and the size of the image of the hand on the image pickup surface 34a has changed, it is possible to compare it with the size of the image of the hand when the hand is on the reference plane S 0 is located, it is possible to change the position (P 2) of the hand in the X-axis direction based on the focal length of the lens 34b and the imaging method of the lens. This allows three-dimensional position information of the hand's trajectory to be obtained in the case of a two-dimensional camera.

[0031] Since the three-dimensional position of the hand's trajectory obtained as described above is position information in a coordinate system fixed to the image acquisition device 34 (hereinafter referred to as the camera coordinate system), the trajectory acquisition unit 311 converts the coordinate values ​​of the trajectory into coordinate values ​​in the robot coordinate system based on information representing the relative positional relationship between the robot 10 and the image acquisition device 34. The information representing the relative positional relationship between the robot 10 and the image acquisition device 34 can also be obtained by a method as described below.For example, the coordinate values ​​of the trajectory may be converted from the camera coordinate system to the robot coordinate system under the condition that the operator performs the gesture input while holding the robot teaching device 30 at a known position and attitude in the robot coordinate system (for example, at a known position in the robot coordinate system in a position where the optical axis of the lens of the image acquisition device 34 substantially coincides with the X-axis of the robot coordinate system). Alternatively, it is also possible to equip the robot teaching device 30 with a sensor for detecting the position and attitude (a gyro sensor, an acceleration sensor, or the like).In this case, for example, when starting the robot teaching device 30, calibration of the relative positional relationship between the robot coordinate system and the camera coordinate system is performed, and the position and attitude of the robot teaching device 30 in the robot coordinate system are always calculated by the above sensor. Other methods for determining the relative positional relationship between the robot coordinate system and the camera coordinate system may also be used.

[0032] Next, the robot teaching device 30 (the trajectory recognition unit 312) recognizes whether the hand trajectory is a rectilinear movement or a rotational movement using the hand trajectory obtained in step S12 (step S13). As an example, assume that the recognition is performed using two-dimensional position information of the hand trajectory (for example, position information when the trajectory is projected onto a certain plane). Hough transform is used to recognize whether the hand trajectory is a rectilinear movement. Specifically, whether the trajectory obtained in step S12 is rectilinear is determined by the following procedure. (E1): For each point (x 0 , y 0 ) on a plane (x, y) expressing the trajectory of the hand is calculated using the relationship r = x 0 cosθ + y 0sinθ a group of all straight lines passing through this point is drawn on a plane (r, θ). (E2): Points where several curves in the plane (r, θ) cross each other represent a straight line in the plane (x, y). Here, for example, it can be determined that the trajectory of the hand is a straight line if it has been determined that at least a certain number of pixels on the trajectory in the plane (x, y) are on the same linear shape.

[0033] A generalized Hough transform is used to detect whether the hand's trajectory is a rotational motion. Specifically, the generalized Hough transform can determine whether the hand's trajectory is circular, elliptical, or quadratic. As an example, the following procedure can be used to detect whether the trajectory is circular or not. (F1): For each point (x 0 , y 0 ) on a plane (x, y) expressing the trajectory of the hand is calculated using the relationship (x 0 - A) 2 + (y 0 - B) 2 = R 2 a group of all circles passing through this point is drawn in a space (A, B, R). (F2): Points in the space (A, B, R) shared by several curved surfaces represent a circle in the (x, y) plane. Here, it can also be determined that the hand's trajectory is a rotation if it has been determined that at least a certain number of pixels along the trajectory in the (x, y) plane are located on the same circle. Using procedures similar to those for a circle, it can also be determined whether pixels along the trajectory in the (x, y) plane fall on the same ellipse or not, or whether they fall on a secondary curve.

[0034] If a rectilinear movement is determined by the above-described procedure (E1), (E2) (S13: rectilinear movement), the robot teaching device 30 (the command generation unit 313) generates a command that causes the tip end 11 of the arm to perform a translational movement and sends it to the robot control device 20 (step S14). The robot control device 20, having received the translational movement command, causes the tip end 11 of the arm of the robot 10 to perform a translational movement. As shown in Fig. 4, the movement path A 2 the translational movement of the tip end 11 of the arm than the movement path A detected in step S12 1The direction of movement is set to be parallel to the hand's trajectory in the robot coordinate system. The direction of movement is set to correspond to the orientation of the palm detected by the procedure described later. For example, if the palm is positioned as shown in Fig. 4, the tip end 11 of the arm is moved upward. Regarding the extent of movement of the tip end 11 of the arm, it may be (1) only a predetermined distance, or (2) a distance equal to the length of the hand's trajectory. The movement speed may be a value predetermined for the robot 10.

[0035] If a rotational movement is determined by the above-described procedure (F1), (F2) (S13: rotational movement), the robot teaching device 30 (the command generation unit 313) generates a command that causes the tip end 11 of the arm to perform a rotational movement and sends it to the robot control device 20 (step S14). The robot control device 20, having received the rotational movement command, causes the tip end 11 of the arm of the robot 10 to perform a rotational movement. As shown in Fig. 5, the rotation axis R 1 during a rotational movement of the tip end 11 of the arm in a direction which is directed to the rotation axis R 0 parallel to the rotational movement of the hand in the robot coordinate system. The direction of rotation C 2 of the tip end 11 of the arm can be rotated in the direction C 1 the movement path of the hand. The detection unit 312 can determine the rotation axis R 0the trajectory of the hand in the robot coordinate system as an example based on the three-dimensional trajectory of the hand determined in step S12.

[0036] If neither of a linear motion and a rotary motion is detected (step 13: no detection possible), the present processing is terminated without operating the robot 10.

[0037] Next, the palm orientation detection performed in step S11 is explained. Fig. 7 is a flowchart illustrating palm recognition processing (hereinafter referred to as palm orientation recognition processing). When executing palm orientation recognition processing, whether the hand used for gestures is the right hand or the left hand and the posture of the hand during gestures are registered in advance in the robot teaching device 30. As an example, assume that the posture of the hand during gestures is the open posture. The trajectory detection unit 311 extracts the contour of the hand using the group of pixels of the hand area detected in step S11 (step S21). Then, the trajectory detection unit 311 sets points at equal intervals on the contour line of the hand (step S22).Subsequently, the trajectory detection unit 311 determines line segments connecting adjacent points among the points determined in step S22, and extracts points for which the angle of the two straight lines connecting this point and its two adjacent points forms an acute angle of at most a certain value (step S23). This extracts acute portions of the contour line, such as fingertips.

[0038] Then, the trajectory detection unit 311 selects five points at a certain distance from the center of gravity of the hand from the points extracted in step S23, which are regarded as fingertip positions (step S24). On the left side in Fig. 8 are the positions of the data obtained by processing steps S21 to S24 of Fig. 7 extracted fingertip positions P11 to P15 when the palm of the right hand is directed towards the image capture device 34. On the right side in Fig. 8 are the positions of the data obtained by processing steps S21 to S24 of Fig. 7, when the back of the right hand is directed toward the image capture device 34. Since the left or right hand has been determined in advance to be used for gesture input, the orientation of the palm shown in the images can be determined based on the fact that the positional relationship of the fingertips is as shown in Fig. 8, for the palm and the back of the hand (step S25). For example, the positions of the fingertips of the index finger, middle finger, ring finger, and little finger, which are relatively close to each other, and whether the thumb is on the left or right side with respect thereto, can determine whether it is the palm or the back of the hand.

[0039] Next, the start and end times for gesture recognition are explained. To prevent a hand movement not intended by the operator from being recognized as a gesture command, a procedure can be adopted in which the detection of the movement trajectory is performed at certain time intervals and no detection of whether it is a linear movement or a rotational movement is performed if the movement extent is within a certain range. For example, the gesture teaching processing of Fig.6 are started repeatedly at specific time intervals, and in step S13, it is determined that no recognition is possible if the movement amount of the hand trajectory detected as a rectilinear movement or a rotational movement by the Hough transform is at most a specific amount. This can prevent a hand movement not intended by the operator from being recognized as a gesture command. Even if, for example, the fingertips cannot be recognized when detecting the orientation of the palm in step S11, it can be determined in step S13 that no recognition is possible.

[0040] As described above, the present embodiment makes it possible to perform teaching of a robot by an intuitive operation with an even higher degree of freedom.

[0041] In the foregoing, an embodiment of the present disclosure has been explained, but those skilled in the art will understand that various improvements and changes can be made without departing from the scope of the disclosure of the following claims.

[0042] In the embodiment described above, the individual functional blocks of the trajectory detection unit 311, the recognition unit 312, and the command generation unit 313 are arranged in the robot teaching device 30, but at least part of these functions may be arranged on the robot control device 20 side.

[0043] The robot teaching device 30 may also include both a range-view camera and a two-dimensional camera as the image acquisition device 34. In this case, the range-view camera and the two-dimensional camera are arranged so that their positional relationship (i.e., the relationship of the pixel arrangements of their images) is known. With such an arrangement, the various recognition methods for using a range-view camera and the various recognition methods for using a two-dimensional camera, which were explained in the above-described embodiment, can be used together.

[0044] The programs for executing the gesture teaching processing and the palm orientation recognition processing shown in the above-described embodiment can be recorded on various computer-readable recording media (for example, a semiconductor memory such as a ROM, an EEPROM, a flash memory, or the like, a magnetic recording medium, or an optical disk such as a CD-ROM, a DVD-ROM, or the like).

Claims

[1] Robot teaching device (30) which teaches a robot, wherein the robot teaching device (30) an image acquisition unit (34) that acquires distance images representing distance information of a subject to be photographed or two-dimensional images of the subject to be photographed as moving images; a trajectory detection unit (311) that detects the movement trajectory of an operator's hand captured as a recording object in the moving images using the moving images; a detection unit (312) which detects whether the detected trajectory represents a straight-line trajectory or a rotational trajectory; and a command generation unit (313) that generates a command that makes a specific movable portion of the robot perform a translational movement based on the movement trajectory when it is detected that the movement trajectory is a rectilinear movement, and makes the specific movable portion of the robot perform a rotational movement based on the movement trajectory when it is detected that the movement trajectory is a rotational movement. [2] The robot teaching device (30) according to claim 1, wherein the trajectory detection unit (311) detects the movement trajectory based on information representing the relative positional relationship of the robot and the image acquisition device (34) as coordinate values ​​in a robot coordinate system, and the command generation unit (313) generates the command such that the specific movable section is caused to execute a movement parallel to the movement path when the recognition unit has recognized that the movement path represents a rectilinear movement. [3] Robot teaching device (30) according to claim 1 or 2, wherein the trajectory detection unit (311) further detects the orientation of the palm based on the moving images, and the command generating unit (313) determines the direction of movement when the specific movable portion is caused to perform a translational movement according to the detected orientation of the palm. [4] Robot teaching device (30) according to one of claims 1 to 3, wherein the recognition unit (312) upon recognition, that the movement path represents a rotational movement, furthermore the rotation axis of the rotational movement of the hand is recorded as coordinate values ​​in the robot coordinate system, and the command generating unit (313) sets the rotation axis when the specific movable portion is caused to perform a rotational movement so that it is parallel to the detected rotation axis of the rotational movement of the hand. [5] Robot system (100) that a robot (10); a robot control device (20) that controls the robot (10); and a robot teaching device (30) according to one of claims 1 to 4 includes, wherein the robot control device (20) controls the robot (10) based on the command output from the command generation unit (313) of the robot control device (30).

Images