Information processing device, robot trajectory generation method, robot control method, program, mobile body, and robot system

Abstract

To suppress increase in burden on an operator when teaching robot operation.SOLUTION: A robot controller 10 includes: an information acquisition part 121 for acquiring position information PINF indicating a position of a terminal device 30, and attitude information PINF indicating the attitude of the terminal device 30; and an information generation part 122 for generating operation information for regulating the position and the attitude of a tip part HP of a robot 20 on the basis of the position information PINF and the attitude information PINF acquired by the information acquisition part 121.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, A method for generating a robot trajectory, The present invention relates to a robot control method, a program, a moving body, and a robot system. [Background technology]

[0002] Direct teaching is known as a method for teaching operations to robots such as industrial robots (see, for example, Patent Document 1). Direct teaching is a teaching method in which, for example, an operator manually moves the robot directly to make the robot memorize an operating point. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-131314 Summary of the Invention [Problem to be solved by the invention]

[0004] However, direct teaching has the problem that depending on the weight and size of the robot, the physical and time burden on the worker increases. [Means for solving the problem]

[0005] An information processing device according to a preferred aspect of the present invention includes an acquisition unit that acquires position information at each position, or position information at each position and posture information at each position, of a moving body that moves sequentially to and is placed at multiple positions in space independently of a robot, and a generation unit that generates trajectory information for the operation of the robot by sequentially specifying the positions and postures of specific parts of the robot based on the multiple pieces of position information, or the position information and posture information, acquired by the acquisition unit.

[0006] A preferred embodiment of the robot control method of the present invention involves acquiring position information and posture information for each of a plurality of positions of a moving body that has been moved independently of the robot and placed at multiple positions in space, and generating trajectory information for the operation of the robot by sequentially specifying the positions and postures of specific parts of the robot based on the acquired plurality of pieces of position information and posture information.

[0007] A preferred aspect of the present invention is a program in which a processor that operates a robot acquires position information and posture information at each position of a moving object that has been moved and placed at multiple positions in space in sequence, and generates trajectory information for the operation of the robot by sequentially specifying the positions and postures of specific parts of the robot based on the acquired multiple pieces of position information and posture information.

[0008] A robot system according to a preferred aspect of the present invention comprises a robot, a moving body, and an information processing device, wherein the information processing device comprises an acquisition unit that acquires position information indicating the position of the moving body, or position information indicating the position of the moving body and posture information indicating the posture of the moving body, and a generation unit that generates operation information that specifies the position or the position and posture of a specific part of the robot based on the position information or the position information and posture information acquired by the acquisition unit. A preferred embodiment of the present invention provides a mobile body that moves independently of a robot through a plurality of positions for generating trajectory information for the robot's operation. The mobile body has an inertial sensor that detects position information or position information and posture information of the mobile body at the plurality of positions, and a communication unit that communicates with at least an information processing device that operates the robot. Based on the posture information and position information detected by the mobile body at a plurality of positions in space, trajectory information for the robot's operation is generated. [Effects of the Invention]

[0009] According to the present invention, when teaching a robot an operation, an increase in the burden on the worker can be suppressed. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram for explaining an overview of a robot system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a robot controller illustrated in FIG. [Figure 3] FIG. 2 is a functional block diagram showing an example of the configuration of a robot controller shown in FIG. [Figure 4] 2 is a diagram illustrating an example of a hardware configuration of the terminal device illustrated in FIG. 1. [Figure 5] 2 is a functional block diagram showing an example of the configuration of a terminal device shown in FIG. 1. FIG. [Figure 6] 2 is a sequence chart showing an example of the operation of the robot system shown in FIG. 1. [Figure 7] FIG. 3 is an explanatory diagram showing an example of an operation table shown in FIG. 2. [Figure 8] FIG. 10 is an explanatory diagram illustrating an example of an operation screen. [Figure 9] 2 is a flowchart showing an example of the operation of the robot controller shown in FIG. 1. [Figure 10] 10 is a sequence chart showing an example of an operation of a robot system according to a first modified example. [Figure 11] 10 is a flowchart showing an example of the operation of a robot controller according to a first modified example. [Figure 12] FIG. 10 is an explanatory diagram for explaining an overview of a robot system according to a third modified example. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that in each drawing, the dimensions and scale of each part are appropriately different from those of the actual parts. Furthermore, the embodiments described below are preferred examples of the present invention, and therefore various technically preferable limitations are applied. However, the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited.

[0012] [1. Embodiment] First, an example of an overview of a robot system 1 according to an embodiment will be described with reference to FIG.

[0013] FIG. 1 is an explanatory diagram for explaining an overview of a robot system 1 according to an embodiment.

[0014] For ease of explanation, the following description introduces a world coordinate system ΣW fixed to the real space and an imaging coordinate system ΣC fixed to the imaging device (detection device) 40. The world coordinate system ΣW is, for example, a three-axis Cartesian coordinate system having an origin at a predetermined position in the real space and an Xw-axis, a Yw-axis, and a Zw-axis that are orthogonal to each other. In this embodiment, it is assumed that the predetermined position that is the origin of the world coordinate system ΣW is position Op. Position Op is, for example, the center of a base BSP of the robot 20 (described later). In this embodiment, it is assumed that the Xw-Zw plane is parallel to the floor to which the base BSP is fixed. In addition, the imaging coordinate system ΣC is, for example, a three-axis Cartesian coordinate system having an origin at a predetermined position of the imaging device 40 and an Xc-axis, a Yc-axis, and a Zc-axis that are orthogonal to each other. In this embodiment, it is assumed that the Zc-axis is parallel to the optical axis of the optical system of the imaging device 40. Hereinafter, the optical axis of the optical system of the imaging device 40 will also be simply referred to as the optical axis of the imaging device 40.

[0015] 1 is, for example, a robot control robot system that controls the operation of a robot 20. For example, the robot system 1 includes a robot controller 10, a robot 20, a terminal device 30, and an imaging device 40. The robot controller 10 is an example of an "information processing device," and the terminal device 30 is an example of a "mobile body."

[0016] 1 are connected to each other via, for example, a wired connection so that they can communicate with each other. Note that the connection between the robot controller 10 and the robot 20 may be wireless, or may be both wired and wireless.

[0017] The robot controller 10, the terminal device 30, and the imaging device 40 each have a communication unit and are connected to each other so that they can communicate with each other. In this embodiment, it is assumed that communication between the robot controller 10, the terminal device 30, and the imaging device 40 is short-range wireless communication such as Bluetooth (registered trademark) and Wi-Fi (registered trademark). Note that the connections between the multiple elements included in the robot system 1 may be any connections that allow the multiple elements to communicate with each other, and may be connections using a network that includes one or both of a wired network and a wireless network.

[0018] Any information processing device capable of communicating with other devices can be used as the robot controller 10. The robot controller 10 controls, for example, the operation of a robot 20. The configuration of the robot controller 10 will be explained later with reference to FIGS. 2 and 3.

[0019] The robot 20 is, for example, an articulated robot installed in a factory or the like. For example, the robot 20 has a base part BSP, a body part BDP, multiple arm parts AP (AP1, AP2, AP3, and AP4), and a tip part HP. The tip part HP is an example of a "specific part."

[0020] The base part BSP is fixed to a predetermined location such as the floor. The body part BDP is connected to the base part BSP so as to be rotatable about a rotation axis AX1. The arm part AP1 is connected to the body part BDP so as to be rotatable about a rotation axis AX2. The arm part AP2 is connected to the arm part AP1 so as to be rotatable about a rotation axis AX3. The arm part AP3 is connected to the arm part AP2 so as to be rotatable about a rotation axis AX4. The arm part AP4 is connected to the arm part AP3 so as to be rotatable about a rotation axis AX5. However, the rotation angle of each of the arm parts AP1, AP2, AP3, and AP4 is limited to less than 360 degrees.

[0021] The tip portion HP also has a first tip portion HP1 and a second tip portion HP2 connected to the first tip portion HP1. The first tip portion HP1 is connected to the arm portion AP4 so as to be rotatable about a rotation axis Hy. The first tip portion HP1 is also rotatable about a rotation axis Hx. The second tip portion HP2 is connected to the first tip portion HP1 so as to be rotatable about a rotation axis Hz. However, the rotation angle of the first tip portion HP1 when rotating about the rotation axis Hx is limited to less than 360 degrees. Similarly, the rotation angle of the second tip portion HP2 when rotating about the rotation axis Hz is limited to less than 360 degrees.

[0022] Here, since the second tip portion HP2 is connected to the first tip portion HP1, when the first tip portion HP1 rotates about the rotation axis Hy or the rotation axis Hx, the second tip portion HP2 rotates together with the first tip portion HP1. That is, the second tip portion HP2 can rotate about each of the rotation axes Hx, Hy, and Hz.

[0023] In this embodiment, it is assumed that the robot 20 is fixed to a predetermined location such as a floor, but the robot 20 may be movable without being fixed to a predetermined location. When the robot 20 itself is movable, the origin (for example, the center of gravity or center) of the robot 20 body may be regarded as the "specific part."

[0024] The terminal device 30 may be any portable information processing device capable of transmitting signals. For example, the terminal device 30 may be a portable information terminal such as a remote controller or a smartphone. For example, when specifying the position and posture of the tip end HP of the robot 20 that performs a predetermined task, the terminal device 30 is held by the worker U and moved by the worker U independently of the robot so as to follow a movement path (trajectory information) corresponding to the predetermined task. The worker U is an example of a "user." Note that the position of the tip end HP of the robot 20 corresponds to, for example, the operating point of the robot 20. The movement path corresponding to the predetermined task is, for example, a path specified as the movement path of the tip end HP of the robot 20 that performs the predetermined task. In other words, the worker U moves the terminal device 30 along the movement path corresponding to the predetermined task, thereby teaching the robot 20 the trajectory of the tip end HP that corresponds to the predetermined task.

[0025] For example, the task of the worker U moving the terminal device 30 reduces the physical and time burden on the worker U compared to the task of the worker U manually and directly moving the robot 20. Therefore, in this embodiment, it is possible to suppress an increase in the physical and time burden on the worker U when teaching a trajectory to the robot 20 compared to the mode in which the worker U manually and directly moves the robot 20.

[0026] In this embodiment, to make it easier to understand the description of the attitude and the like of the terminal device 30, it is assumed that the shape of the terminal device 30 is a rectangular parallelepiped. Furthermore, for convenience, the attitude and the like of the terminal device 30 will be described below using the Xm axis, Ym axis, and Zm axis, which are orthogonal to each other. For example, the Xm axis is an axis that passes through position P and is perpendicular to face SF3, the Ym axis is an axis that passes through position P and is perpendicular to face SF1, and the Zm axis is an axis that passes through position P of the terminal device 30 and is perpendicular to face SF2. Position P is, for example, the center of face SF1 of the terminal device 30. Note that the shape of the terminal device 30 is not limited to a rectangular parallelepiped.

[0027] The position of the terminal device 30 is represented, for example, by the coordinates of position P of the terminal device 30. The attitude of the terminal device 30 is represented, for example, by the rotation angle of the Xm axis when rotated about the Xm axis, the rotation angle of the Ym axis when rotated about the Ym axis, and the rotation angle of the Zm axis when rotated about the Zm axis. Note that in this embodiment, it is assumed that the rotation angles of the Xm axis, the Ym axis, and the Zm axis are represented by taking as a reference (0 degrees) the attitude of the terminal device 30 when the Xm axis is parallel to the Xw axis, the Ym axis is parallel to the Yw axis, and the Zm axis is parallel to the Zw axis.

[0028] The position of the tip part HP of the robot 20 is expressed, for example, by the coordinates of the center of the surface SFh of the second tip part HP2. Hereinafter, the surface SFh of the second tip part HP2 will also be referred to as the surface SFh of the tip part HP. The posture of the tip part HP is expressed using the rotation angle of the rotation axis Hx when rotated about the rotation axis Hx, the rotation angle of the rotation axis Hy when rotated about the rotation axis Hy, and the rotation angle of the rotation axis Hz when rotated about the rotation axis Hz. Note that in this embodiment, it is assumed that the rotation angle of the rotation axis Hx, the rotation angle of the rotation axis Hy, and the rotation angle of the rotation axis Hz are expressed using the posture of the tip part HP when the rotation axis Hx is parallel to the Xw axis, the rotation axis Hy is parallel to the Yw axis, and the rotation axis Hz is parallel to the Zw axis as the reference (0 degrees).

[0029] In this embodiment, it is assumed that the position of the terminal device 30 is calculated by the imaging device 40, and the attitude of the terminal device 30 is calculated by the terminal device 30. The configuration of the terminal device 30 will be described later with reference to FIG. 4. For example, the terminal device 30 includes an inertial sensor 38 for detecting the attitude of the terminal device 30. In addition, in the terminal device 30 shown in FIG. 1, two different marks MK (MK1 and MK2) are provided on the surface SF2 so that the imaging device 40 can easily recognize the terminal device 30. For example, the imaging device 40 can easily recognize the portion of the entire image representing the terminal device 30 by recognizing the mark MK in an image of the terminal device 30. Note that the number of marks MK may be one. In the case where the mark MK is a single mark whose position is known, the position P of the center of the terminal device 30 can be calculated from the position of the mark MK on the screen and the attitude of the terminal device 30 detected by the inertial sensor 38 described later. In addition, in the case where the mark MK is two or more marks whose positions are known, the position P of the center of the terminal device 30 can be calculated from the two marks MK. Furthermore, if the number of marks MK is small, it is possible that the marks MK cannot be captured by the imaging device 40 for some reason, and therefore it is preferable to provide the marks MK on multiple surfaces of the terminal device 30. Furthermore, light-emitting elements such as LEDs (Light Emitting Diodes) may be used as the marks MK.

[0030] The imaging device 40 is hardware for capturing an image of a subject using optical means such as a camera. In this embodiment, it is assumed that the imaging device 40 is a three-dimensional camera. A three-dimensional camera is a camera that can calculate depth (distance to an object). For example, the imaging device 40 captures an image of the terminal device 30 to calculate the position of the terminal device 30 in the imaging coordinate system ΣC. Then, the imaging device 40 transforms coordinates indicating the position of the terminal device 30 from the imaging coordinate system ΣC to a world coordinate system ΣW, and transmits position information indicating the position of the terminal device 30 in the world coordinate system ΣW to the robot controller 10. Note that this embodiment assumes a case where information such as parameters for transforming coordinates in the imaging coordinate system ΣC into coordinates in the world coordinate system ΣW is known. Furthermore, the coordinate transformation for transforming the position of the terminal device 30 in the imaging coordinate system ΣC to the position of the terminal device 30 in the world coordinate system ΣW may be performed by the robot controller 10. Furthermore, the timing at which the imaging device 40 captures an image is determined by a signal from the terminal device 30 or the robot controller 10.

[0031] As described above, in this embodiment, the imaging device 40 is used as a distance measuring device that measures the distance between the terminal device 30 and the imaging device 40. As a distance measurement method, known methods such as a time-of-flight (TOF) method, a frequency-modulated continuous wave (FMCW) method, and a stereo camera method can be adopted.

[0032] The TOF method is a method for measuring the distance to an object based on the time it takes for a distance measuring device (e.g., the image capturing device 40) to project light such as infrared light onto the object and for the distance measuring device to receive the light reflected from the object. The FMCW method is a method for measuring the distance to an object based on the frequency difference between a transmission signal transmitted from the distance measuring device and a reflection signal reflected from the object (a reception signal received by the distance measuring device). For example, a TOF LiDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging) or an FMCW LiDAR may be adopted as a sensor for measuring distance. The stereo camera method is a method for measuring the distance to an object based on the parallax when two cameras capture images of the same object. Note that the distance measurement method is not limited to the above-mentioned examples.

[0033] In this embodiment, for example, the robot controller 10 acquires position information indicating the position of the terminal device 30 from the imaging device 40, and acquires posture information indicating the posture of the terminal device 30 from the terminal device 30. Then, the robot controller 10 determines the position and posture of the tip part HP of the robot 20 based on the position information acquired from the imaging device 40 and the posture information acquired from the terminal device 30. The method for determining the position and posture of the tip part HP of the robot 20 will be described later with reference to FIG. 6.

[0034] The configuration of the robot system 1 is not limited to the example shown in Fig. 1. For example, the robot controller 10 may be included in the robot 20. Also, for example, the imaging device 40 may be provided in the robot 20. Also, for example, the robot 20 may be communicably connected to the terminal device 30 and the imaging device 40. In this case, the connection between each of the terminal device 30 and the imaging device 40 and the robot 20 may be either a wired connection or a wireless connection, or may be both a wired and wireless connection.

[0035] Furthermore, the imaging device 40 may be a monocular camera if the error in the distance between the terminal device 30 and the imaging device 40, calculated assuming that the Xm axis of the terminal device 30 is perpendicular to the optical axis of the imaging device 40, is within an acceptable range. In this case, the imaging device 40 calculates the distance between the terminal device 30 and the imaging device 40, for example, assuming that the Xm axis of the terminal device 30 is perpendicular to the optical axis of the imaging device 40, based on the known distance between the marks MK1 and MK2, the positions of the marks MK1 and MK2 in the image, and the focal length. Note that the focal length is the focal length of the optical system of the imaging device 40 and is known information. Furthermore, for example, if the robot controller 10 can identify the position of the terminal device 30 based on satellite signals transmitted from GPS (Global Positioning System) satellites and the accuracy is sufficient for the purpose, a GPS may be used as the detection device instead of the imaging device (detection device) 40. Furthermore, the detection device is not limited to a GPS, and a device using laser light or sound waves may also be used as the detection device. Alternatively, the laser light or sound waves may be emitted from the terminal device 30 and detected by the detection device 40.

[0036] Next, the hardware configuration of the robot controller 10 will be described with reference to FIG.

[0037] FIG. 2 is a diagram illustrating an example of a hardware configuration of the robot controller 10 shown in FIG.

[0038] The robot controller 10 has a processing device (control unit) 12 that controls each part of the robot controller 10, a memory (storage unit) 13 that stores various information, a communication device (communication unit) 14, an operation device (operation unit) 15 that accepts operations by a worker U or the like, a display device (display unit) 16, and a driver circuit 17.

[0039] The memory 13 includes, for example, one or both of a volatile memory such as a random access memory (RAM) that functions as a work area for the processing device 12 and a non-volatile memory such as an electrically erasable programmable read-only memory (EEPROM) that stores various information such as the control program PGr. The memory 13 may be detachable from the robot controller 10. Specifically, the memory 13 may be a storage medium such as a memory card that is detachable from the robot controller 10. The memory 13 may also be, for example, a storage device (e.g., online storage) that is communicatively connected to the robot controller 10 via a network or the like.

[0040] The memory 13 shown in Fig. 2 stores a control program PGr and an operation table MTBL. The control program PGr is an example of a "program." In this embodiment, the control program PGr includes, for example, an application program that causes the robot controller 10 to control the operation of the robot 20. However, the control program PGr may also include, for example, an operating robot system program that causes the processing device 12 to control each part of the robot controller 10.

[0041] Details of the operation table MTBL will be explained later in Fig. 7, but the operation table MTBL stores, for example, position information PINF and posture information AINF. For example, the position information PINF is position information indicating the position of the terminal device 30, and is used as position information that defines the position of the tip end HP of the robot 20. Furthermore, the posture information AINF is posture information that indicates the posture of the terminal device 30, and is used as posture information that defines the posture of the tip end HP of the robot 20.

[0042] The processing device 12 is a processor that controls the entire robot controller 10, and is configured to include, for example, one or more CPUs (Central Processing Units). The processing device 12 executes, for example, a control program PGr stored in the memory 13, and operates in accordance with the control program PGr, thereby functioning as a robot control unit 120 shown in Fig. 3 (to be described later). The control program PGr may be transmitted from another device via a network or the like.

[0043] Furthermore, for example, when the processing device 12 is configured to include multiple CPUs, some or all of the functions of the processing device 12 may be realized by the multiple CPUs operating cooperatively in accordance with a program such as the control program PGr. Furthermore, the processing device 12 may be configured to include hardware such as a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array) in addition to one or more CPUs, or instead of some or all of the one or more CPUs. In this case, some or all of the functions of the processing device 12 may be realized by hardware such as a DSP.

[0044] The communication device 14 is hardware for communicating with an external device that exists outside the robot controller 10. For example, the communication device 14 has a function of communicating with the external device via short-range wireless communication. Note that the communication device 14 may further have a function of communicating with the external device via a mobile communication network or a network.

[0045] The operation device 15 is an input device (for example, a keyboard, a mouse, a switch, a button, a sensor, etc.) that accepts input from the outside. For example, the operation device 15 accepts an operation by the worker U and outputs operation information corresponding to the operation to the processing device 12. Note that, for example, a touch panel that detects contact with the display surface of the display device 16 may be adopted as the operation device 15.

[0046] The display device 16 is an output device such as a display that outputs to the outside. The display device 16 displays an image, for example, under the control of the processing device 12. The operation device 15 and the display device 16 may be integrated into one unit (for example, a touch panel).

[0047] The driver circuit 17 is hardware that outputs signals to the robot 20 to drive the robot 20 under the control of the processing device 12. For example, the driver circuit 17 outputs signals to the robot 20 to drive the body part BDP, the arm part AP, the tip part HP, etc. of the robot 20 under the control of the processing device 12.

[0048] Next, the function of the robot controller 10 will be described with reference to FIG.

[0049] FIG. 3 is a functional block diagram showing an example of the configuration of the robot controller 10 shown in FIG.

[0050] The robot control unit 120 is realized by the processing device 12, as described in Fig. 2. If the processing device 12 is configured to include multiple CPUs, some or all of the functions of the robot control unit 120 may be realized by these multiple CPUs operating in cooperation with each other in accordance with the control program PGr. If the processing device 12 is configured to include hardware such as a DSP, some or all of the functions of the robot control unit 120 may be realized by the hardware such as a DSP.

[0051] The robot control unit 120 includes, for example, an information acquisition unit 121, an information generation unit 122, an operation control unit 123, a display control unit 124, and a warning unit 125. The information acquisition unit 121 is an example of an "acquisition unit," and the information generation unit 122 is an example of a "generation unit." Furthermore, the operation control unit 123 is an example of a "control unit."

[0052] The information acquisition unit 121, for example, acquires position information PINF indicating the position of the terminal device 30 from the imaging device (detection device) 40, and acquires posture information AINF indicating the posture of the terminal device 30 from the terminal device 30. For example, when the worker U performs a transmission operation on the terminal device 30 to transmit information such as the position information PINF and the posture information AINF to the robot controller 10, the posture information AINF is transmitted from the terminal device 30, and the position information PINF is transmitted from the imaging device 40. Therefore, when the worker U performs a transmission operation on the terminal device 30, the information acquisition unit 121 acquires the position information PINF and the posture information AINF. For example, if the operation device (operation unit) 35 includes a transmission button, the transmission operation may be, for example, pressing the transmission button. The transmission operation is an example of a "predetermined operation."

[0053] The information generation unit 122 generates operation information that defines the position and posture of the tip end HP of the robot 20 based on the position information PINF and posture information AINF acquired by the information acquisition unit 121. For example, the information generation unit 122 stores the position information PINF and posture information AINF acquired by the information acquisition unit 121 in an operation table MTBL. As a result, the position indicated by the position information PINF and the posture indicated by the posture information AINF are registered in the operation table MTBL. Hereinafter, the position indicated by the position information PINF stored in the operation table MTBL will also be referred to as the position registered in the operation table MTBL. Similarly, the posture indicated by the posture information AINF stored in the operation table MTBL will also be referred to as the posture registered in the operation table MTBL.

[0054] For example, the information generation unit 122 determines the position and posture registered in the motion table MTBL as the position and posture of the tip part HP of the robot 20, thereby defining the position and posture registered in the motion table MTBL as the position and posture of the tip part HP of the robot 20. In this case, the position information PINF and posture information AINF stored in the motion table MTBL (the position information PINF and posture information AINF indicating the determined position and posture, respectively) correspond to the motion information defining the position and posture of the tip part HP of the robot 20. In other words, the information generation unit 122 generates motion information defining the position and posture of the tip part HP of the robot 20 by determining the position and posture registered in the motion table MTBL.

[0055] The operation control unit 123 operates the robot 20 based on the position information PINF and posture information AINF acquired by the information acquisition unit 121. For example, the operation control unit 123 operates the robot 20 by controlling the driver circuit 17 based on the position and posture registered in the operation table MTBL.

[0056] The operation control unit 123 may operate the robot 20 based on correction information for correcting the position and posture registered in the operation table MTBL. For example, the operation control unit 123 may accept input of correction information indicating the position and posture of the tip part HP instructed by the operator U via a GUI (Graphical User Interface) displayed on an operation screen CHS shown in Fig. 8 (to be described later) or the like. Then, the operation control unit 123 may correct the position and posture registered in the operation table MTBL based on the correction information.

[0057] Furthermore, the operation control unit 123 may correct the position and posture registered in the operation table MTBL based on correction information obtained by the operator U manually and directly moving the robot 20. Specifically, for example, the operator U visually checks the position and posture of the tip part HP of the robot 20 that has operated based on the position and posture registered in the operation table MTBL. Then, the operator U manually and directly moves the robot 20 to move the position and posture of the tip part HP to a desired position and posture. In this case, the operation control unit 123 obtains correction information indicating the position and posture of the tip part HP moved by the operator U, and corrects the position and posture registered in the operation table MTBL based on the correction information.

[0058] The position and orientation corrected based on the correction information are registered in, for example, the motion table MTBL, and are determined by the information generating unit 122 as the position and orientation of the tip part HP defined by the motion information.

[0059] The display control unit 124 causes the display device 16 to display various images such as the operation screen CHS shown in FIG.

[0060] For example, each time the information acquisition unit 121 acquires position information PINF, the warning unit 125 determines whether the position indicated by the position information PINF is within the movable range of the tip HP of the robot 20, and issues a warning if the position indicated by the position information PINF is not within the movable range.

[0061] For example, the warning unit 125 may warn the worker U that the position of the terminal device 30 is outside the movable range of the robot 20 by outputting a warning sound or the like. Alternatively, the warning unit 125 may transmit warning information indicating that the position of the terminal device 30 is outside the movable range of the robot 20 to the terminal device 30. In this case, the terminal device 30 may warn the worker U that the position of the terminal device 30 is outside the movable range of the robot 20 by outputting a warning sound or the like.

[0062] The warning, such as a warning sound, enables the worker U to know whether or not the terminal device 30 has been moved outside the movable range of the tip end HP of the robot 20 while moving the terminal device 30 along a movement path corresponding to a predetermined task. As a result, the worker U can efficiently move the terminal device 30 within the movable range of the tip end HP of the robot 20. Therefore, in this embodiment, the efficiency of the task of moving the terminal device 30 along a movement path corresponding to a predetermined task can be improved.

[0063] The configuration of the robot controller 10 is not limited to the examples shown in Figures 2 and 3. For example, the warning unit 125 may be omitted.

[0064] Next, the hardware configuration of the terminal device 30 will be described with reference to FIG.

[0065] FIG. 4 is a diagram illustrating an example of a hardware configuration of the terminal device 30 illustrated in FIG.

[0066] The terminal device 30 has a processing device 32 that controls each part of the terminal device 30, a memory 33 that stores various information, a communication device (communication unit) 34, an operation device (operation unit) 35, a display device (display unit) 36, a speaker 37, an inertial sensor 38, and an imaging device 39.

[0067] The memory 33 includes, for example, one or both of a volatile memory such as a RAM that functions as a work area for the processing device 32 and a non-volatile memory such as an EEPROM that stores various information such as the control program PGt. Note that, like the memory 13 of the robot controller 10 described in Fig. 2, the memory 33 may be detachable from the terminal device 30, or may be a storage device (for example, online storage) communicably connected to the terminal device 30 via a network or the like.

[0068] The processing device 32 is a processor that controls the entire terminal device 30, and is configured similarly to the processing device 12 of the robot controller 10 described in FIG. 2. For example, the processing device 32 includes one or more CPUs. The processing device 32 executes a control program PGt stored in the memory 33 and operates in accordance with the control program PGt, thereby functioning as a terminal control unit 320 shown in FIG. 5, which will be described later. The control program PGt may be transmitted from another device via a network or the like.

[0069] Furthermore, for example, if the processing device 32 is configured to include multiple CPUs, some or all of the functions of the processing device 32 may be realized by these multiple CPUs operating in cooperation with each other in accordance with a program such as the control program PGt. Furthermore, the processing device 32 may be configured to include hardware such as a GPU, DSP, or FPGA in addition to one or more CPUs, or instead of some or all of the one or more CPUs. In this case, some or all of the functions of the processing device 32 may be realized by hardware such as a DSP.

[0070] The communication device 34 is hardware for communicating with an external device that exists outside the terminal device 30. For example, the communication device 34 has a function of communicating with the external device via short-range wireless communication. Note that the communication device 34 may further have a function of communicating with the external device via a mobile communication network or a network.

[0071] The operation device 35 is an input device (for example, a keyboard, a mouse, a switch, a button, a sensor, etc.) that accepts input from the outside. For example, the operation device 35 accepts an operation by the worker U and outputs operation information corresponding to the operation to the processing device 32. Note that, for example, a touch panel that detects contact with the display surface of the display device 36 may be adopted as the operation device 35.

[0072] The display device 36 is an output device such as a display that outputs to the outside. The display device 36 displays an image, for example, under the control of the processing device 32. The operation device 35 and the display device 36 may be integrated into one unit (for example, a touch panel).

[0073] The speaker 37 is hardware that outputs various sounds under the control of the processing unit 32 .

[0074] The inertial sensor 38 detects, for example, the acceleration of the terminal device 30 on each of the Xw-axis, Yw-axis, and Zw-axis that represent a three-dimensional space, and the angular velocity of the terminal device 30 when rotated about each of the Xw-axis, Yw-axis, and Zw-axis. By detecting the acceleration on the Xw-axis, Yw-axis, and Zw-axis, it is possible to measure the travel distance of the terminal device 30. Furthermore, by detecting the angular velocity on the Xw-axis, Yw-axis, and Zw-axis, it is possible to detect the tilt (attitude) of the terminal device 30 with respect to the direction of gravity. In this embodiment, in order to detect the tilt (attitude) of the terminal device 30, it is essential to detect the angular velocity of the terminal device 30 when rotated about each of the Xw-axis, Yw-axis, and Zw-axis.

[0075] The imaging device 39 is hardware for capturing an image of a subject by optical means such as a camera. For example, the imaging device 39 captures an image of the subject and generates image information indicating the captured image of the subject. In this embodiment, the position of the terminal device 30 is measured by the imaging device 40, so the imaging device 39 does not need to have a function for measuring the distance to the object. However, the imaging device 39 may have a function for measuring the distance to the object. In other words, the imaging device 39 may be a 3D camera or a camera other than a 3D camera.

[0076] Next, the functions of the terminal device 30 will be described with reference to FIG.

[0077] FIG. 5 is a functional block diagram showing an example of the configuration of the terminal device 30 shown in FIG.

[0078] The terminal control unit 320 is realized by the processing device (control unit) 32, as described in Fig. 4. If the processing device 32 is configured to include multiple CPUs, some or all of the functions of the terminal control unit 320 may be realized by these multiple CPUs operating in cooperation with each other in accordance with the control program PGt. If the processing device 32 is configured to include hardware such as a DSP, some or all of the functions of the terminal control unit 320 may be realized by the hardware such as a DSP.

[0079] The terminal control unit 320 includes, for example, an operation notification unit 322, an attitude detection unit 324, a warning control unit 326, and a display control unit 328.

[0080] When the operation device 35 receives an operation related to the operation of one or both of the robot controller 10 and the image capture device 40, the operation notification unit 322 notifies the operation device 35 of the content of the operation received. For example, when the operation device 35 receives a transmission operation to transmit information such as position information PINF and posture information AINF to the robot controller 10, the operation notification unit 322 transmits instruction information to the image capture device 40 to instruct the image capture device 40 to transmit the position information PINF. Note that the instruction information may be transmitted from the terminal device 30 to the image capture device 40 via the robot controller 10.

[0081] The attitude detection unit 324 detects the attitude of the terminal device 30 based on the detection results of the acceleration and angular velocity of the inertial sensor 38, and generates attitude information AINF indicating the detected attitude. Then, the attitude detection unit 324 transmits the attitude information AINF indicating the attitude of the terminal device 30 to the robot controller 10 via the communication device (communication unit) 34. For example, when the operation device 35 accepts a transmission operation, the attitude detection unit 324 detects the attitude of the terminal device 30, and transmits the attitude information AINF indicating the detected attitude to the robot controller 10.

[0082] For example, when the communication device 34 receives warning information from the robot controller 10, the warning control unit 326 outputs a warning sound from the speaker 37. This allows the worker U holding the terminal device 30 to know that the position of the terminal device 30 is outside the movable range of the robot 20.

[0083] The display control unit 328, for example, causes various images to be displayed on the display device (display unit) 36. For example, the display control unit 328 may cause the display device 36 to display images of GUIs corresponding to various operations.

[0084] The configuration of the terminal device 30 is not limited to the examples shown in FIGS. 4 and 5 . For example, some or all of the display device 36, the speaker 37, and the imaging device 39 may be omitted. Furthermore, for example, the terminal device 30 may include a vibration generator such as a vibrator. In this case, the warning control unit 326 may warn the worker U that the position of the terminal device 30 is outside the movable range of the robot 20 by vibrating the vibration generator. Furthermore, for example, the terminal device 30 may include a light-emitting element such as an LED. In this case, the warning control unit 326 may warn the worker U that the position of the terminal device 30 is outside the movable range of the robot 20 by emitting light from the light-emitting element. Furthermore, for example, the sensor used to detect the posture of the terminal device 30 is not limited to the inertial sensor 38. For example, the terminal device 30 may include a magnetic sensor in addition to or instead of the inertial sensor 38.

[0085] Next, an outline of the operation of the robot system 1 will be described with reference to FIG.

[0086] Fig. 6 is a sequence chart showing an example of the operation of the robot system 1 shown in Fig. 1. Fig. 6 mainly describes the operation of the robot system 1 when an operation corresponding to a predetermined task is taught to the robot 20. In the example shown in Fig. 6, it is assumed that the robot controller 10 corrects the position indicated by the position information PINF and the attitude indicated by the attitude information AINF every time it acquires position information PINF and attitude information AINF.

[0087] First, the terminal device 30 transmits teaching start information TSINF, which indicates the start of teaching the robot 20, to the robot controller 10 and the imaging device (detection device) 40 (S300). For example, when the worker U performs an operation on the terminal device 30 to start teaching the robot 20, the terminal device 30 transmits the teaching start information TSINF to the robot controller 10 and the imaging device 40. The robot controller 10 and the imaging device 40 receive the teaching start information TSINF (S100 and S400). By receiving the teaching start information TSINF, the robot controller 10 and the imaging device 40 recognize that teaching the robot 20 will start.

[0088] For example, when the imaging device 40 receives the teaching start information TSINF, it may start capturing an image of the terminal device 30. Furthermore, when the robot controller 10 receives the teaching start information TSINF, it may retract the tip portion HP of the robot 20 and the like to a position (for example, outside the imaging range of the imaging device 40) that does not interfere with the imaging of the terminal device 30 by the imaging device 40. Note that the imaging of the terminal device 30 by the imaging device 40 and the retraction of the tip portion HP of the robot 20 may be performed before the teaching start information TSINF is transmitted (before teaching of the robot 20 is started).

[0089] Next, the terminal device 30 accepts a transmission operation to transmit information such as position information PINF and posture information AINF to the robot controller 10. In the example shown in FIG. 6, it is assumed that the operator U performs the transmission operation three times in teaching the robot 20. In one transmission operation, a series of processes from steps S320 to S324, a series of processes from steps S420 to S424, a series of processes from steps S120 to S160, and a series of processes from steps S200 to S220 are executed. In FIG. 6, to distinguish between the repeatedly executed processes, a lowercase alphabet (a, b, or c) is added to the end of each of the reference numerals for steps S120 to S160, S200 to S220, S320 to S324, and S420 to S424. Similarly, a lowercase alphabet (a, b, or c) is added to the end of each of the reference numerals for the position information PINF and posture information AINF.

[0090] For example, upon receiving a first transmission operation, the terminal device 30 transmits instruction information TINF to the imaging device 40 instructing the terminal device 30 to transmit position information PINF (S320a). As a result, the imaging device 40 receives the instruction information TINF (S420a). Furthermore, upon receiving the first transmission operation, the terminal device 30 detects the orientation of the terminal device 30 and generates orientation information AINFa indicating the detected orientation (S322a). Then, the terminal device 30 transmits the generated orientation information AINFa to the robot controller 10 (S324a). As a result, the robot controller 10 receives the orientation information AINFa generated upon receiving the first transmission operation by the terminal device 30 (S122a).

[0091] Furthermore, upon receiving the instruction information TINF, the imaging device 40 generates position information PINFa indicating the position of the terminal device 30 (S422a). Then, the imaging device 40 transmits the generated position information PINFa to the robot controller 10 (S424a). That is, upon the terminal device 30 accepting the first transmission operation, the imaging device 40 generates position information PINFa indicating the position of the terminal device 30 and transmits the generated position information PINFa to the robot controller 10. As a result, the robot controller 10 receives the position information PINFa generated upon the terminal device 30 accepting the first transmission operation (S120a).

[0092] In this way, the robot controller 10 acquires the position information PINFa and the orientation information AINFa when the terminal device 30 accepts the first transmission operation. Then, the robot controller 10 stores the position information PINFa acquired from the imaging device 40 and the orientation information AINFa acquired from the terminal device 30 in the action table MTBL (S124a). As a result, the position indicated by the position information PINFa and the orientation indicated by the orientation information AINFa are registered in the action table MTBL.

[0093] Next, the robot controller 10 outputs a drive signal CTLm to the robot 20 to change the position and posture of the tip HP of the robot 20 to the position and posture registered in the motion table MTBL (S140a). As a result, the robot 20 receives the drive signal CTLm (S200a). Then, the robot 20 moves the position and posture of the tip HP of the robot 20 in accordance with the drive signal CTLm (S202a).

[0094] Next, the robot controller 10 corrects the position and posture registered in the motion table MTBL (S160a). For example, the robot controller 10 controls the motion of the robot 20 based on correction information acquired via a GUI displayed on an operation screen CHS shown in Fig. 8, which will be described later. As a result, the position and posture of the tip part HP of the robot 20 move to the position and posture indicated by the correction information (S220a).

[0095] The processes of steps S160a and S220a are executed, for example, until the position and posture of the tip part HP of the robot 20 become the target position and posture (i.e., the position and posture to be specified). Also, for example, after the correction of the position and posture registered in the operation table MTBL is completed, the robot controller 10 retreats the tip part HP of the robot 20 to a position that does not interfere with the imaging device 40 capturing an image of the terminal device 30.

[0096] The movement of the position and posture of the tip part HP of the robot 20 may be achieved by the operator U directly and manually moving the robot 20. In this case, the robot controller 10 corrects the position and posture registered in the operation table MTBL based on correction information indicating the position and posture of the tip part HP moved by the operator U.

[0097] When the processes of steps S160a and S220a are completed, the process corresponding to the first transmission operation (the series of processes of steps S120a to S160a, S200a to S220a, S320a to S324a, and S420a to S422a) is completed.

[0098] The process corresponding to the second transmission operation and the process corresponding to the third transmission operation are executed in the same manner as the process corresponding to the first transmission operation. For example, when the terminal device 30 accepts the second transmission operation, the robot controller 10 acquires the position information PINFb and the orientation information AINFb (S120b and S122b). When the terminal device 30 accepts the third transmission operation, the robot controller 10 acquires the position information PINFc and the orientation information AINFc (S120c and S122c).

[0099] For example, after the worker U performs a third transmission operation on the terminal device 30, the worker U performs an operation on the terminal device 30 to end the teaching of the robot 20. Upon receiving the operation to end the teaching of the robot 20, the terminal device 30 transmits teaching end information TEINF indicating the end of the teaching of the robot 20 to the robot controller 10 and the image capture device 40 (S340). The robot controller 10 and the image capture device 40 receive the teaching end information TEINF (S180 and S440). By receiving the teaching end information TEINF, the robot controller 10 and the image capture device 40 recognize that the teaching of the robot 20 has ended.

[0100] For example, when the robot controller 10 receives the teaching end information TEINF, it determines the position and posture registered in the motion table MTBL as the position and posture of the tip HP of the robot 20 that executes the motion corresponding to the predetermined task. As a result, the motion information that defines the position and posture of the tip HP of the robot 20 that executes the motion corresponding to the predetermined task is stored in the motion table MTBL. Also, for example, when the imaging device 40 receives the teaching end information TEINF, it ends imaging of the terminal device 30.

[0101] After completing the teaching of the robot 20, the robot controller 10 outputs a drive signal CTLop to the robot 20 to cause the robot 20 to perform a specified action (S190). Then, the robot 20 performs the specified action in accordance with the drive signal CTLop (S242). The specified action is an action in accordance with the position and posture specified by the action information.

[0102] The operation of the robot system 1 is not limited to the example shown in FIG. 6 . For example, an operation to start teaching the robot 20 and an operation to end teaching the robot 20 may be executed on the operation device 15 of the robot controller 10. For example, the process of correcting the position and posture registered in the motion table MTBL may be executed as needed and may be omitted. For example, the robot controller 10 may correct the position and posture indicated by the position information PINF and posture information AINF corresponding to the first transmission operation after the second or third transmission operation. Similarly, the robot controller 10 may correct the position and posture indicated by the position information PINF and posture information AINF corresponding to the second transmission operation after the third transmission operation.

[0103] Also, for example, the terminal device 30 may not need to transmit the instruction information TINF to the image capture device 40. In this case, the robot controller 10 may request the image capture device 40 to transmit the position information PINF when it acquires the posture information APINF from the terminal device 30.

[0104] Next, the operation table MTBL will be described with reference to FIG.

[0105] FIG. 7 is an explanatory diagram showing an example of the operation table MTBL shown in FIG.

[0106] The motion table MTBL shown in FIG. 7 stores motion information for each of a plurality of motions that correspond one-to-one to a plurality of tasks. Each of the plurality of motions is identified, for example, by a motion ID. The motion information corresponding to each motion ID has the same number of records as the number of pieces of position information PINF acquired by the robot controller 10 (more specifically, the information acquisition unit 121) in teaching the motion indicated by the motion ID. Each record has a position number, position information PINF that defines the position of the tip end HP of the robot 20, attitude information AINF that defines the attitude of the tip end HP of the robot 20, and information indicating the motion of the robot 20. In this embodiment, the order in which the information acquisition unit 121 acquires the position information PINF is set as the position number.

[0107] The position number indicates, for example, the order in which the tip part HP moves among the multiple positions indicated by the operation information. For example, the position number included in the operation information corresponding to the operation ID "m001" indicates that the tip part HP moves from the position indicated by the position information PINFa to the position indicated by the position information PINFc via the position indicated by the position information PINFb.

[0108] Furthermore, for example, the action information corresponding to the action ID "m001" includes multiple pieces of position information PINFa, PINFb, and PINFc and multiple pieces of posture information AINFa, AINFb, and AINFc. Furthermore, the action information corresponding to the action ID "m001" includes information indicating actions to be performed by the robot 20 at the positions indicated by each of the multiple pieces of position information PINFa, PINFb, and PINFc. For example, the action information corresponding to the action ID "m001" includes information indicating the start of applying pigment to an object as information indicating the action to be performed by the robot 20 at the position indicated by the position information PINFa. Furthermore, the action information corresponding to the action ID "m001" includes information indicating the continuation of applying pigment to an object as information indicating the action to be performed by the robot 20 at the position indicated by the position information PINFb. Furthermore, the action information corresponding to the action ID "m001" includes information indicating the end of applying pigment to an object as information indicating the action to be performed by the robot 20 at the position indicated by the position information PINFc.

[0109] The action table MTBL is not limited to the example shown in Fig. 7. For example, the action information corresponding to each action ID may not have one or both of the information indicating the action to be performed by the robot 20 at the position indicated by the position information PINF and the position number. In addition, the work performed by the robot 20 is not limited to applying pigment to an object.

[0110] Next, an overview of the operation screen CHS for correcting the positions and postures registered in the operation table MTBL will be described with reference to FIG.

[0111] FIG. 8 is an explanatory diagram illustrating an example of the operation screen CHS. Note that FIG. 8 assumes a case where the operation screen CHS can be switched between a confirmation screen and a correction screen. The confirmation screen is an operation screen CHS for operating the robot 20 in accordance with the position and posture registered in the operation table MTBL. Note that the confirmation screen does not allow correction of the position and posture registered in the operation table MTBL. The correction screen is an operation screen CHS for operating the robot 20 in accordance with the position and posture registered in the operation table MTBL and correcting the position and posture registered in the operation table MTBL. FIG. 8 will explain an overview of the operation screen CHS, taking as an example a case where the operation screen CHS is a correction screen.

[0112] For example, the display control unit 124 of the robot controller 10 outputs display information for displaying the operation screen CHS on the display device 16 to the display device 16. As a result, the operation screen CHS is displayed on the display device 16. The generation of the display information by the display control unit 124 may be triggered when an operation for displaying the operation screen CHS is performed on the robot controller 10, or may be triggered when the robot controller 10 receives teaching start information TSINF.

[0113] The operation screen CHS includes multiple display windows WD (WDs, WDid, WDp, and WDc). The display window WDs displays whether the current operation screen CHS is a confirmation screen or a correction screen. The display window WDid displays an operation ID corresponding to the operation to be executed. The display window WDp displays the current position number. The display window WDc displays the position and posture of the tip part HP of the robot 20 at the current position number. For example, as described in FIG. 1, the position of the tip part HP is represented by the coordinates of the center of the surface SFh of the tip part HP, and the posture of the tip part HP is represented using the rotation angle of the rotation axis Hx, the rotation angle of the rotation axis Hy, and the rotation angle of the rotation axis Hz.

[0114] Furthermore, the operation screen CHS displays a plurality of GUI buttons BT (BTs, BTm, BTc, BTcn, BTd, BTh, and BTf). The button BTs is a GUI for switching the current operation screen CHS from one of the confirmation screen and the correction screen to the other. In the example shown in FIG. 8, when the worker U selects (for example, presses) the button BTs, the operation screen CHS switches from the correction screen to the confirmation screen.

[0115] The button BTm is a GUI for controlling the operation of the robot 20. For example, when the "Start" button BTm is pressed, the operation control unit 123 of the robot controller 10 operates the robot 20 according to the position and posture registered in the operation table MTBL. When the "Stop" button BTm is pressed, the operation control unit 123 stops the operation of the robot 20. When the "Previous" button BTm is pressed, for example, the operation control unit 123 moves the position and posture of the tip part HP from the current position number to the position and posture corresponding to the previous position number. When the "Next" button BTm is pressed, for example, the operation control unit 123 moves the position and posture of the tip part HP from the current position number to the position and posture corresponding to the next position number.

[0116] The button BTc is a GUI that accepts input of correction information for correcting the position and orientation registered in the operation table MTBL. The button BTc is an example of a "reception unit." The display device 16 that displays the operation screen CHS including the button BTc and the like may also be considered as the "reception unit." The operation control unit 123 moves one or both of the position and orientation of the tip part HP based on the correction information accepted via the button BTc, for example.

[0117] For example, when the "+" button BTc of the "+" and "-" buttons BTc corresponding to the Xw axis is pressed, the operation control unit 123 moves the position of the tip portion HP in the positive direction of the Xw axis. Also, when the "-" button BTc of the "+" and "-" buttons BTc corresponding to the Xw axis is pressed, the operation control unit 123 moves the position of the tip portion HP in the negative direction of the Xw axis.

[0118] Furthermore, for example, when the "+" button BTc of the "+" and "-" buttons BTc corresponding to the rotation axis Hx is pressed, the operation control unit 123 rotates the tip end portion HP around the rotation axis Hx as an axis so as to increase the rotation angle of the rotation axis Hx. Furthermore, for example, when the "-" button BTc of the "+" and "-" buttons BTc corresponding to the rotation axis Hx is pressed, the operation control unit 123 rotates the tip end portion HP around the rotation axis Hx as an axis so as to decrease the rotation angle of the rotation axis Hx. For example, when the rotation axis Hx is viewed from a predetermined direction along the rotation axis Hx, the rotation angle of the rotation axis Hx increases by rotating the tip end portion HP clockwise around the rotation axis Hx, and decreases by rotating the tip end portion HP counterclockwise around the rotation axis Hx.

[0119] Button BTcn is a GUI for canceling the position and orientation corrected based on the correction information without finalizing them as the position and orientation of the tip part HP defined by the operation information. For example, when button BTcn is pressed, the operation control unit 123 moves the position and orientation of the tip part HP to the position and orientation of the tip part HP before correction (the position and orientation of the tip part HP at the current position number).

[0120] The button BTd is a GUI for determining the position and orientation registered in the motion table MTBL as the position and orientation of the tip part HP defined by the motion information. For example, when the button BTcn is pressed, the information generator 122 determines the position and orientation registered in the motion table MTBL as the position and orientation of the tip part HP defined by the motion information.

[0121] Furthermore, the button BTh is a GUI for causing the operation control unit 123 to acquire correction information indicating the position and posture of the tip part HP when the worker U manually and directly moves the robot 20. For example, when the button BTh is pressed, the operation control unit 123 acquires correction information indicating the position and posture of the tip part HP when the worker U manually and directly moves the robot 20. Then, the display control unit 124 controls the display device 16 to display the position (coordinates) and posture (rotation angle) indicated by the correction information in the display window WDc.

[0122] Furthermore, the button BTf is a GUI for terminating the display of the operation screen CHS. Note that, if the button BTf is pressed without pressing the button BTd after a correction has been made by pressing the button BTc, etc., the position and orientation registered in the operation table MTBL may not be changed from the contents before the correction was made. Alternatively, if the button BTf is pressed without pressing the button BTd after a correction has been made by pressing the button BTc, etc., the position and orientation registered in the operation table MTBL may be determined as the position and orientation of the tip part HP defined by the operation information. Furthermore, if the button BTf is pressed without pressing the button BTd after a correction has been made by pressing the button BTc, etc., a GUI may be displayed for selecting whether or not to change the position and orientation registered in the operation table MTBL.

[0123] Note that examples of the operation screen CHS are not limited to the example shown in FIG. 8. For example, the operator U may directly input numerical values ​​(coordinates, angles, etc.) indicating the target position and orientation of the tip part HP into the display window WDc. In this case, the display window WDc corresponds to the "reception unit." For example, the operation control unit 123 receives correction information via the display window WDc. Then, the operation control unit 123 moves one or both of the position and orientation of the tip part HP based on the correction information received via the display window WDc.

[0124] Furthermore, when the operation screen CHS is a confirmation screen, for example, the buttons BTc, BTcn, and BTd are displayed on the operation screen CHS so that they cannot be operated by the worker U. Alternatively, when the operation screen CHS is a confirmation screen, the buttons BTc, BTcn, and BTd do not have to be displayed on the operation screen CHS. Furthermore, since the operation of the robot 20 can also be confirmed on the correction screen, an operation screen CHS that corresponds only to the correction screen out of the confirmation screen and the correction screen may be employed.

[0125] Furthermore, in FIG. 8, the button BTc is shown as an example of the "reception unit," but if a physical button to which a function similar to that of the button BTc is assigned is provided on the robot controller 10 as the operation device 15, the operation device 15 is also an example of the "reception unit."

[0126] The operation screen CHS may also be displayed on a display device 36 included in the terminal device 30. In this case, display information for displaying the operation screen CHS on the display device 36 may be generated by the display control unit 124 of the robot controller 10, or may be generated by the display control unit 328 of the terminal device 30. Also, for example, a physical button to which a function similar to that of button BTc or the like is assigned may be provided on the terminal device 30 as the operation device 35.

[0127] Next, an outline of the operation of the robot controller 10 will be described with reference to FIG.

[0128] 9 is a flowchart showing an example of the operation of the robot controller 10 shown in FIG. 9. FIG. 9 shows an example of the operation of the robot controller 10 when the worker U performs an operation to start teaching the robot 20 on the terminal device 30 or the robot controller 10. A detailed description of the operation described in FIG. 6 will be omitted. FIG. 9 assumes a case where an operation to start teaching the robot 20 and an operation to end teaching the robot 20 are performed on the operation device 35 of the terminal device 30. For example, the processing of step S120 is performed after an operation to start teaching the robot 20 is performed on the operation device 35. Also, FIG. 9 assumes a case where the operation screen CHS is displayed before the processing of step S120 is performed.

[0129] First, in step S120, the processing device 12 functions as the information acquisition unit 121 and acquires the position information PINF indicating the position of the terminal device 30 from the imaging device 40.

[0130] Next, in step S122, the processing device 12 functions as the information acquisition unit 121 and acquires, from the terminal device 30, the attitude information AINF indicating the attitude of the terminal device 30.

[0131] Next, in step S124, the processing device 12 functions as the information generating unit 122, and stores the position information PINF acquired in step S120 and the attitude information AINF acquired in step S122 in the motion table MTBL.

[0132] Next, in step S130, the processing device 12 functions as the operation control unit 123 and determines whether or not the operation of the robot 20 is to be confirmed. Whether or not the operation of the robot 20 is to be confirmed may be determined based on whether or not the robot 20 and the imaging device 40 have been calibrated in advance. If the robot 20 and the imaging device 40 have been calibrated, the terminal device 30 and the robot 20 have also been calibrated, and therefore the operation of the robot 20 does not need to be confirmed. On the other hand, if the robot 20 and the imaging device 40 have not been calibrated in advance, the operation of the robot 20 needs to be confirmed to align the position taught by the terminal device 30 with the position to which the robot 20 will move. Furthermore, the operation of the robot 20 may be confirmed not only to calibrate the robot 20 and the imaging device 40, but also to check the actual operation of the robot 20 to check for obstacles and correct its position and posture. Regarding the determination operation, for example, the operation control unit 123 may determine whether or not the operation of the robot 20 is to be checked by determining whether or not a button BTm displayed on the operation screen CHS has been pressed. In this case, the operation control unit 123 determines that the operation of the robot 20 is to be checked if, for example, the button BTm has been pressed before the communication device 14 of the robot controller 10 receives teaching end information TEINF, next position information PINF, next posture information APINF, or the like.

[0133] If the result of the determination in step S130 is positive, the processing device 12 advances the process to step S140. On the other hand, if the result of the determination in step S130 is negative, the processing device 12 advances the process to step S182.

[0134] In step S140, the processing device 12 functions as the operation control unit 123, and moves the position and posture of the tip part HP based on the position and posture registered in the operation table MTBL. Then, the processing device 12 advances the process to step S150.

[0135] In step S150, the processing device 12 functions as the operation control unit 123 and determines whether or not correction of the position and attitude registered in the operation table MTBL is to be performed. For example, the operation control unit 123 may determine whether or not correction is to be performed by determining whether or not the button BTc or BTh displayed on the operation screen CHS has been pressed. In this case, the operation control unit 123 determines that correction is to be performed, for example, if the button BTc or BTh has been pressed before the communication device 14 receives the teaching end information TEINF, the next position information PINF, the next attitude information APINF, or the like. Alternatively, the operation control unit 123 may determine that correction is to be performed, for example, if a numerical value indicating the position of the tip end HP, etc., has been input into the display window WDc before the communication device 14 receives the teaching end information TEINF, the next position information PINF, the next attitude information APINF, or the like.

[0136] If the result of the determination in step S150 is positive, the processing device 12 advances the process to step S160. On the other hand, if the result of the determination in step S150 is negative, the processing device 12 advances the process to step S182.

[0137] In step S160, the processing device 12 functions as the movement control unit 123 and corrects the positions and postures registered in the movement table MTBL. For example, the movement control unit 123 corrects the positions and postures registered in the movement table MTBL based on correction information received via a button BTc or the like displayed on the operation screen CHS. Then, the processing device 12 advances the process to step S162.

[0138] In step S162, the processing device 12 functions as the information generation unit 122 and confirms or cancels the correction performed in step S160. For example, when button BTd is pressed, the information generation unit 122 confirms the correction performed in step S160, and when button BTcn is pressed, the information generation unit 122 cancels the correction performed in step S160. After performing the process of step S162, the processing device 12 proceeds to step S182.

[0139] In step S182, the processing device 12 functions as the operation control unit 123 and determines whether or not teaching of the robot 20 has ended. For example, the operation control unit 123 determines that teaching of the robot 20 has ended when the communication device 14 receives teaching end information TEINF before the communication device 14 receives the next position information PINF or the next posture information APINF, etc.

[0140] If the result of the determination in step S182 is positive, the processing device 12 advances the process to step S184. On the other hand, if the result of the determination in step S182 is negative, the processing device 12 returns the process to step S120.

[0141] In step S184, the processing device 12 functions as the information generation unit 122 and determines the position and posture registered in the motion table MTBL as the position and posture of the tip part HP defined by the motion information. This generates motion information defining the position and posture of the tip part HP. Trajectory information for the robot 20 is generated by sequentially connecting these defined positions and postures. The robot 20 operates in sequence to achieve these defined positions and postures, thereby realizing motion in accordance with the trajectory information. Note that the position and posture correction in the motion confirmation of steps 130 to 182 may be performed after the trajectory information is generated, or the trajectory information may be generated from the corrected position and posture.

[0142] The operation of the robot controller 10 is not limited to the example shown in Fig. 9. For example, the process of step S122 may be executed before the process of step S120, or may be executed in parallel with the process of step S120.

[0143] As described above, in this embodiment, the robot controller 10 has an information acquisition unit 121 and an information generation unit 122. The information acquisition unit 121 acquires position information PINF indicating the position of the terminal device 30 and posture information AINF indicating the posture of the terminal device 30. The information generation unit 122 generates operation information that defines the position and posture of the tip end HP of the robot 20, based on the position information PINF and posture information AINF acquired by the information acquisition unit 121.

[0144] As described above, in this embodiment, operation information that defines the position and posture of the tip end HP of the robot 20 is generated based on the position and posture of the terminal device 30. For example, when the worker U moves the position and posture of the terminal device 30, the physical and time burden on the worker U is reduced compared to when the worker U manually and directly moves the robot 20. Therefore, in this embodiment, by having the worker U manually and directly move the robot 20 from the beginning, it is possible to suppress an increase in the physical and time burden on the worker U compared to a teaching method in which operation information that defines the position and posture of the tip end HP of the robot 20 is generated.

[0145] Furthermore, in this embodiment, the information acquisition unit 121 acquires the position information PINF and the posture information AINF when the worker U performs a predetermined operation on the terminal device 30. For example, the predetermined operation is a transmission operation for transmitting information such as the position information PINF and the posture information AINF to the robot controller 10. For example, the worker U can cause the information acquisition unit 121 to acquire the position information PINF indicating the target position and the posture information AINF indicating the target posture by performing the predetermined operation at the timing when the position and posture of the terminal device 30 are moved to a target position and posture. As a result, in this embodiment, the information acquisition unit 121 can easily acquire the position information PINF indicating the target position and the posture information AINF indicating the target posture.

[0146] In this embodiment, the position information PINF is generated by the imaging device 40, which captures an image of the terminal device 30. In this embodiment, the information acquisition unit 121 acquires the position information PINF generated by the imaging device 40. In this manner, in this embodiment, the position information PINF is generated by the imaging device 40, which makes it possible to prevent the processing of the robot controller 10 from becoming complicated compared to an embodiment in which the robot controller 10 measures the position of the terminal device 30. In addition, the robot system 1 including the robot controller 10 can employ, as the imaging device 40, a known camera, such as a three-dimensional camera, that measures the position of an object. As a result, in this embodiment, the robot system 1 can be easily realized.

[0147] In this embodiment, the robot controller 10 further includes an operation control unit 123 that operates the robot 20 based on the position information PINF and posture information AINF acquired by the information acquisition unit 121. As a result, in this embodiment, the worker U can easily confirm the operation of the robot 20 based on the position information PINF and posture information AINF acquired by the information acquisition unit 121. As a result, in this embodiment, the efficiency of the work for generating operation information that defines the position and posture of the tip end HP of the robot 20 can be improved.

[0148] Furthermore, in this embodiment, the operation control unit 123 may acquire correction information indicating the position and posture of the tip part HP moved by the worker U, and correct the position indicated by the position information PINF and the posture indicated by the posture information AINF based on the correction information. In this way, in this embodiment, the position and posture indicated by the position information PINF and the posture information AINF acquired by the information acquisition unit 121 can be corrected based on the correction information, so that the position and posture of the tip part HP of the robot 20 can be accurately determined.

[0149] Furthermore, this embodiment further includes a receiving unit (for example, a button BTc displayed on the operation screen CHS as a GUI) that receives input of correction information for correcting the position indicated by the position information PINF and the attitude indicated by the attitude information AINF. By pressing the button BTc, for example, the worker U can easily move the position and attitude of the tip end HP of the robot 20 to a target position and attitude. As a result, this embodiment can accurately determine the position and attitude of the tip end HP of the robot 20 while suppressing an increase in the burden on the worker U.

[0150] In this embodiment, the robot controller 10 further includes a warning unit 125 that determines whether the position indicated by the position information PINF is within the movable range of the tip part HP and issues a warning if the position indicated by the tip part HP is not within the movable range of the tip part HP. In this case, the worker U can recognize that the position of the terminal device 30 is outside the movable range of the robot 20 by recognizing a warning, such as a warning sound. For example, if the worker U recognizes that the movement path of the terminal device 30 is outside the movable range of the robot 20 after completing the operation of moving the terminal device 30, the worker U may have to move the terminal device 30 again. In this case, the efficiency of the operation for generating operation information that defines the position and posture of the tip part HP of the robot 20 decreases. In other words, in this embodiment, the efficiency of the operation for generating operation information that defines the position and posture of the tip part HP of the robot 20 can be improved.

[0151] [2. Modifications] The present invention is not limited to the above-described exemplary embodiments. Specific modified embodiments are exemplified below. Two or more embodiments selected from the following examples may be combined.

[0152] [First Modification] In the above-described embodiment, the information acquisition unit 121 acquires the position information PINF and the posture information AINF when the worker U performs a predetermined operation on the terminal device 30. However, the present invention is not limited to this example. For example, the information acquisition unit 121 may repeatedly acquire the position information PINF and the posture information AINF during the acquisition period, as shown in Fig. 10, which will be described later.

[0153] Fig. 10 is a sequence chart showing an example of the operation of the robot system 1 according to the first modified example. Detailed description of operations similar to those described in Fig. 6 will be omitted. In the operation shown in Fig. 10, the robot controller 10 (more specifically, the information acquisition unit 121) repeatedly acquires position information PINF and posture information AINF during an acquisition period whose start timing T1 and end timing T2 are specified by the worker U.

[0154] 10, it is assumed that the worker U performs a start operation to start the acquisition period and a stop operation to end the acquisition period on the operation device 35 of the terminal device 30. For example, if the operation device 35 includes a send button, the start operation may be pressing the send button, and the stop operation may be releasing the send button. In this case, the period during which the send button is pressed corresponds to the acquisition period. Also, for example, if the operation device 35 includes a start button and an end button, the start operation may be pressing the start button, and the stop operation may be pressing the end button.

[0155] 10, similarly to Fig. 6, in order to distinguish the repeatedly executed processes from one another, the reference numerals of steps S120 to S124, S322 to S324, and S422 to S424 are each suffixed with a lowercase alphabet (a, b, or c). The reference numerals of position information PINF and posture information AINF are also each suffixed with a lowercase alphabet (a, b, or c).

[0156] First, upon receiving an operation to start teaching the robot 20, the terminal device 30 transmits teaching start information TSINF to the robot controller 10 and the imaging device (detection device) 40 (S300). As a result, the robot controller 10 and the imaging device 40 receive the teaching start information TSINF (S100 and S400).

[0157] Next, upon receiving a start operation to start the acquisition period, the terminal device 30 transmits start information PSINF indicating the start of the acquisition period to the robot controller 10 and the imaging device 40 (S302). As a result, the robot controller 10 and the imaging device 40 receive the start information PSINF (S102 and S402). Note that the start information PSINF may be transmitted from the terminal device 30 to the imaging device 40 via the robot controller 10. By receiving the start information PSINF, the robot controller 10 and the imaging device 40 recognize that the acquisition period will start.

[0158] During the acquisition period, the terminal device 30 repeatedly executes a series of processes including a process of generating orientation information AINF indicating the orientation of the terminal device 30 and a process of transmitting the generated orientation information AINF to the robot controller 10 (S322 and S324). Also, the imaging device 40 repeatedly executes a series of processes including a process of generating position information PINF indicating the position of the terminal device 30 and a process of transmitting the generated position information PINF to the robot controller 10 (S422 and S424). Then, the robot controller 10 repeatedly executes a series of processes including a process of acquiring the position information PINF and orientation information AINF and a process of storing the acquired position information PINF and orientation information AINF in the operation table MTBL (S120, S122, and S124).

[0159] Next, upon receiving the end operation to end the acquisition period, the terminal device 30 transmits end information PEINF indicating the end of the acquisition period to the robot controller 10 and the imaging device 40 (S326). As a result, the robot controller 10 and the imaging device 40 receive the end information PEINF (S126 and S426). Note that the end information PEINF may be transmitted from the terminal device 30 to the imaging device 40 via the robot controller 10. By receiving the end information PEINF, the robot controller 10 and the imaging device 40 recognize that the acquisition period has ended.

[0160] The robot controller 10 outputs a drive signal CTLm to the robot 20 to change the position and posture of the tip HP of the robot 20 to the position and posture registered in the motion table MTBL (S140). As a result, the robot 20 receives the drive signal CTLm (S200a). Then, the robot 20 changes the position and posture of the tip HP of the robot 20 in accordance with the drive signal CTLm (S202).

[0161] Next, the robot controller 10 corrects the position and orientation registered in the motion table MTBL (S160). For example, the position and orientation of the tip part HP of the robot 20 is moved to the position and orientation indicated by the correction information (S220).

[0162] When the terminal device 30 receives an operation to end teaching the robot 20, it transmits teaching end information TEINF to the robot controller 10 and the image capturing device 40 (S340). The robot controller 10 and the image capturing device 40 receive the teaching end information TEINF (S180 and S440). By receiving the teaching end information TEINF, the robot controller 10 and the image capturing device 40 recognize that teaching the robot 20 has ended.

[0163] After completing the teaching of the robot 20, the robot controller 10 outputs a drive signal CTLop to the robot 20 to cause the robot 20 to perform a specified action (S190). Then, the robot 20 performs the specified action in accordance with the drive signal CTLop received from the robot controller 10 (S240 and S242).

[0164] The operation of the robot system 1 is not limited to the example shown in FIG. 10. For example, the process of correcting the position and orientation registered in the operation table MTBL may be executed as needed and may be omitted. Furthermore, the information acquisition unit 121 may repeatedly acquire the position information PINF and the orientation information AINF during the acquisition period at predetermined intervals. Alternatively, the terminal device 30 may transmit the next position information PINF and the instruction information TINF described in FIG. 6 each time it moves a predetermined distance after transmitting the position information PINF. In this case, the movement amount of the terminal device 30 may be calculated based on, for example, the detection result of the inertial sensor 38.

[0165] Next, an outline of the operation of the robot controller 10 according to the first modified example will be described with reference to FIG.

[0166] Fig. 11 is a flowchart showing an example of the operation of the robot controller 10 according to the first modified example. The operation shown in Fig. 11 is the same as the operation shown in Fig. 9, except that after the acquisition period ends, the operation of the robot 20 is checked. Detailed description of the operations described in Figs. 9 and 10 will be omitted. Note that the processing of step S104 is executed, for example, after an operation to start teaching the robot 20 is executed on the operating device 35.

[0167] First, in step S104, the processing device 12 functions as the information acquisition unit 121 and determines whether or not the acquisition period has started. For example, the information acquisition unit 121 determines whether or not the acquisition period has started by determining whether or not the communication device 14 of the robot controller 10 has received start information PSINF. In this case, for example, the information acquisition unit 121 determines that the acquisition period has started when the communication device 14 has received start information PSINF.

[0168] If the result of the determination in step S104 is positive, the processing device 12 advances the process to step S120. On the other hand, if the result of the determination in step S104 is negative, the processing device 12 returns the process to step S104.

[0169] The series of processes in steps S120, S122, and S124 are the same as the series of processes in steps S120, S122, and S124 shown in Fig. 6. For example, in steps S120, S122, and S124, the processing device 12 functions as the information acquisition unit 121, acquires position information PINF and orientation information AINF, and stores the acquired position information PINF and orientation information AINF in the operation table MTBL. Then, after executing the process of step S124, the processing device 12 proceeds to the process of step S128.

[0170] In step S128, the processing device 12 functions as the information acquisition unit 121 and determines whether the acquisition period has ended. For example, the information acquisition unit 121 determines whether the acquisition period has ended by determining whether the communication device 14 has received end information PEINF. In this case, the information acquisition unit 121 determines that the acquisition period has ended, for example, when the communication device 14 has received end information PEINF.

[0171] If the result of the determination in step S128 is positive, the processing device 12 advances the process to step S130. On the other hand, if the result of the determination in step S128 is negative, the processing device 12 returns the process to step S120.

[0172] The series of processes from step S130 to step S184 is the same as the series of processes from step S130 to step S184 shown in Fig. 6. For example, in steps S130, S140, S150, S160, and S182, the processing device 12 functions as the operation control unit 123, and in steps S162 and S184, the processing device 12 functions as the information generation unit 122.

[0173] For example, in step S130, the operation control unit 123 determines whether or not the operation of the robot 20 is to be confirmed. If the result of the determination in step S130 is positive, in step S140, the operation control unit 123 moves the position and posture of the tip part HP based on the position and posture registered in the movement table MTBL. Then, the processing device 12 proceeds to step S150. On the other hand, if the result of the determination in step S130 is negative, in step S182, the operation control unit 123 determines whether or not teaching of the robot 20 has ended.

[0174] In step S150, the movement control unit 123 determines whether or not the position and posture registered in the movement table MTBL are to be corrected. If the result of the determination in step S150 is positive, the movement control unit 123 corrects the position and posture registered in the movement table MTBL in step S160. The processing device 12 then proceeds to step S162. On the other hand, if the result of the determination in step S150 is negative, the movement control unit 123 determines in step S182 whether or not teaching of the robot 20 has been completed.

[0175] In step S162, the information generator 122 confirms or cancels the correction executed in step S160. After the processing of step S162 is executed, in step S182, the operation controller 123 determines whether or not teaching the robot 20 has ended.

[0176] If the result of the determination in step S182 is positive, the processing device 12 advances the process to step S184. On the other hand, if the result of the determination in step S182 is negative, the processing device 12 returns the process to step S130.

[0177] In step S184, the information generation unit 122 determines the position and posture registered in the movement table MTBL as the position and posture of the tip part HP defined by the movement information. This defines the position and posture of the tip part HP. Trajectory information for the robot 20 is generated by sequentially connecting these defined positions and postures. The robot 20 operates in sequence to achieve these defined positions and postures, thereby realizing movement in accordance with the trajectory information. Note that the position and posture correction in the movement confirmation in steps 130 to 182 may be performed after the trajectory information is generated, or the trajectory information may be generated from the corrected position and posture.

[0178] The operation of the robot controller 10 is not limited to the example shown in Fig. 11. For example, the process of step S122 may be executed before the process of step S120, or may be executed in parallel with the process of step S120.

[0179] As described above, this modification can also achieve the same effects as the above-described embodiment. Furthermore, in this modification, the information acquisition unit 121 repeatedly acquires position information PINF and orientation information AINF during an acquisition period for which the start timing T1 and end timing T2 are specified by the worker U. Therefore, in this modification, even when a large number of motion points of the robot 20 are specified, the information acquisition unit 121 can acquire the position information PINF and orientation information AINF while suppressing an increase in the burden on the worker U. Furthermore, in this modification, the path of the position of the tip end HP of the robot 20 can be specified as a trajectory determined based on multiple points indicated by multiple pieces of position information PINF (for example, a trajectory determined by interpolating multiple points). As a result, in this modification, the position and orientation of the tip end HP of the robot 20 can be smoothly moved.

[0180] [Second Modification] In the above-described embodiment and modified example, the inertial sensor 38 detects the tilt (attitude) of the terminal device 30, and the imaging device (detection device) 40 detects the position of the terminal device 30. In this modified example, the inertial sensor 38 of the terminal device 30 detects the tilt (attitude) and position of the terminal device 30. Therefore, the imaging device 40 is not necessarily required in this modified example. As described above, the inertial sensor 38 in this modification includes acceleration sensors for the terminal device 30 on the Xw, Yw, and Zw axes, which represent three-dimensional space, and angular velocity sensors for the terminal device 30 when rotated about the Xw, Yw, and Zw axes. The tilt (orientation) of the terminal device 30 is measured by the angular velocity sensors for the Xw, Yw, and Zw axes. Meanwhile, the movement distance of the terminal device 30 is detected by the acceleration sensors for the Xw, Yw, and Zw axes when the terminal device 30 is moved sequentially to multiple positions in three-dimensional space. In this modification, the position and orientation to be taught to the robot 20 are corrected at at least one of the positions to which the terminal device 30 is moved, similar to the operation check shown in S130 to S182 of the flowchart of the first embodiment shown in FIG. 9. As a result, the relative position of the terminal device 30 with respect to the robot 20 can be calculated from the movement distance of the terminal device 30 detected by the acceleration sensors of the Xw axis, Yw axis, and Zw axis described above. In this modification, the position information PINF (S422a, S422b, S422c) generated by the imaging device 40 in the sequence chart of the first embodiment shown in Fig. 6 is generated by the terminal device 30 and transmitted to the robot controller 10. Other than this, the process is the same as in the first embodiment. As described above, this modification can also achieve the same effects as the above-described embodiment. Furthermore, in this modification, the inertial sensor 38 of the terminal device 30 is used to detect the position and orientation of the terminal device 30, so there is no need to prepare an imaging device 40, and the robot system can be easily prepared. Furthermore, because no imaging is performed using the imaging device 40, it is possible to continue operating the terminal device 30 without worrying about areas or orientations that are difficult to image due to obstacles, etc., and operability is extremely excellent. It is also possible to use both position detection using the terminal device 30 and position detection using the imaging device 40 shown in the first embodiment. When position detection is performed using the imaging device 40, the mark MK may not be detected due to obstacles or the angle of the terminal device 30. In such cases, detection can be continued by switching to position detection using the terminal device 30. In this case, the position detected using the inertial sensor 38 of the terminal device 30 can be identified by calculating the travel distance from the position detected using the imaging device 40 to the position detected using the inertial sensor 38 of the terminal device 30. [Third Modification] In the above-described embodiment and modified example, the position information PINF is generated by the imaging device (detection device) 40, but the present invention is not limited to such an embodiment. For example, the position information PINF may be generated by the terminal device 30. In this case, the information acquisition unit 121 acquires the position information PINF and the attitude information AINF generated by the terminal device 30.

[0181] Fig. 12 is an explanatory diagram for explaining an overview of a robot system 1A according to a third modified example. Elements similar to those explained in Fig. 1 to Fig. 11 are given the same reference numerals, and detailed explanations thereof will be omitted.

[0182] 1, except that the imaging device 40 is omitted, and instead of the marks MK1 and MK2 being provided on the terminal device 30, multiple marks MK3 (MK3a, MK3b, and MK3c) are provided on the robot 20. In addition, in this modification, for convenience of explanation, a moving body coordinate system ΣM fixed to the terminal device 30 is introduced instead of the imaging coordinate system ΣC.

[0183] The mobile body coordinate system ΣM is a three-axis Cartesian coordinate system having an origin at a predetermined position of the terminal device 30 and having an Xm-axis, a Ym-axis, and a Zm-axis that are orthogonal to each other. In this modification, it is assumed that the Xm-axis, the Ym-axis, and the Zm-axis of the mobile body coordinate system ΣM are the Xm-axis, the Ym-axis, and the Zm-axis described in Fig. 1. In addition, it is assumed that the Zm-axis is parallel to the optical axis of the imaging device 39 included in the terminal device 30 (more specifically, the optical axis of the optical system included in the imaging device 39).

[0184] Also, it is assumed that the positions of the marks MK3a, MK3b, and MK3c in the world coordinate system ΣW are known. That is, the relative positions of the marks MK3a, MK3b, and MK3c (corresponding to the posture of the mark MK3) are also known. The mark MK3 may be provided on the robot 20 (on the base part BSP in the example shown in FIG. 12) or in the vicinity of the robot 20.

[0185] The configuration of the terminal device 30 is the same as the configuration of the terminal device 30 shown in Figures 4 and 5. The imaging device 39 included in the terminal device 30 captures an image of marks MK3 (MK3a, MK3b, and MK3c) placed at positions whose relative positional relationship with a predetermined position (for example, position Op) of the robot 20 is known. Then, the terminal device 30 calculates the position of the terminal device 30 relative to the predetermined position using the image of the marks MK3 captured by the imaging device 39, and generates information indicating the calculated position as position information PINF.

[0186] Below, an example of a method for calculating the position of the terminal device 30 using an image of the mark MK3 captured by the imaging device 39 will be described, but the method for calculating the position of the terminal device 30 is not limited to the following example.

[0187] For example, let Va be the vector that starts from the optical system of the imaging device 39 and ends at mark MK3a. Let Vb be the vector that starts from the optical system of the imaging device 39 and ends at mark MK3b. And let Vc be the vector that starts from the optical system of the imaging device 39 and ends at mark MK3c.

[0188] Furthermore, let Vab be the vector that starts from mark MK3a and ends at mark MK3b. Let Vac be the vector that starts from mark MK3a and ends at mark MK3c. And let Vbc be the vector that starts from mark MK3b and ends at mark MK3c. In this case, vectors Va, Vb, Vc, Vab, Vac, and Vbc satisfy the relationships of the following equations (1), (2), and (3). |Va-Vb|=|Vab| (1) |Va-Vc|=|Vac| (2) |Vb-Vc|=|Vbc| (3)

[0189] The lengths of the vectors Vab, Vac, and Vbc (|Vab|, |Vac|, and |Vbc|) are known because the positions of the marks MK3a, MK3b, and MK3c in the world coordinate system ΣW are known.

[0190] For example, the processing device 32 included in the terminal device 30 acquires information indicating the direction of each of the vectors Va, Vb, and Vc from an image capturing the marks MK3a, MK3b, and MK3c (for example, two-dimensional information indicating the position of each mark MK3 on the image).The processing device 32 then calculates the length of each of the vectors Va, Vb, and Vc based on the above formulas (1), (2), and (3), the information indicating the direction of each of the vectors Va, Vb, and Vc, and the focal length.The focal length is the focal length of the optical system included in the imaging device 39, and is known information.

[0191] The processing device 32 also calculates the position of each of the marks MK3a, MK3b, and MK3c in the moving body coordinate system ΣM based on information indicating the direction of each of the vectors Va, Vb, and Vc and information indicating the length of each of the vectors Va, Vb, and Vc. The processing device 32 also calculates the position of the terminal device 30 in the world coordinate system ΣW based on the positions of each of the marks MK3a, MK3b, and MK3c in the world coordinate system ΣW and the positions of each of the marks MK3a, MK3b, and MK3c in the moving body coordinate system ΣM. The processing device 32 then transmits position information PINF indicating the position of the terminal device 30 in the world coordinate system ΣW to the robot controller 10.

[0192] The process of calculating the position of the terminal device 30 in the world coordinate system ΣW described above may be executed by the orientation detection unit 324, or may be executed by a functional block separate from the orientation detection unit 324. For example, the processing device 32 may function as a functional block that executes the process of calculating the position of the terminal device 30 in the world coordinate system ΣW, separate from the orientation detection unit 324.

[0193] Furthermore, the attitude detection unit 324 may detect the attitude of the terminal device 30 using the positions of the marks MK3a, MK3b, and MK3c in the moving body coordinate system ΣM instead of the detection result of the inertial sensor 38. For example, the attitude detection unit 324 may detect the attitude of the terminal device 30 based on the positions of the marks MK3a, MK3b, and MK3c in the world coordinate system ΣW and the positions of the marks MK3a, MK3b, and MK3c in the moving body coordinate system ΣM. In this case, the terminal device 30 does not need to have the inertial sensor 38.

[0194] In this manner, in this modification, the position information PINF and the posture information AINF are generated by the terminal device 30. Then, the information acquisition unit 121 of the robot controller 10 acquires the position information PINF and the posture information AINF generated by the terminal device 30.

[0195] The configuration of the robot system 1A in this modified example is not limited to the example shown in FIG. 12. For example, the mark MK3 may be a QR code (registered trademark). Also, a light-emitting element such as an LED may be used as the mark MK3. Also, for example, if the imaging device 39 is a three-dimensional camera, the number of marks MK3 may be one.

[0196] As described above, this modification can also achieve the same effects as the above-described embodiment and modification. Furthermore, in this modification, the terminal device 30 has an imaging device 39 that captures an image of a mark MK3 that is placed at a position whose relative positional relationship with a predetermined position of the robot 20 is known. The terminal device 30 then calculates the position of the terminal device 30 relative to the predetermined position using the image of the mark MK3 captured by the imaging device 39, and generates information indicating the calculated position as position information PINF. The information acquisition unit 121 of the robot controller 10 acquires the position information PINF generated by the terminal device 30. In other words, in this modification, there is no need to place an imaging device 40 separately from the terminal device 30, which can prevent the installation space for the robot system 1A from becoming large and the arrangement of devices such as the robot 20 of the robot system 1A from becoming complicated.

[0197] [Fourth Modification] In the third modified example described above, the case where the position and orientation of the terminal device 30 are identified by the terminal device 30 has been exemplified, but the present invention is not limited to this aspect. For example, one or both of the position and orientation of the terminal device 30 may be identified by the robot controller 10. In this case, the information acquisition unit 121 of the robot controller 10 acquires, for example, image information indicating images of marks MK3a, MK3b, and MK3c captured by the imaging device 39 and information indicating the focal length of the imaging device 39 from the terminal device 30. Then, for example, the information generation unit 122 of the robot controller 10 identifies the position, etc. of the terminal device 30 using the same method as described in the third modified example (a method of calculating the position of the terminal device 30 using the image of mark MK3 captured by the imaging device 39).

[0198] When the robot controller 10 identifies the position of the terminal device 30 from image information showing images of the marks MK3a, MK3b, and MK3c, the image information corresponds to "position information." Similarly, when the robot controller 10 identifies the posture of the terminal device 30 from image information showing images of the marks MK3a, MK3b, and MK3c, the image information corresponds to "posture information." In this modification, too, the same effects as in the third modification described above can be obtained.

[0199] [Fifth Modification] In the above-described embodiment and first modified example, the orientation information AINF is generated by the terminal device 30, but the present invention is not limited to this. For example, the orientation information AINF may be generated by the imaging device (detection device) 40 based on images of the marks MK1 and MK2 captured by the imaging device 40. In this case, the information acquisition unit 121 acquires the position information PINF and the orientation information AINF generated by the imaging device 40. Alternatively, as in the fourth modified example, one or both of the position and orientation of the terminal device 30 may be identified by the robot controller 10. In this case, the information acquisition unit 121 of the robot controller 10 acquires, for example, image information indicating the images of the marks MK1 and MK2 captured by the imaging device 40 and information indicating the focal length of the imaging device 40 from the imaging device 40. In this modified example, the same effects as those of the above-described embodiment and first modified example can be obtained.

[0200] [Sixth Modification] In the above-described embodiment and modified example, a portable information processing device such as the terminal device 30 is used as the “moving object.” However, the present invention is not limited to such an embodiment. For example, an object other than an information processing device may be used as the “moving object” as long as it is provided with marks MK1 and MK2 and can be moved by the worker U. When an object other than an information processing device is used as the “moving object,” for example, the robot system 1 further includes a transmission instruction device that transmits instruction information TINF, etc., instructing the image capture device (detection device) 40 to transmit position information PINF upon receiving a transmission operation. For example, the worker U holds an object provided with marks MK1 and MK2 with one hand and operates the transmission instruction device with the other hand. The object’s posture and position are identified based on the images of the marks MK1 and MK2 captured by the image capture device 40. The posture and position of the object may be identified by the image capture device 40 or the robot controller 10. This modified example also achieves the same effects as the above-described embodiment and modified example. [Seventh Modification] In the above-described embodiment and modified example, an example has been given in which operation information defining the position and posture of the tip part HP of the robot 20 is generated, but the present invention is not limited to such an embodiment. For example, the operation information may be information defining only the position of the position and posture of the tip part HP of the robot 20. In this case, for example, the information acquisition unit 121 acquires position information PINF indicating the position of the terminal device 30. Furthermore, the information generation unit 122 generates operation information defining the position of the tip part HP of the robot 20 based on the position information PINF acquired by the information acquisition unit 121. In this modified example, the same effects as those of the above-described embodiment and modified example can be obtained. In the above-described embodiment and modified examples, the worker U holding the mobile object 30 may perform teaching at a position shifted a predetermined distance from the robot 20 itself. In this case, teaching can be completed by offsetting the position information of the mobile object 30 by a predetermined distance. This allows teaching without being disturbed by the robot 20 itself, and more complex teaching can be performed. For example, by attaching the mobile object 30 to the worker U's wrist and automatically performing teaching at predetermined intervals, teaching can be performed while the worker U is actually working, eliminating the need for a separate task called "teaching." It also makes it possible to teach the movements of people who perform a variety of intense and detailed movements, such as dancers. [Explanation of symbols]

[0201] 1, 1A...robot system, 10...robot controller, 12...processing device, 13...memory, 14...communication device, 15...operation device, 16...display device, 17...driver circuit, 20...robot, 30...terminal device, 32...processing device, 33...memory, 34...communication device, 35...operation device, 36...display device, 37...speaker, 38...inertial sensor, 39...imaging device, 40...imaging device (detection device), 120...robot control unit, 121...information acquisition unit, 122...information generation unit, 123...operation control unit, 124...display control unit, 125...warning unit, 320...terminal control unit, 322...operation notification unit, 324...posture detection unit, 326...warning control unit, 328...display control unit, AINF...posture information, CHS...operation screen, HP...tip unit, MTBL...operation table, PINF...position information.

Claims

1. An information processing device that generates position information at a plurality of positions that form a trajectory along which a robot moves, or trajectory information for teaching the robot that is composed of the position information and posture information, an acquisition unit that acquires the position information or the position information and the posture information at the plurality of positions of a moving body that is moved to and placed at the plurality of positions in sequence independent of the robot; a generation unit that generates the trajectory information by sequentially defining a position or a position and a posture of the robot based on the position information or the position information and the posture information at the plurality of positions acquired by the acquisition unit; An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, further comprising a control unit that operates the robot based on the trajectory information.

3. 3. The information processing device according to claim 2, further comprising an operation unit that accepts input of correction information for correcting the position indicated by the position information, or the position indicated by the position information and the attitude indicated by the attitude information, and the control unit corrects the trajectory information based on the correction information.

4. A robot system having a robot, a moving body, a detection means, and an information processing device, The moving body is an inertial sensor that detects attitude information of the moving body; a communication unit capable of communicating with the information processing device; It has The detection means a detection unit that detects position information of the moving object; a communication unit capable of communicating with the information processing device; It has The information processing device includes: an acquisition unit that acquires the posture information and the position information of the moving body detected at a plurality of positions that form a trajectory along which the robot moves, where the moving body is placed; a generation unit that generates trajectory information for teaching a motion of the robot by sequentially defining a position and a posture of the robot based on the position information and posture information at the plurality of positions acquired by the acquisition unit; a control unit that operates the robot based on the trajectory information; A robot system comprising:

5. In a robot system having a robot, a mobile body, and an information processing device, The moving body is an inertial sensor that detects position information or attitude information of the moving body and the position information; a communication unit capable of communicating with the information processing device; It has The information processing device includes: an acquisition unit that acquires the position information of the moving body, or the posture information and the position information, detected at a plurality of positions that form a trajectory along which the robot moves, where the moving body is placed; a generation unit that generates trajectory information for teaching a motion of the robot by sequentially defining a position and a posture of the robot based on the position information or the position information and the posture information at the plurality of positions acquired by the acquisition unit; a control unit that operates the robot based on the trajectory information; A robot system comprising:

6. A robot system having a robot, a moving body, a detection means, and an information processing device, The moving body is a communication unit capable of communicating with the information processing device, The detection means an imaging detection unit that detects the moving object to detect position information of the moving object, or the position information and attitude information of the moving object; a communication unit capable of communicating with the information processing device; It has The information processing device includes: an acquisition unit that acquires the position information, or the position information and the posture information, of the moving body detected at a plurality of positions that form a trajectory along which the robot moves, where the moving body is placed; a generation unit that generates trajectory information for teaching a motion of the robot by sequentially defining a position of the robot, or a position and a posture of the robot, based on the position information, or the position information and the posture information, at the plurality of positions acquired by the acquisition unit; a control unit that operates the robot based on the trajectory information; A robot system comprising:

7. In a robot system having a robot, a mobile body, and an information processing device, The moving body is an inertial sensor that detects attitude information of the moving body; an imaging means for capturing an image of a mark placed at a position whose relative positional relationship with a predetermined position of the robot is known, and detecting relative position information with respect to the robot; a communication unit capable of communicating with the information processing device; It has The information processing device includes: an acquisition unit that acquires the posture information and position information of the moving body detected at a plurality of positions that form a trajectory along which the robot moves, where the moving body is placed; a generation unit that generates trajectory information for teaching a motion of the robot by sequentially defining a position and a posture of the robot based on the position information and posture information at the plurality of positions acquired by the acquisition unit; a control unit that operates the robot based on the trajectory information; A robot system comprising:

8. The robot system according to any one of claims 4, 6, and 7, characterized in that the moving body or the information processing device is provided with an operation unit that instructs the detection of the position information, or the position information and the posture information.

9. A robot trajectory generation method for generating, using a moving object, position information at a plurality of positions forming a trajectory along which the robot will move, or trajectory information for teaching the robot movement, the trajectory information comprising the position information and orientation information, the method comprising: The moving body is moved to and placed at the plurality of positions in sequence, independently of the robot, to acquire the position information, or the position information and the posture information, at the plurality of positions; A robot trajectory generation method, characterized in that the trajectory information is generated by sequentially specifying the position, or the position and posture, of the robot based on the position information, or the position information and posture information, at the acquired multiple positions.

10. 10. The robot trajectory generation method according to claim 9, further comprising the steps of: moving the robot to at least one of the plurality of positions; and correcting the position information or the position information and the posture information at the at least one position, thereby correcting the trajectory information; or generating the trajectory information using the corrected position information and posture information.

11. 11. A robot control method, comprising: operating the robot based on the trajectory information generated by the robot trajectory generation method according to claim 9 or 10.

12. The processor that operates the robot acquires position information or the position information and posture information at a plurality of positions of a moving body that is moved sequentially and independently of the robot to a plurality of positions that form a trajectory along which the robot moves; A program characterized by generating trajectory information for teaching the robot's movements by sequentially specifying the position and posture of the robot based on the position information at the acquired multiple positions, or the position information and posture information.

13. a mobile object that moves sequentially and independently of a robot to a plurality of positions that form a trajectory along which the robot moves, in order to generate trajectory information for teaching an information processing device that operates the robot to move the robot; an inertial sensor that detects position information of the moving body at the plurality of positions, or the position information and attitude information; a communication unit capable of communicating with the information processing device; It has The mobile body is characterized in that the position information, or the position information and the attitude information, detected at the plurality of positions, are transmitted to the information processing device.

Images