A robot system including a robot having a display unit

Abstract

This robot system comprises: a display control unit which controls an image to be displayed in a display region of a display unit; and a reference direction setting unit which sets a reference direction for determining the orientation of an image. On the basis of the axial positions of respective joint shafts of the robot and the reference direction, the display control unit controls the orientation of an image shown on the display unit so as to perform display that represents a prescribed postural relation with respect to the reference direction.

Description

Technical Field

[0001] The present invention relates to a robot system including a robot having a display unit.

Background Art

[0002] A robot system includes a robot and a work tool attached to the robot. The robot can perform a predetermined operation while changing its position and posture. The robot is composed of a plurality of components such as an arm. In the prior art, a robot system in which a display device is attached to a component of the main body of the robot is known. For example, in Japanese Patent Publication No. 2018-529488, a display unit composed of a ring of LED (Light Emitting Diode) members or the like is arranged on a joint axis, and a holding device for medical use in which the display unit indicates a moving direction or the like is proposed.

[0003] Also, in the prior art, a camera that can acquire an image that is not tilted even when the camera captures an image in a tilted state is known (for example, Japanese Patent Application Laid-Open No. 4-81081).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] When the robot moves the work tool, its position and posture change. When a display unit is arranged on a component of the robot, when the robot is driven, the position and posture of the display unit change together with the component. As a result, the orientation of the image displayed on the display unit changes for the operator, and the image displayed on the display unit may be difficult to see.

Means for Solving the Problem

[0006] The Robot system first aspect of the present disclosure includes a robot including a plurality of joint axes and a display unit disposed on a main body portion of the robot. The robot system includes a display control unit that controls an image displayed in a display area of the display unit, and a reference direction setting unit that sets a reference direction which is a direction used as a reference when determining the orientation of the image displayed on the display unit. The robot system includes a coordinate system setting unit that sets a display coordinate system which is a coordinate system set for the display area of the display unit, a reference plane which is a reference plane of the display area of the display unit and is set on the display coordinate system, and a reference point which is a reference point for display on the display area of the display unit and is set on the display coordinate system. The robot system includes a coordinate system calculation unit that calculates the position and orientation of the display coordinate system based on the axis positions of the respective joint axes of the robot, and a display phase calculation unit that calculates a display phase direction, which is a direction serving as a reference for the orientation of the display on the display area of the display unit, by projecting a reference direction onto a reference plane in the display coordinate system. The display control unit The display phase direction calculated by the display phase calculation unit and controls and displays the orientation of the image on the display area of the display unit based on the reference point. The robot system according to the second aspect of the present disclosure includes a robot including a plurality of joint axes and a display unit disposed on a main body portion of the robot. The robot system includes a display control unit that controls an image displayed in the display area of the display unit, and a reference direction setting unit that sets a reference direction, which is a direction serving as a reference when determining the orientation of the image displayed in the display unit. The robot system includes a coordinate system setting unit that sets a display coordinate system, which is a coordinate system set for the display area of the display unit, a reference plane that is a reference plane of the display area of the display unit and is set on the display coordinate system, and a reference point that is a reference point for the display on the display area of the display unit and is set on the display coordinate system. The robot system includes a coordinate system calculation unit that calculates the position and orientation of the display coordinate system based on the axis positions of the respective joint axes of the robot. The robot system includes a display phase calculation unit that projects a reference direction onto the reference plane in the display coordinate system and further rotates it about the reference point by an offset angle, which is a predetermined angle, to calculate a display phase direction, which is a direction serving as a reference for the orientation of the display on the display area of the display unit. The display control unit controls and displays the orientation of the image on the display area of the display unit based on the display phase direction and the reference point calculated by the display phase calculation unit. The Third aspect robot system of the present disclosure includes a robot including a plurality of joint axes and a display unit disposed on a main body portion of the robot. The robot system includes a display control unit that controls an image displayed in a display area of the display unit, and a reference direction setting unit that sets a reference direction which is a direction used as a reference when determining the orientation of the image displayed on the display unit. The display control unit controls and displays the orientation of the image in the On the display area display unit so as to display a display having a predetermined posture relationship with respect to the reference direction based on the axial position of each joint axis of the robot and the reference direction. The reference direction setting unit sets the reference direction based on a position specified by an operator on the display area of the display unit.

Advantages of the Invention

[0007] According to the aspect of the present disclosure, it is possible to provide a robot system including a display unit attached to a main body portion of a robot, and in which an image displayed on the display unit is easy to view when the position and orientation of the robot change.

Brief Description of the Drawings

[0008]

FIG. 1

FIG. 2

FIG. 3

FIG. 4

FIG. 5

FIG. 6

FIG. 7

FIG. 8

FIG. 9

FIG. 10

FIG. 11

FIG. 12

FIG. 13

FIG. 14

FIG. 15

FIG. 16

FIG. 17

FIG. 18

FIG. 19

FIG. 20

FIG. 21

FIG. 22

FIG. 23

FIG. 24

FIG. 25

FIG. 26

FIG. 27

FIG. 28

FIG. 29

FIG. 30

FIG. 31

FIG. 32

FIG. 33

FIG. 34

FIG. 35

FIG. 36

FIG. 37

FIG. 38

FIG. 39

FIG. 40

Embodiments for Carrying Out the Invention

[0009] With reference to FIGS. 1 to 40, a robot system according to an embodiment will be described. The robot system of the present embodiment includes a robot including a plurality of joint axes and a display device that displays predetermined information. The display unit of the display device is disposed on the main body portion of the robot.

[0010] (Robot System) FIG. 1 shows a schematic diagram of a first robot system according to the present embodiment. The robot system 6 of the present embodiment has a function of transporting a workpiece. The robot system 6 includes a robot device including a hand 2 as a work tool (end effector) and a robot 1 that changes the position and posture of the hand 2. The robot system 6 includes a control device 4 that controls the robot 1 and the hand 2.

[0011] The robot 1 of this embodiment includes a base portion 14 fixed to the installation surface and a swivel base 13 supported by the base portion 14. The swivel base 13 is formed to rotate with respect to the base portion 14. The robot 1 includes an upper arm 11 and a lower arm 12. The lower arm 12 is rotatably supported by the swivel base 13 via a joint portion. The upper arm 11 is rotatably supported by the lower arm 12 via a joint portion. Further, the upper arm 11 rotates around a rotation axis parallel to the extending direction of the upper arm 11. The robot 1 includes a wrist 15 connected to the end of the upper arm 11. The wrist 15 is rotatably supported by the upper arm 11 via a joint portion. The wrist 15 includes a flange 16 that rotates around a rotation axis along the extending direction of the wrist 15. The hand 2 is fixed to the flange 16.

[0012] FIG. 2 shows a schematic diagram of the robot for explaining the joint axes of the robot of this embodiment. In the present invention, the joint axis is the axis of the joint portion connecting the links constituting the robot, and is the portion that changes the positional relationship or angular relationship between the links. By changing the axial position of each axis (the position in the case of a rotation axis is an angle, and the length of displacement in the case of a linear motion axis), the positional relationship between the links can be changed. As a result, at least one of the position and posture of the tip of the robot can be changed. Note that an actuator for moving the axial position of the joint axis may be installed at a location different from the portion that becomes the joint axis.

[0013] Referring to FIGS. 1 and 2, the robot 1 of this embodiment is composed of six joint axes. These six joint axes are, in order from the side of the pedestal 59 which is the base portion 14 of the robot 1, the first joint axis is the J1 axis 51, the second joint axis is the J2 axis 52, the third joint axis is the J3 axis 53, the fourth joint axis is the J4 axis 54, the fifth joint axis is the J5 axis 55, and the sixth joint axis is the J6 axis 56.

[0014] As shown to the right of FIG. 2, the J1 axis 51, J4 axis 54, and J6 axis 56 are configured as a rotation axis R1 that rotates around the link connecting the joint axes. Also, the J2 axis 52, J3 axis 53, and J5 axis 55 are configured as a rotation axis R2 that rotates around an axis orthogonal to the link connecting the axes.

[0015] The hand 2 of the present embodiment grips and releases the workpiece. The hand 2 grips the workpiece by closing the claw portions facing each other. The working tool is not limited to the hand that grips the workpiece. Any working tool can be attached to the robot according to the work performed by the robot system. For example, when the robot system performs arc welding, a welding torch can be attached to the robot.

[0016] FIG. 2 is a simple explanatory diagram for showing the configuration of the axes of the robot 1. Also, if the origin of the joint axis is set as the origin of the coordinate system (joint coordinate system) set for each axis and the point where the links are connected, the position of the origin of the axis is represented as a position on the coordinate system set in space. Hereinafter, the coordinate system used as a reference when representing the position and orientation of the robot, such as the joint axes, links, tip portions, and working tools attached to the tip portions of the robot set in space, is referred to as the reference coordinate system 87 of the robot. The reference coordinate system 87 is a coordinate system in which the position of the origin and the direction of the coordinate axes are fixed in the space where the robot is arranged.

[0017] In this embodiment, when the joint axis is a rotation axis, the position of the joint axis is the angle of the rotation axis. Moving the position of the joint axis means rotating the rotation axis to change the position. When the joint axis is a linear motion axis, the position of the joint axis is the position in the moving direction of the linear motion axis. Moving the position of the joint axis means moving the linear motion axis to change the position. When referring to the position of the origin of the joint axis, it represents the position of the origin of the coordinate system set for each axis on the coordinate system set for the space. The coordinate system set for the space is a coordinate system for representing at least one of the position and orientation of the tip (working tool) of the robot 1, or the flange 16 for attaching the tip to the robot 1, or the coordinate system set for each axis on the orthogonal coordinate system fixed to the space.

[0018] Also, in order to represent at least one of the position and orientation of the robot 1 on the reference coordinate system 87 of the robot set for the space, the coordinate system set for the robot device is defined as the tool coordinate system. The origin of the tool coordinate system, the point to be translated, and further, the center point when performing rotational movement are defined as the control points. In this embodiment, all six axes of the robot 1 are rotation axes, but the robot 1 may include a linear motion axis. Also, the robot 1 of this embodiment is a vertically articulated robot composed of six axes. However, any robot having other forms may be used as long as the position of each axis is controllable and the orthogonal position is controllable.

[0019] When the robot is mounted on a device that changes the position and orientation of the robot, such as a traveling axis or another driving device, the reference coordinate system of the robot may be set for the space so that the position and orientation of the robot or a part of the robot considering the driving of those driving devices are determined.

[0020] Fig. 3 shows a block diagram of the robot system according to the present embodiment. Referring to Figs. 1 to 3, the robot 1 includes a robot drive unit that changes the position and orientation of the robot 1. The robot drive unit includes a robot drive motor 19 as an actuator that drives a constituent member such as an arm. The robot drive unit drives an actuator arranged on a joint axis so as to move the rotational position at each joint axis of the robot 1.

[0021] Note that the robot drive unit may be configured by any principle or power as long as it can drive and displace the joint axis of the robot. Also, when the joint axis is a linear motion axis, the robot drive unit drives an actuator arranged on the joint axis so as to move the position on the linear motion axis. The hand 2 includes a hand drive unit that drives the hand 2. The hand drive unit includes a pressure pump, a valve, etc. for driving the claw portion of the hand 2.

[0022] The control device 4 includes an arithmetic processing unit (computer) having a CPU (Central Processing Unit) as a processor. The arithmetic processing unit has a RAM (Random Access Memory) and a ROM (Read Only Memory) connected to the CPU via a bus. The operation program 41 includes instruction statements for driving the robot 1 and the hand 2. The robot system 6 conveys a workpiece by being driven based on the operation program 41.

[0023] The control device 4 includes a storage unit 42 that stores information. The storage unit 42 stores information related to the control of the robot 1 and the hand 2. The operation program 41 is stored in the storage unit 42. The storage unit 42 can be configured by a non-temporary storage medium. For example, the storage unit 42 can be configured by a storage medium capable of storing information such as a volatile memory, a non-volatile memory, a magnetic storage medium, or an optical storage medium.

[0024] The control device 4 includes an operation command unit 43 that sends out operation commands. The operation command unit 43 corresponds to a processor that drives according to the operation program 41. The processor reads the operation program 41 and functions as the operation command unit 43 by implementing the control defined in the operation program 41. Based on the operation commands of the robot 1 output from the operation command unit 43, the robot drive unit is driven, and the rotational position of the robot drive motor 19 on the joint axis of the robot 1 is changed.

[0025] The operation command unit 43 sends operation commands for driving the robot 1 to the robot drive circuit 45. The robot drive circuit 45 includes an electric circuit for driving the robot drive motor 19. The robot drive circuit 45 supplies electricity to the robot drive motor 19 based on the operation commands. Also, the operation command unit 43 sends operation commands for driving the hand 2 to the hand drive circuit 44. The hand drive circuit 44 includes an electric circuit for driving the hand drive unit. The hand drive circuit 44 supplies electricity to the hand drive unit based on the operation commands. The control device 4 of the present embodiment is installed away from the robot 1, but is not limited to this form. The control device may be arranged inside the robot.

[0026] The robot 1 is provided with a rotational position detector 18 that outputs axis position data, which is the rotational position on the joint axis of the robot 1. The rotational position detector 18 is constituted by, for example, an encoder. The position and posture of the robot 1 can be detected by the output of the rotational position detector 18. The rotational position detector 18 of the present embodiment is attached to the robot drive motor 19 arranged corresponding to each joint axis.

[0027] The robot system 6 of this embodiment includes a display device that displays arbitrary information. The display device includes at least one display unit 60 disposed on the main body portion of the robot 1. In FIG. 1, as an example of the display unit 60, a display unit 61a is attached to a joint portion between the swivel base 13 and the lower arm 12. The display device includes a processing unit 21 that generates an image to be displayed on the display unit 60 and processes commands from the display unit 60. In this embodiment, the processing unit 21 is configured by the control device 4.

[0028] The processing unit 21 includes a reference direction setting unit 22 that sets a reference direction which is a direction serving as a reference when determining the orientation of the image displayed on the display unit 60. The processing unit 21 includes a coordinate system setting unit 23 that sets a display coordinate system which is a coordinate system set for the display area of the display unit 60.

[0029] The processing unit 21 includes a coordinate system calculation unit 24 that calculates the position and orientation of the display coordinate system based on the axis positions of the respective joint axes of the robot 1. The processing unit 21 includes a display phase calculation unit 25 that calculates a display phase direction which is a direction serving as a reference for the orientation of the display on the display area of the display unit 60. The processing unit 21 includes an offset angle setting unit 26 that sets an offset angle of the image displayed by the display unit. The processing unit 21 includes a command processing unit 29 that processes commands input by an operation on the display unit 60. The processing unit 21 includes a display control unit 27 that controls the image displayed in the display area of the display unit 60.

[0030] The processing unit 21 corresponds to a processor that is driven according to the operation program 41. The processor reads the operation program 41 and functions as the processing unit 21 by implementing the control defined in the operation program 41. Further, each unit of the reference direction setting unit 22, the coordinate system setting unit 23, the coordinate system calculation unit 24, the display phase calculation unit 25, the offset angle setting unit 26, the display control unit 27, and the command processing unit 29 included in the processing unit 21 corresponds to a processor that is driven according to the operation program 41. By the processor implementing the control defined in the operation program 41, each unit functions as such.

[0031] (Display unit) The robot system 6 of this embodiment includes one or more display units 60 on the main body part of the robot, such as links like the robot's arm, joint axes, and the tip of the arm. The display unit 60 is attached to the components of the robot whose position and orientation change due to the drive of the robot drive unit. The display unit 60 changes its position and orientation when the robot is driven by the joint axis. Also, there are cases where it is mounted on the drive device that changes the position and orientation of the robot. The present invention is also applicable when the position and orientation of the display unit 60 are changed by the drive device. The display unit 60 of the display device can adopt any display panel that can display characters, pictures, etc. so that they can be recognized. For example, as the display unit 60, a liquid crystal display panel or an organic EL (Electro Luminescence) display panel can be adopted.

[0032] The display unit 60 is configured to be appropriately attached to the installation part and is preferably thin. The display unit 60 preferably has a shape that does not protrude significantly from the main body part of the robot. Also, when the display unit 60 is arranged at the end of the robot, it preferably has a shape that is integrated with the main body part of the robot and has a smooth outer surface.

[0033] Fig. 4 shows a schematic diagram of a robot for explaining examples of various display units of this embodiment. The display units 61a, 61b, 61c, 62, 63a, 63b, 64, 65 are formed in a plate shape so as to have a planar display area. The display units 61a, 61b are fixed to the joint part so that the display area is perpendicular to the J2 axis 52 and the J3 axis 53. The display unit 61c is fixed to the joint part so that the display area is perpendicular to the J5 axis 55. The display units 61a, 61b, 61c move together with the joint part.

[0034] The display unit 62 is fixed to the swivel base 13 such that the display area is parallel to the J1 axis. The display unit 62 rotates together with the swivel base 13. The display units 63a and 63b are fixed to the upper arm 11 such that the display area is parallel to the J4 axis 54. The display units 63a and 63b move together with the upper arm 11. The display unit 64 is arranged such that the display area is parallel to the J5 axis 55. The display unit 65 is fixed to the list 15 such that the display area is parallel to the J6 axis 56. The display units 64 and 65 move together with the list 15. Thus, the display units 61a, 61b, 61c, 62, 63a, 63b, 64, and 65 are fixed to the components of the robot 1 and move together with the components of the robot 1. Also, each display unit rotates about the joint axis of the robot 1 as the rotation axis.

[0035] Fig. 5 shows a schematic diagram of another robot for explaining examples of various display units according to the present embodiment. The display units 66 and 67 are constituted by flexible display panels. The display units 66 and 67 are formed such that the display area extends in the circumferential direction along the surface of the component of the robot 1. The display units 66 and 67 can be constituted by a thin and flexible display panel such as an organic EL display panel. The display unit 66 is fixed so as to be wound around the lower arm 12. The display unit 66 is formed such that the display area is parallel to the extending direction of the lower arm 12. The display unit 67 is fixed so as to be wound around the upper arm 11. The display unit 67 is formed such that the display area is parallel to the extending direction of the upper arm 11. The display units 66 and 67 are fixed to the components of the robot 1 and move together with the components of the robot 1.

[0036] The display units 66 and 67 according to the present embodiment are constituted by one display panel, but the present invention is not limited to this form. A plurality of display panels may be arranged around the component of the robot 1 to constitute a display unit so as to surround the component. One image may be displayed on a plurality of display panels.

[0037] FIG. 6 shows a first image displayed on the display unit. FIG. 7 shows a second image displayed on the display unit. In the following embodiments, among various display units, the display unit 61a disposed between the swivel base 13 and the lower arm 12 will be taken as an example for explanation. Display control unit 27 Based on data transmitted from another device, an image can be displayed in the display area of the display unit 61a. Information to be displayed on the display unit includes, for example, the operating state of the robot, the name of the operating program being executed, the execution status of the operating program, the alarm status, the driving speed of the robot, the number of executions of the executed work, and the number of times of failed retrievals. Here, the driving speed or operating speed of the robot may be the translational speed or rotational speed such as the tip of the robot or the tool coordinate system. Furthermore, the driving speed or operating speed of the robot may be the rotational speed of each joint axis, etc. The same shall apply in the following description unless otherwise specified.

[0038] In addition, information to be displayed on the display unit includes the cause and countermeasure method when a problem occurs, the content of the next work to be executed, the connection status of the network to which the robot is connected, the load status of the actuator included in the robot drive unit that drives the robot, the current time, date and time, and the elapsed time of the work. As information to be displayed on the display unit, it is preferably useful information for the operator by being displayed on the main body part of the robot, such as information related to the work or setting of the robot, or any information transmitted from an external device or the like to the robot.

[0039] In the first image 71a, information (State) indicating whether the robot drive motor is driving, the name (Prog.) of the operating program being executed, the driving speed (Speed) around the joint axis, the number of executions (Count) of the program, time, and date are displayed. Information indicating that the robot is being driven is also displayed in the second image 71b.

[0040] FIG. 8 shows a third image displayed on the display unit. The display unit may have not only a display function for displaying information but also an input function for inputting predetermined information by an operator's operation. The display device can display an input image for inputting predetermined information on the display unit. The third image 72a is an input image for setting the drive speed around the joint axis where the display unit is disposed or a desired joint axis. Note that there may be a case where another joint axis other than one joint axis is driven together with one joint axis. In this case, when changing the drive speed around one joint axis, the drive speeds of all the joint axes may be adjusted so that the position and orientation of the tip of the robot do not change. Further, in the input image displayed on the display unit, the moving speed of the tip of the robot in the running operation program may be changed.

[0041] The display device may function as an input device for inputting information by an operator's operation according to the input image. Further, the display device can function as a teaching device for manually driving the robot or creating an operation program. For example, the display device can include a touch panel type display panel capable of input operations as the display unit. Alternatively, the display device may include a button for emergency stop or performing a desired function.

[0042] As described above, the display device in the present embodiment includes both a device having only a display function and a device having both a display function and an input function. Referring to FIG. 3, a command input to the display unit 60 of the display device is sent to the command processing unit 29. The command processing unit 29 processes the input command. For example, the command processing unit 29 sends a robot operation command to the operation command unit 43 or changes set values stored in the storage unit 42 for driving the robot.

[0043] The display control unit 27 can display any input image in the display area. For example, it can display an image of a software keyboard, an image with only numbers, or an image for changing a numerical value by moving a bar-shaped image such as a slide bar. Further, the display control unit 27 can display an image for continuously performing an input operation by rotating a predetermined image, an image to be rotated like a dial of an analog phone, or buttons related to screen changes.

[0044] As an information input method, any input method can be adopted, such as input by contacting the display area like a touch panel, or input non - contact by an electrostatic capacitance type sensor. When performing non - contact input, information can be input by an operation in the space above the display area. As a device for detecting an input operation, a device using any sensing method such as an electrostatic capacitance type sensor, an optical sensor, an optical camera, an infrared camera, or an ultrasonic sensor can be used.

[0045] Furthermore, a contact - type or non - contact type sensor can be operated as a contact sensor for detecting contact with a person or surrounding objects in addition to an input device for information input. That is, by switching the operation mode, the sensor for detecting an input operation can be used as a sensor for ensuring safety.

[0046] The information input to the display device is transmitted to another device such as a control device of a robot or a device connected by a network by wired or wireless communication. Or, the input information may be stored and remembered in a teaching device including a display function. Information calculated or processed based on the input information may be displayed on the display unit of the display device.

[0047] The input image, which is an image for an operator to input information, is preferably configured to allow input of any information for settings or confirmation. For example, the input image is preferably configured to allow input of changes in set values, changes in the driving speed of the robot, instructions to make the robot wait, changes in the operation of the robot, driving of the robot, or interruption of the operation. Further, the input image may be an image for performing any input such as input of information such as tools or workpieces for replacing tools attached to the tip of the robot, setting of the coordinate system, position teaching, change to the direct teach mode, or change to the automatic operation mode. As a function that can operate the set value, it is preferable to limit it to functions with high usage frequency or simple functions. Or, the input image may be formed so as to be able to change the type of input function.

[0048] Also, when inputting information to the display device, the display unit may display an image that accepts input of specific information for enabling the input operation of the operator. The display unit may switch to an image that enables the input operation of the operator when the operator inputs specific information. For example, it may be possible to press a button that enables input for a predetermined time or more, input a password, or move a finger that touches so as to pass through a predetermined point on the display area. In this way, the display control unit may display an image that prohibits the input operation in order to prevent incorrect input operations and the like. And, an image that enables the input operation may be displayed by an operation such as input of a password by the operator.

[0049] When inputting information in the display device, at least one of the authority for enabling the input operation and the authority for the information to be displayed may be changed by a predetermined input operation. For example, by performing a predetermined input operation, it is possible to change the items that can be set, change the operation of the robot, change the range of values that can be set, or change the information to be displayed. In this way, the display device may change the inputtable instructions or information, etc., or change the displayable data or information.

[0050] The display unit may be formed to be detachable from the main body of the robot. In particular, the display device constituting the display device and the teaching device having the display unit may be formed to be detachable from the main body of the robot. In this case, the display device can be provided with a battery. Or, it is preferable that power is supplied when the display device is attached to the main body of the robot. Also, a device for transmitting information to be displayed on the display device or a device for transmitting information input to the display device can be formed to be connected by wire or wirelessly.

[0051] Also, the display coordinate system set for the display unit can be configured such that a coordinate system generated in advance according to the attachment position is applied by attaching the display unit to a predetermined position of the main body of the robot. Or, the display coordinate system may be directly input after attaching the display unit, or may be set by a conventional method.

[0052] When a display device detachable from the robot functions as a teaching device, it can be used limitedly when a teaching device is required. Or, when a plurality of display units are provided, the teaching device can be installed and used limitedly to the necessary parts of the robot. For this purpose, while improving the operability of the robot, the cost of the robot system can be suppressed, or the risk that the teaching device is broken by a collision with a peripheral device during the operation of the robot can be reduced.

[0053] (Basic control for displaying an image) FIG. 9 shows a flowchart of control for displaying an image on the display unit in the present embodiment. Referring to FIG. 3, before the display control unit 27 displays an image, the coordinate system setting unit 23 sets a display coordinate system corresponding to the display unit, a reference plane and a reference point set in the display coordinate system. The coordinate system setting unit 23 sets the display coordinate system, the reference plane and the reference point according to, for example, the operation of the operator. Further, the reference direction setting unit 22 sets a reference direction, which is a direction serving as a reference for display, by predetermined control. The storage unit 42 stores the display coordinate system, the reference plane, the reference point, and the reference direction.

[0054] Referring to FIGS. 3 and 9, in step 111, the processing unit 21 acquires the display coordinate system, the reference plane, and the reference point from the storage unit 42. In step 112, the processing unit 21 acquires the reference direction from the storage unit 42.

[0055] In step 113, the coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system based on the position and orientation of the robot 1. Specifically, the coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system based on the output of the rotational position detector 18. The position and orientation of the reference plane expressed in the display coordinate system and the position of the reference point are calculated.

[0056] In step 114, the display phase calculation unit 25 calculates the display phase direction in the reference plane based on the reference direction and the reference point. That is, the display phase calculation unit 25 sets the orientation of the image in the display unit. In step 115, the display control unit 27 generates an image in the reference plane based on the display phase direction. Then, in step 116, the display control unit 27 displays an image on the display unit 60 based on the image in the reference plane. Next, these controls will be described in detail.

[0057] (Coordinate System Setting Unit) Fig. 10 shows a schematic plan view for explaining a display coordinate system, a reference plane, a reference point, and a display phase direction. Fig. 10 corresponds to a plan view of the display area of the display unit 61a. The coordinate system setting unit 23 of the processing unit 21 sets a display coordinate system 88, which is a coordinate system set for the display area of the display unit. The coordinate system setting unit 23 sets, for the display area of the display unit, a reference plane 82, which is a reference plane of the display area of the display unit and is set on the display coordinate system, and a reference point 83, which is a reference point for display on the display area of the display unit and is set on the display coordinate system.

[0058] The display coordinate system can be set at an arbitrary position and orientation with respect to the display area of the display unit. The display coordinate system represents the position and orientation on a coordinate system such as the reference coordinate system of the robot for the display unit or the display area installed on the main body portion of the robot. In the display unit 61a, the display coordinate system is set such that the plane composed of two coordinate axes and the display area are in the same plane.

[0059] The reference plane is a plane used as a reference when representing an image output to the display area of the display unit on a plane. Also, the reference plane is a plane set on the display coordinate system. The position and orientation of the reference plane in the display coordinate system can be determined in advance. In the display unit 61a, the reference plane is set in a plane including two coordinate axes of the display coordinate system. The reference point is a point that determines the center position of the image output to the display area of the display unit. The orientation of the image is determined by rotating the image with the reference point as the center of rotation in the reference plane.

[0060] The reference direction can be set in an arbitrary direction independently of the display coordinate system. The reference point is set on the reference plane. The reference plane can be set independently of the directions of the coordinate axes of the display coordinate system. For example, the reference plane may be inclined with respect to the XY plane of the display coordinate system. However, when the display area of the display unit is planar, it is preferable that the display area, the reference plane, and the plane composed of two coordinate axes of the display coordinate system are in the same plane. By setting in this way, the amount of calculation can be reduced.

[0061] When the display area of the display unit is not planar but curved, a reference point can be set on the display area, and the tangent plane at the reference point can be used as the reference plane. In this case, even if the XY plane in the display coordinate system and the reference plane are in the same plane, the reference plane does not include the display area.

[0062] By determining the display phase direction on the reference plane by the display phase calculation unit 25, the display control unit 27 can generate an image to be displayed on the reference plane. Next, the display control unit 27 can generate an image to be displayed on the display unit by projecting the image on the reference plane onto the display area of the display unit.

[0063] The coordinate system setting unit 23 can set the display coordinate system 88, the reference plane 82, and the reference point 83 by arbitrary control. For example, an operator can determine in advance the display coordinate system 88, the reference plane 82, and the reference point 83 based on the shape of the display area of the display unit and input them to the processing unit 21. Basically, these setting values are determined for the display unit and can be determined in advance as standard setting values before attaching the display unit to the robot. The operator can change the setting values as needed. For example, after setting the standard value for the display unit, the operator can change the standard value of the display so as to be convenient for the work according to the state of the robot to be displayed on the display unit.

[0064] Also, the coordinate system setting unit 23 may automatically set the display coordinate system 88, the reference plane 82, and the reference point 83 for the display unit. For example, there are cases where the display unit has a wide display area. As shown in the display units 66, 67 of FIG. 5, there are cases where a flexible display unit is wound around a component of the robot. In such a case, the coordinate system setting unit 23 may automatically set the display coordinate system, the reference plane, and the reference point so as to display an image in the same direction regardless of the rotational position of the component. For example, the coordinate system setting unit 23 may automatically set by moving the setting value to be displayed around the joint axis.

[0065] Also, when forming the display units 66 and 67, there may be a case where a plurality of display units are combined and arranged to surround the constituent members of the robot. In this case, a display coordinate system, a reference plane, and a reference point may be set for each display area of one display unit. Alternatively, a plurality of display units may be combined and configured as one display unit at a predetermined part of the robot, and a display coordinate system, a reference plane, and a reference point may be set. In this case, control may be performed so that images are distributed and displayed on the respective display units.

[0066] The display control unit 27 controls and displays the position and orientation (phase) of the display in the display area of the display unit based on the reference point 83 which is the reference point of the display on the display area of the display unit, and the display phase direction (the direction indicated by the arrow 84) which is calculated by the display phase calculation unit 25 and serves as the reference for the orientation of the display on the display area of the display unit. For example, the display control unit 27 can display an image so that the direction indicated by the arrow 84 is the upper side of the image.

[0067] In the present embodiment, the display control unit 27 determines the center position of the display on the display area of the display unit by the reference point 83. The display control unit 27 determines the orientation of the display by rotating the image around the reference point 83 in the reference plane 82. In the present embodiment, the origin of the display coordinate system 88 and the reference point are arranged at the same position, but they may be arranged at different positions.

[0068] The display phase direction indicated by the arrow 84 exists on the reference plane 82 and is set to pass through the reference point 83. When the display area of the display unit is planar, it is preferable that the reference plane 82 exists parallel to the display area or on the same plane as the display area. In the present embodiment, the display area of the display unit is planar, but it may have a shape such as a curved surface instead of a planar shape.

[0069] When the display area of the display unit is curved, the tangent plane at a predetermined point on the display area can be used as the reference plane. By setting a predetermined point on the display area as the origin of the display coordinate system, the reference plane can be made into a plane constituted by two coordinate axes of the display coordinate system. The image displayed on the display area of the display unit can be displayed based on the reference point and the display phase direction, or can be displayed by projecting the image on the reference plane onto the display area. The image displayed on the display area may be displayed by any method, such as being displayed based on the calculated information.

[0070] Note that although the display area of the display unit 61a (the part that displays the image of the display panel) is circular, it may have any shape such as a regular polygon. In this case, the reference point can be arranged at the center of the display area. For example, the reference point can be set at the position of the centroid of the shape of the display area.

[0071] (Coordinate system calculation unit) The coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system 88 based on the axial positions of the respective joint axes of the robot 1. The coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system 88 on a coordinate system serving as a reference such as the reference coordinate system 87 based on the axial positions (angles in the case of a rotation axis) of the respective joint axes of the robot 1, the relationship between the positions and orientations of the joint axes represented by DH (Denavit Hartenberg) parameters, etc., and the position and orientation of the coordinate system of the joint axis (joint coordinate system).

[0072] Here, the relationship between the joint coordinate system and the position and orientation of the display unit (the position and orientation of the display coordinate system 88) is defined. In particular, it is preferable to select a joint coordinate system whose relationship with the position and orientation of the display unit is fixed, and to calculate in advance the relationship between the position and orientation of the display coordinate system and the joint coordinate system. In the case of this embodiment, for the joint axes connected in sequence via the links, it is preferable to set in advance the relationship between the position and orientation and the joint coordinate system set for the nearest joint axis from the display unit toward the pedestal 59 side of the robot.

[0073] Note that, for any form of robot, it is possible to calculate the position and orientation of the display coordinate system on the reference coordinate system by a method widely recognized by those skilled in the art. For example, up to the joint axis where the relationship between the position and orientation with respect to the display unit is fixed, in the reference coordinate system Homogeneous A transformation matrix (relationship between position and orientation) is determined. From the homogeneous transformation matrix between the joint axis and the display unit, it is possible to obtain the position and orientation of the display coordinate system on the reference coordinate system.

[0074] When the robot is mounted on a device that changes the position and orientation of the robot, such as a travel axis or another drive device, the position and orientation of the display coordinate system may be calculated taking into account the amount of movement of those drive parts. Also, in some cases, the relative position and orientation relationship with respect to a driving peripheral device installed separately from the robot may be taken as the position and orientation of the display coordinate system. In this case, considering the changes in the position and orientation of all the driving parts up to the display coordinate system, including the driving device installed separately from the robot, the relative position and orientation of the display coordinate system may be calculated.

[0075] (Reference direction setting unit) The reference direction setting unit 22 sets a reference direction which is the direction used as a reference when determining the orientation of the image displayed on the display unit. The reference direction setting unit 22 sets the direction on the reference coordinate system 87 of the robot 1 as the reference direction. Also, the reference direction may be the direction on a convenient coordinate system for setting the reference direction. Further, the reference direction setting unit 22 may set the reference direction using a coordinate system set to be fixed with respect to the space where the robot system exists.

[0076] The reference direction setting unit 22 may be set for the robot 1, and the reference direction may be set according to the direction on the reference coordinate system 87 of the robot using the reference coordinate system 87 for representing the position and orientation of the robot, such as each joint part or the tip of the arm of the robot 1. Further, the reference direction setting unit 22 may set the reference direction using the coordinate system set for a predetermined part of the robot 1. In this case, the reference direction changes as the predetermined part of the robot is driven or moved.

[0077] Alternatively, the reference direction setting unit 22 may set the reference direction using the reference coordinate system of the devices around the robot, such as devices, jigs, other robots, or other driving devices arranged near the robot 1. At this time, the reference direction setting unit 22 may set the reference direction for the part whose position and orientation change due to the driving or movement of the surrounding devices in the surrounding devices. In this case, the reference direction changes as the surrounding devices are driven or moved.

[0078] By setting the reference direction by the reference direction setting unit in a desired coordinate system, it is possible to set the direction in which the image displayed on the display unit is easy for the operator to view. Also, in the input image for performing the input operation, it is possible to set the direction in which the input is easy. In addition, various implementation methods are conceivable for the method of setting the reference direction, and examples will be shown below.

[0079] FIG. 11 shows a perspective view of a reference plane for explaining the first control in which the reference direction setting unit sets the reference direction. The reference direction setting unit 22 may set the reference direction so as to form a predetermined posture relationship with respect to the normal direction at the reference point 83 of the reference plane 82 set for the display area in the reference coordinate system 87 of the robot. For example, the reference direction setting unit 22 may use the direction obtained by a predetermined homogeneous transformation matrix with respect to the normal direction as the reference direction. In the example here, the reference plane 82 and the display area of the display unit are in the same plane. For this reason, the reference direction setting unit 22 may set the reference direction on the reference coordinate system 87 of the robot so as to form a predetermined posture relationship with respect to the normal direction of the display area.

[0080] That is, as shown by arrow 82a, the reference direction setting unit 22 calculates the normal direction of the reference plane 82 so as to pass through the reference point 83. The reference direction setting unit 22 tilts the normal direction by the direction and angle of arrow 91 expressed in the reference coordinate system 87. The direction shown by arrow 81 becomes a direction parallel to the reference direction. Then, the reference direction setting unit 22 sets the direction parallel to the direction shown by arrow 81 as the reference direction shown by arrow 80.

[0081] By determining the reference direction in this way, even when the positions of the respective axes of the joint axes of the robot 1 change due to the drive of the robot 1, the reference direction can be stably set. For example, as will be described later, a predetermined direction in the reference coordinate system of the robot such as the direction of gravity can be set as the reference direction. In this case, there is a problem that the display phase direction cannot be determined when the normal direction of the display area of the display unit becomes parallel to the reference direction. However, in the control of setting the reference direction from the normal direction of the reference plane, there is no such problem, and the reference direction can be stably set.

[0082] Alternatively, during the period when the control of setting a predetermined direction in the reference coordinate system of the robot as the reference direction is being performed, the posture relationship between the display unit and the reference direction may change. In this case, the reference plane and the reference direction on the display area of the display unit may be orthogonal or in a posture relationship close to orthogonal, and depending on the posture relationship, the change in the display phase direction may become large or large changes in the display phase direction may occur frequently. As a result, the calculation of the display phase direction may become unstable. At this time, it may be switched to the control of setting the reference direction from the normal direction of the reference plane.

[0083] When the robot is driven, an image is displayed in a direction that maintains a predetermined posture relationship in the reference coordinate system of the robot with respect to the display area of the display unit, so that the image displayed by the display unit can be made easier for the operator to see. Alternatively, an input image that facilitates the operator's input operation can be displayed.

[0084] Fig. 12 shows a schematic plan view for explaining a second control in which a reference direction setting unit sets a reference direction. The reference direction setting unit 22 may set a reference direction according to the range of the position where the display unit exists and the range of the posture of the display unit on a coordinate system serving as a reference such as the reference coordinate system 87 of the robot. As the position where the display unit exists, the position where the origin or reference point of the display coordinate system exists can be adopted. As the posture of the display unit, the posture of the display coordinate system can be adopted. For example, when the position of the display unit changes with the driving of the robot, the reference direction may be changed according to the position where the display unit exists on the reference coordinate system.

[0085] When setting the reference direction according to the position where the display unit exists, the range may be defined by two coordinate axes of the reference coordinate system of the robot. Also, the range for determining the position where the display unit exists may be defined by one or three coordinate axes of the reference coordinate system of the robot.

[0086] In the example shown in Fig. 12, four ranges 101a, 101b, 101c, 101d are defined based on two coordinate axes (X-axis and Y-axis) of the reference coordinate system 87 of the robot. When the display unit is arranged within the range 101a, the reference direction setting unit 22 can set the reference direction indicated by the arrow 80a. When the display unit is arranged within the range 101b, the reference direction setting unit 22 can set the reference direction indicated by the arrow 80b. Similarly to this control, when the display unit is arranged within the ranges 101c, 101d, the reference direction setting unit 22 can set the reference directions indicated by the respective arrows 80c, 80d. The reference directions in the respective ranges 101a, 101b, 101c, 101d can be set in the reference coordinate system 87.

[0087] By setting the reference direction in this way, when the robot is driven, according to the range or posture range of the position where the display unit exists, the image displayed on the display unit can be made easier for the operator to view. Also, it becomes easier for the operator to input in the input image. For example, when multiple operators are working around the robot, by changing the reference direction according to the working range of the operators, the display becomes easier to view for each operator.

[0088] FIG. 13 shows a first schematic diagram of a robot for explaining a third control in which the reference direction setting unit sets the reference direction. FIG. 14 shows a second schematic diagram of a robot for explaining a third control in which the reference direction setting unit sets the reference direction. FIG. 15 shows a third schematic diagram of a robot for explaining a third control in which the reference direction setting unit sets the reference direction. In FIGS. 13 to 15, in addition to the schematic diagram of the robot, an enlarged view of the image 71a displayed on the display unit 61a is shown.

[0089] In FIG. 13, the robot 1 is disposed above the installation surface 105. In FIG. 14, the robot 1 is disposed below the installation surface 105. In FIG. 14, the robot 1 is suspended. In FIG. 15, the installation surface 105 is inclined. In any of the robots 1, the base portion 14 is fixed to the installation surface 105. The arrow 90 indicates the direction of gravity (downward in the vertical direction).

[0090] The reference direction setting unit 22 may set the reference direction in the direction of gravity, the reverse direction of the direction of gravity, or the direction obtained by rotating the direction of gravity by a predetermined posture. That is, the reference direction setting unit 22 may set the direction defined in the reference coordinate system 87 as the reference direction. In the example shown in FIGS. 13 to 15, the reference direction is set in the upward vertical direction (opposite to the direction shown by the arrow 90). Each display unit 61a is arranged in various directions. However, in any direction of the display unit 61a, the display control unit 27 displays the image 71a such that the upward vertical direction is the upper side of the image.

[0091] The third control is effective when the normal direction of the display area (reference plane) of the display unit forms an angle greater than or equal to a certain degree with the direction of gravity. Since the reference direction determined by this control is based on the direction of gravity, it is an intuitive and easy-to-understand direction for the operator. For example, the display device may be controlled to display an image while maintaining a predetermined angle of 0 degrees or more with respect to the horizontal direction regardless of the angle of the joint axis of the robot provided with the display unit. Further, when the display unit is configured by a touch panel for teaching operations, the input image such as a button to be displayed may also be controlled to be displayed while maintaining a predetermined angle with respect to the horizontal direction.

[0092] By setting the reference direction based on the direction of gravity in this way, the image displayed on the display unit becomes easier to view for the operator regardless of the installation state and posture of the robot. Also, it becomes easier for the operator to input in the input image.

[0093] FIG. 16 shows a perspective view of a reference plane for explaining a fourth control in which the reference direction setting unit sets the reference direction. The angle formed by the reference direction and the normal direction of the reference plane may change due to the driving of the robot, and the angle formed by the reference direction and the normal direction of the reference plane may become smaller than a predetermined threshold value. For example, when the position and posture of the robot 1 change, the angle θ1 formed by the reference direction indicated by the arrow 80 and the normal direction of the reference plane 82 indicated by the arrow 82a may become smaller than a predetermined threshold value.

[0094] In this case, the reference direction setting unit 22 may not change the reference direction before the angle θ1 becomes smaller than the predetermined threshold value. That is, when the position and posture of the robot change and the angle θ1 becomes smaller than the predetermined threshold value, the reference direction setting unit 22 may maintain the previous reference direction.

[0095] When the relationship between the display area of the display unit and the reference direction changes due to the driving of the robot, and the reference direction approaches the normal direction of the reference plane 82, there may be a case where the orientation (display phase direction) of the image on the display unit cannot be stably calculated. In such a situation, by continuing without changing the reference direction, the orientation of the image can be determined and the image can be displayed.

[0096] FIG. 17 shows a perspective view of a reference plane for explaining a fifth control in which the reference direction setting unit sets a reference direction. The reference direction setting unit 22 may set the reference direction based on the position specified by the operator on the display area of the display unit. In the example here, the reference plane 82 is set in the same plane as the display area of the display unit.

[0097] The operator changes the position and orientation of the robot 1 so that the reference plane 82 is in a desired orientation. The display control unit 27 displays a reference point 83 on the display area of the display unit. When the display unit is configured with a touch panel type display panel, the operator designates a designated point 102 by pressing a desired point in the display area of the display unit. The reference direction setting unit 22 calculates the direction from the reference point 83 shown by the arrow 81 to the designated point 102. The reference direction setting unit 22 sets the direction parallel to the arrow 81 as the reference direction shown by the arrow 80. The reference direction can be set in the reference coordinate system of the robot.

[0098] In the fifth control, the reference direction is set in a direction within the plane of the display area of the display unit. Here, when the desired reference direction is a direction intersecting the plane of the display area of the display unit, the operator can change the position and orientation of the robot to change the orientation of the display area of the display unit.

[0099] In the fifth control, the reference direction can be easily set so that the orientation of the image is adjusted to a convenient direction for the operator according to the state of the robot. By this control, the operator can easily set a reference direction that makes it easier to view the display on the display unit. Also, it becomes easier for the operator to input in the input image.

[0100] Note that depending on the shape of the display area and the like, there may be a case where the display area and the reference plane are not in the same plane. For example, the display area of the display unit may be formed in a curved surface shape. In this case, the reference direction setting unit can project the designated point specified by the operator onto the reference plane in a predetermined direction. Then, the reference direction may be set based on the point projected onto the reference plane.

[0101] In the above embodiment, the direction from the reference point to the designated point is set as the reference direction, but it is not limited to this form. Any direction based on the designated point specified by the operator on the display area can be set as the reference direction. For example, in the first control for setting the reference direction, when setting the posture of the reference direction with respect to the normal direction of the reference plane, control for designating a designated point on the display area may be performed.

[0102] (Display phase calculation unit) FIG. 18 shows a perspective view of a reference plane for explaining the first control for calculating the display phase direction. The display phase calculation unit 25 projects the reference direction indicated by the arrow 80 onto the reference plane 82, which is the reference plane of the display area of the display unit, thereby calculating the display phase direction (the direction indicated by the arrow 84), which is the reference direction of the display orientation on the display area of the display unit. FIG. 18 shows a case where the reference direction indicated by the arrow 80 is orthogonal to the normal direction of the reference plane 82 indicated by the arrow 82a. In this case, the display phase calculation unit 25 can project the reference direction in an arbitrary direction as shown by the arrow 92. The display phase calculation unit 25 projects the reference direction toward the reference plane 82. Then, the display phase direction indicated by the arrow 84 can be determined so as to pass through the reference point 83.

[0103] FIG. 19 shows a perspective view of a reference plane for explaining the second control for calculating the display phase direction. FIG. 19 shows a case where the reference direction indicated by the arrow 80 is inclined without being orthogonal to the normal direction of the reference plane 82 indicated by the arrow 82a. The display phase calculation unit 25 calculates a direction parallel to the reference direction passing through the reference point 83 as shown by the arrow 81 based on the reference direction indicated by the arrow 80.

[0104] The display phase calculation unit 25 projects the direction parallel to the reference direction passing through the reference point 83 onto the normal direction of the display area as shown by the arrow 92. The display phase calculation unit 25 projects the direction parallel to the reference direction passing through the reference point 83 toward the reference plane 82. By this control, as shown by the arrow 84, the display phase direction serving as the reference for the phase of the display on the display coordinate system can be determined.

[0105] Alternatively, the display phase calculation unit 25 calculates the direction obtained by projecting the reference direction shown by the arrow 80 onto the reference plane 82 toward the normal direction of the display area. Then, the display phase calculation unit 25 calculates the direction passing through the reference point 83 so as to be parallel to the projected direction. The display phase calculation unit 25 can determine this direction as the display phase direction.

[0106] Fig. 20 shows a schematic diagram of a robot for explaining the third control for calculating the display phase direction. Fig. 21 shows a plan view of the reference plane for explaining the third control for calculating the display phase direction. Referring to Fig. 3, Fig. 20, and Fig. 21, the display phase calculation unit 25 projects the reference direction onto the reference plane 82, which is the reference plane of the display area of the display unit 61a. Further, the display phase calculation unit 25 may calculate the display phase direction by rotating at an offset angle, which is a predetermined angle, about the reference point 83, which is the reference point for the display on the display area of the display unit.

[0107] Here, the reference direction is defined as the upward vertical direction as shown by the arrow 80. The display phase calculation unit 25 calculates the direction parallel to the reference direction in the reference plane 82 by projecting the reference direction onto the reference plane 82. The display phase calculation unit 25 calculates the direction parallel to the reference direction passing through the reference point 83 as shown by the arrow 81.

[0108] The processing unit 21 of the present embodiment includes an offset angle setting unit 26 that sets an offset angle, which is a predetermined angle centered on a reference point on the display area. The offset angle setting unit 26 sets the offset angle according to the operation of the operator. For example, the operator inputs the offset angle θ2 using the teaching operation panel. Then, the offset angle setting unit 26 can set the offset angle θ2 with respect to the direction indicated by the arrow 81 obtained by projecting the reference direction onto the reference plane.

[0109] Next, the display phase calculation unit 25 rotates the direction indicated by the arrow 81 by an offset angle θ2, which is a predetermined rotation angle, around the reference point 83. The display phase calculation unit 25 rotates the direction obtained by projecting the reference direction within the reference plane 82. Then, the display phase calculation unit 25 calculates the display phase direction indicated by the arrow 84. The display control unit 27 controls and displays the orientation of the image on the display area of the display unit 61a based on the display phase direction calculated by the display phase calculation unit 25 and the reference point 83.

[0110] As shown in FIG. 20, the image 71a displayed on the display unit 61a is inclined by an offset angle θ2 with respect to the direction obtained by projecting the reference direction onto the reference plane. By this control, the orientation of the image displayed in the display area of the display unit can be easily offset and displayed for the operator. The image can be displayed so as to have a predetermined phase relationship with respect to a predetermined direction based on the reference direction. When the operator views the display unit from a specific direction, information is displayed according to the specific direction, making the information easier to see. Also, when the display device has an input function, it becomes easier for the operator to perform input operations on the input image.

[0111] FIG. 22 shows a plan view of the reference plane for explaining the fourth control for setting the display phase direction. In the example here, the reference plane 82 and the display area of the display unit are arranged in the same plane. The offset angle setting unit 26 is configured to be able to set the offset angle θ2 by operating the display area of the display unit by the operator.

[0112] The display control unit 27 displays the direction obtained by projecting the current reference direction onto the reference plane 82, as indicated by arrow 81. The operator designates the designated point 103 by pressing the display area of the display unit. The offset angle setting unit 26 calculates an offset angle θ2 with respect to the direction indicated by arrow 81. The display phase calculation unit 25 sets the display phase direction based on the offset angle θ2. The display control unit 27 controls and displays the orientation of the image on the display area of the display unit 61a based on the display phase direction calculated by the display phase calculation unit 25 and the reference point 83.

[0113] In the case where the display area of the display unit is curved, the offset angle may be set based on the point obtained by projecting the point where the operator touched the display area onto the reference plane. For example, the offset angle may be calculated based on the point obtained by projecting the designated point designated by the operator in the normal direction of the reference plane. Alternatively, the control to rotate the image based on such an offset angle may not be performed.

[0114] (Display control unit) The display control unit 27 controls the image displayed in the display area of the display unit. Also, when the display unit displays an input image, the display control unit 27 controls and displays the orientation of the input image. The display control unit 27 controls and displays the orientation of the image on the display area of the display unit so as to display an image having a predetermined posture relationship with respect to the reference direction based on the axis positions of the respective axes of the joint axes of the robot and the reference direction.

[0115] Also, the display control unit 27 controls and displays the position and orientation of the image on the display area of the display unit based on the axis positions of the respective axes of the joint axes of the robot, the reference direction, the reference point in the display coordinate system, and the display phase direction in the display coordinate system calculated by the display phase calculation unit 25.

[0116] Note that it may be mounted on a device that changes the position and orientation of a robot, such as a device having a traveling axis or another driving device. In this case, it is preferable that the processing unit 21 displays an image based on the movement amounts of these driving parts together with the axis positions of the respective axes of the joint axes of the robot.

[0117] When the display control unit 27 controls the orientation of the image on the display area of the display unit, and thus the orientation of the display unit changes due to the driving of the robot, the image in the display area of the display unit can be displayed in a desired orientation. Alternatively, an input image can be displayed in a desired orientation to perform an input operation.

[0118] Next, a specific example of the image displayed by the display control unit 27 will be described. FIG. 23 shows a schematic side view of the robot when the robot is in the first position and orientation. Here, the case where the reference direction indicated by the arrow 80 is the upward vertical direction for the image of the display unit 61a will be described. Such a reference direction can be set, for example, by the first control or the third control for setting the reference direction of the present embodiment. In the display unit 61a, the display phase direction indicated by the arrow 84 faces the same direction as the reference direction indicated by the arrow 80. That is, the display phase direction indicated by the arrow 84 faces the upper side in the vertical direction.

[0119] FIG. 24 shows a schematic view of the robot when the robot is driven from the first position and orientation. FIG. 25 shows a schematic view of a comparative example robot when the robot is driven from the first position and orientation. Here, the axis position of the J2 axis 52 between the swivel base 13 and the lower arm 12 has changed. As indicated by the arrow 93, the lower arm 12 has rotated and the orientation has changed.

[0120] FIG. 24 shows the state of the image on the display unit 61a when the display control by the processing unit 21 of the present embodiment is carried out. The display control unit 27 controls the orientation of the image on the display area of the display unit 61a so as to display an image having a predetermined posture relationship with respect to the reference direction based on the axis positions of the respective axes of the joint axes of the robot 1 and the reference direction. The display phase direction indicated by the arrow 84 maintains a direction parallel to the reference direction. That is, the display phase direction is maintained in the upward vertical direction.

[0121] In the comparative example shown in FIG. 25, the display control of the present embodiment is not carried out. In the display unit 61a, the display phase direction indicated by the arrow 85 rotates as the lower arm 12 rotates. As a result, the image displayed on the display unit 61a is tilted. The image displayed on the display unit 61a becomes difficult for the operator to see.

[0122] On the other hand, referring to FIG. 24, by carrying out the display control by the processing unit 21 of the present embodiment, even if the axis position of the joint axis changes, the display phase direction indicated by the arrow 84 faces the upper side in the vertical direction. For this reason, a state in which the image displayed on the display unit 61a is easy to see is maintained.

[0123] FIG. 26 shows a schematic side view of the robot when the robot is in the second position and posture. FIG. 27 shows a schematic plan view of the robot when the robot is in the second position and posture. In this example, the display area (reference plane) of the display unit 63b is parallel to the horizontal direction. For the display unit 63b, the reference direction indicated by the arrow 80 is set in the horizontal direction. Such a reference direction can be set by the first control, the second control, or the third control for setting the reference direction of the present embodiment.

[0124] In the display unit 63b, the display phase direction indicated by the arrow 84 faces the horizontal direction. Further, as shown by the arrow 94, the operator is viewing the display unit 63b from above the robot 1. The display phase direction indicated by the arrow 84 is in a direction in which the operator can easily view the image.

[0125] Fig. 28 shows a schematic diagram of the robot when it is driven from the second position and posture. Fig. 29 shows a schematic diagram of a comparative example robot when it is driven from the second position and posture. In the example here, as shown by arrow 95, the swivel base 13 rotates around the J1 axis 51.

[0126] In the example shown in Fig. 28, the processing unit 21 performs display control. The display control unit 27 controls the orientation of the image on the display area of the display unit so as to display a display having a predetermined posture relationship with respect to the reference direction based on the axis position and the reference direction of each axis of the joint axis of the robot. The display phase direction shown by arrow 84 is parallel to the reference direction shown by arrow 80.

[0127] In the comparative example shown in Fig. 29, the display control of the present embodiment is not performed. The display phase direction shown by arrow 85 in the display unit 63b changes in direction as the upper arm 11 rotates. As a result, when the operator views the display unit 63b, the image displayed on the display unit 63b is tilted. The image displayed on the display unit 63b becomes difficult for the operator to view.

[0128] Referring to Fig. 28, by performing the display control by the processing unit 21 of the present embodiment, even when the axis position of the joint axis changes, the display phase direction shown by arrow 84 maintains a direction parallel to the reference direction. The image displayed on the display unit 63b is displayed in the same orientation even when the robot is driven. Therefore, the operator can maintain a state where the image displayed on the display unit 63b is easy to view.

[0129] FIG. 30 shows a schematic side view of the robot when it is in the third position and posture. The upper arm 11 to which the display unit 63b is fixed is inclined with respect to the horizontal direction. For the display unit 63b, the reference direction indicated by the arrow 80 is set to face the horizontal direction. Such a reference direction can be set by the second control or the third control for setting the reference direction of the present embodiment. In the example here, the normal direction of the display area (reference plane) of the display unit 63b and the reference direction are not orthogonal. And the operator views the display unit 63b in the direction indicated by the arrow 96.

[0130] FIG. 31 shows an explanatory diagram of the display phase direction when the robot is driven from the third position and posture. Referring to FIGS. 30 and 31, in the example here, as shown by the arrow 95, the swivel base 13 rotates around the J1 axis 51. In the example shown in FIG. 31, the display control by the processing unit 21 is being performed. The display control unit 27 controls the orientation of the image on the display area of the display unit so as to display an image having a predetermined posture relationship with respect to the reference direction based on the axis positions of the respective axes of the joint axes of the robot and the reference direction, and then displays it.

[0131] As shown by the arrows 95a and 95b, even when the upper arm 11 rotates, the display phase direction indicated by the arrow 84 is maintained in the direction obtained by projecting the reference direction onto the display area (reference plane) of the display unit 63b. The display phase direction indicated by the arrow 84 is a direction in which it is easy for the operator to view the image. When the operator views the display unit 63b in the direction indicated by the arrow 96, the image is displayed in the same orientation.

[0132] FIG. 32 shows an explanatory diagram of the display phase direction of a comparative example when the robot is driven from the third position and posture. In the comparative example, the display control of the present embodiment is not performed. The display phase direction indicated by the arrow 85 in the display unit 63b changes in direction as the upper arm 11 rotates as indicated by the arrows 95a and 95b. As a result, when the operator views the display unit 63b, the image displayed on the display unit 63b is tilted. It becomes difficult for the operator to view the image displayed on the display unit 63b.

[0133] Referring to FIG. 31, by controlling the display by the processing unit 21 of the present embodiment, even when the axial position of the joint axis changes, the display phase direction maintains the direction in which the operator can easily view the image. For this reason, the operator can maintain a state in which the image displayed on the display unit 63b is easy to view.

[0134] In the above-described embodiment, the display coordinate system, the reference plane, and the reference point are set for the display area of the display unit to calculate the display phase direction, but the present invention is not limited to this form. The processing unit does not have to set the display coordinate system, the reference plane, and the reference point. The processing unit can calculate the display phase direction by any control based on the reference direction. For example, when the reference direction and the display area of the display unit are parallel, the processing unit may set the reference direction as the display phase direction.

[0135] Furthermore, the display control unit may perform image processing on the image generated on the reference plane. For example, any image conversion operation such as image enlargement, image reduction, or predetermined image conversion can be performed. Then, the display control unit can display the converted image on the display unit. For example, as described above, when the display area of the display unit is curved, an image on the reference plane can be projected onto the display area of the display unit to generate an image to be displayed in the display area.

[0136] (Example of an image displayed on the display unit) Next, an example of an image displayed in the display area of the display unit by the display control unit will be described. Here, an example of an image displayed on the display unit 61a will be described.

[0137] FIG. 33 shows a fourth image displayed on the display unit. FIG. 34 shows a fifth image displayed on the display unit. The fourth image 71c is an image showing the driving state of the robot. The fifth image 72b is an input image for inputting the driving speed around the joint axis at the joint axis where the display unit is arranged or at a desired joint axis. Note that there may be a case where other joint axes other than one joint axis are driven together with one joint axis. In this case, when changing the driving speed around one joint axis, the driving speeds of all the joint axes may be adjusted so that the position and orientation of the tip of the robot do not change. Also, in the input image displayed on the display unit, the moving speed of the tip of the robot in the running operation program may be changed. The display control unit 27 can display information or an input image in at least one of characters, figures, symbols, colors, and patterns in the display area of the display unit.

[0138] Furthermore, the display control unit 27 may display so as to indicate the orientation of the image by at least one of characters, figures, symbols, colors, and patterns. In the fourth image 71c and the fifth image 72b, an arrow image 73 indicating the upper side of the image is displayed.

[0139] In this way, by the display control unit generating an image in the display area so that the orientation of the image can be understood, the operator can easily recognize the orientation of the image in the display area on the display unit. Also, when the display unit displays an input image, the operability of the operator can be improved.

[0140] FIG. 35 shows a sixth image displayed on the display unit. FIG. 36 shows a seventh image displayed on the display unit. In the sixth image 71d, the character "up" indicating the upper side of the image is displayed. In the seventh image 71e, a part of the frame surrounding the information indicating the operation state is recessed to indicate the upper side of the image.

[0141] FIG. 37 shows the eighth image displayed on the display unit. FIG. 38 shows the ninth image displayed on the display unit. In the eighth image 71f and the ninth image 71g, a pattern indicating the upper side is displayed at the upper part. Further, in the ninth image 71g, the pattern indicating the upper side is displayed in a color different from the color of the background of the display area. Thus, the display control unit 27 can generate an image so as to indicate the orientation of the image in various forms.

[0142] Regarding the above-described embodiment, the image indicating the upper side has been described, but the present invention is not limited to this form. The display control unit may display the image so that the orientation of the image can be understood. For example, the display control unit may display characters or the like indicating the lower side of the image.

[0143] FIG. 39 shows the tenth image displayed on the display unit. FIG. 40 shows the eleventh image displayed on the display unit. The display control unit 27 may perform control to change the size of an image of a predetermined portion on the display area of the display unit 61a according to the moving speed of the display unit 61a of the display device accompanying the driving of the robot. For example, when the moving speed of the display unit 61a is small, the display control unit 27 may display a predetermined image in a small size, and when the moving speed of the display unit 61a is large, the display control unit 27 may display a predetermined image in a large size.

[0144] In the tenth images 71ha and 71hb of FIG. 39, the number of times of performing the work is displayed on the display unit 61a. The display control unit 27 can change the size of the image of the number of times of execution as shown by the arrow 97 according to the moving speed of the display unit 61a. For example, the display control unit 27 can calculate the moving speed of the display coordinate system as the moving speed of the display unit 61a. The display control unit 27 acquires the position and time of the origin of the display coordinate system at predetermined time intervals. The display control unit 27 can calculate the moving speed of the origin of the display coordinate system based on the position and time of the origin of the display coordinate system.

[0145] The display control unit 27 can display a small image of the number of executions as shown in the image 71ha when the moving speed of the display unit 61a is smaller than a predetermined threshold value. The display control unit 27 can display a large image of the number of executions as shown in the image 71hb when the moving speed of the display unit 61a is equal to or greater than the predetermined threshold value. By performing this control, when the driving speed of the robot is high, a desired part of the image is displayed large, so that the image displayed is easier for the operator to see.

[0146] The 11th images 72ca and 72cb in FIG. 40 are input images for the operator to perform input operations. In the 11th images 72ca and 72cb, a predetermined set value can be changed by pressing the “+” button or the “-” button. The display control unit 27 can change the size of the image showing the set value as shown by the arrow 98 according to the moving speed of the display unit 61a. Also, the display control unit 27 can change the size of the button.

[0147] The display control unit 27 can display a large image of the set value and a small image of the button as shown in the image 72ca when the moving speed of the display unit 61a is smaller than a predetermined threshold value. The display control unit 27 can 72cb display a small image of the set value and a large image of the button as shown in the image. By this control, even when the driving speed of the robot increases, the image of the button becomes large, so that the operator can stably press the button.

[0148] In the robot system of this embodiment, when the display device has an input function, an operator can perform an input operation during the period when the robot is driven. By performing control to change the size of an image according to the moving speed of the display unit, the operator can make the image easier to view or the input operation easier. For example, by displaying a large image of a button when the moving speed of the display unit increases, the operator can easily perform an input operation. In addition, when enabling an input operation from the display device during the period when the robot is driven, for safety, it is preferable to provide a limit on the moving speed of the robot or the driving speed of the joint axis at which the input operation can be performed. For example, it is preferable to enable the input operation when the moving speed of the robot or the driving speed of the joint axis is equal to or lower than a predetermined speed. Also, the set value at which the input operation is possible may be changed according to the moving speed of the robot or the driving speed of the joint axis.

[0149] In each of the above controls, the order of steps can be appropriately changed within the range where the functions and operations are not changed. The above embodiments can be combined as appropriate. In each of the above figures, the same or equivalent parts are denoted by the same reference numerals. Note that the above embodiments are examples and do not limit the invention. Also, the embodiments include changes to the embodiments shown in the claims.

Explanation of Reference Numerals

[0150] 1 Robot 2 Hand 4 Control Device 6 Robot System 11 Upper Arm 12 Lower Arm 13 Swivel Base 14 Base Portion 15 List 16 Flange 21 Processing Unit 22 Reference Direction Setting Unit 23 Coordinate System Setting Unit 24 Coordinate System Calculation Unit 25 Display Phase Calculation Unit 26 Offset angle setting unit 27 Display control unit 42 Memory unit 43 Operation instruction unit 51 J1 axis 52 J2 axis 53 J3 axis 54 J4 axis 55 J5 axis 56 J6 axis 59 Pedestal R1, R2 Rotation axes 60, 61a, 61b, 61c, 62, 63a, 63b, 64, 65, 66, 67 Display units 71a, 71b, 71c, 71d, 71e, 71f, 71g, 71ha, 71hb Images 72a, 72b, 72ca, 72cb Images 82 Reference plane 83 Reference point 87 Reference coordinate system 88 Display coordinate system 101a~101d Range 102, 103 Designated points θ1 Angle θ2 Offset angle

Claims

1. A robot including a plurality of joint axes, A display unit disposed on the main body portion of the robot, A display control unit that controls an image displayed in the display area of the display unit, A reference direction setting unit that sets a reference direction which is a reference when determining the orientation of the image displayed on the display unit, A display coordinate system which is a coordinate system set for the display area of the display unit, a reference plane which is the reference plane of the display area of the display unit and is set on the display coordinate system, and a reference point which is a reference for display on the display area of the display unit and is set on the display coordinate system, and a coordinate system setting unit that sets the reference point, A coordinate system calculation unit that calculates the position and orientation of the display coordinate system based on the axis positions of the respective joint axes of the robot, A display phase calculation unit that calculates a display phase direction which is a reference for the orientation of the display on the display area of the display unit by projecting the reference direction onto the reference plane on the display coordinate system, and includes, The display control unit controls and displays the orientation of the image on the display area of the display unit based on the display phase direction calculated by the display phase calculation unit and the reference point. A robot system characterized by this.

2. A robot including a plurality of joint axes, A display unit disposed on the main body portion of the robot, A display control unit that controls an image displayed in the display area of the display unit, A reference direction setting unit that sets a reference direction which is a reference when determining the orientation of the image displayed on the display unit, A display coordinate system which is a coordinate system set for the display area of the display unit, a reference plane which is the reference plane of the display area of the display unit and is set on the display coordinate system, and a reference point which is a reference for display on the display area of the display unit and is set on the display coordinate system, and a coordinate system setting unit that sets the reference point, A coordinate system calculation unit that calculates the position and orientation of the display coordinate system based on the axis positions of the respective joint axes of the robot, A display phase calculation unit that projects the reference direction onto the reference plane on the display coordinate system and further rotates it by an offset angle which is a predetermined angle around the reference point to calculate a display phase direction which is a reference for the orientation of the display on the display area of the display unit, and includes, The display control unit controls and displays the orientation of the image on the display area of the display unit based on the display phase direction calculated by the display phase calculation unit and the reference point. A robot system characterized by this.

3. The display unit displays an input image for inputting information, and has an input function for inputting information by an operation according to the input image. The display control unit controls the orientation of the input image and displays the input image. The robot system according to claim 1 or 2.

4. The reference direction setting unit sets a reference direction on the reference coordinate system of the robot so as to form a relationship between the normal direction of the reference plane and a predetermined posture. The robot system according to claim 1 or 2.

5. The reference direction setting unit sets a reference direction in the reference coordinate system of the robot according to the range where the display unit exists. The robot system according to any one of claims 1 to 3.

6. The reference direction setting unit sets the reference direction to the gravity direction, the direction opposite to the gravity direction, or the direction obtained by rotating the gravity direction by a predetermined posture. The robot system according to any one of claims 1 to 3.

7. The reference direction setting unit sets a reference direction based on a position designated by an operator on the display area of the display unit. The robot system according to any one of claims 1 to 3.

8. When the angle formed by the reference direction and the normal direction of the reference plane changes due to the drive of the robot and the angle formed with the normal direction becomes smaller than a predetermined threshold value, the reference direction setting unit maintains the reference direction before the angle formed with the normal direction becomes smaller than the predetermined threshold value without changing it. The robot system according to claim 1 or 2.

9. An offset angle setting unit that sets an offset angle, which is a predetermined angle centered on the reference point, based on a position designated by an operator on the display area of the display unit. The robot system according to claim 2.

10. When inputting information on the display unit, the display unit displays an image for receiving input of specific information for enabling an operator's input operation, and when the operator inputs the specific information, switches to an image that enables the operator's input operation. The robot system according to claim 3.

11. The display unit is formed so that at least one of the authority for enabling an input operation and the authority for information to be displayed can be changed by a predetermined input operation by an operator. The robot system according to claim 3.

12. The display control unit displays, on the display area of the display unit, so as to indicate the orientation of the image, by at least one of characters, figures, symbols, colors, and patterns, the robot system according to any one of claims 1 to 11.

13. The display control unit controls so that the size of an image of a predetermined portion on the display area of the display unit changes according to the moving speed of the display unit accompanying the driving of the robot, the robot system according to any one of claims 1 to 12.

14. The display unit is detachable from the main body portion of the robot, the robot system according to any one of claims 1 to 13.

15. A robot including a plurality of joint axes, A display unit disposed on the main body portion of the robot, A display control unit that controls an image displayed in the display area of the display unit, A reference direction setting unit that sets a reference direction, which is a direction serving as a reference when determining the orientation of the image displayed in the display unit, and includes: The display control unit controls and displays the orientation of the image on the display area of the display unit so as to display a display having a predetermined posture relationship with respect to the reference direction based on the axial position of each joint axis of the robot and the reference direction. The reference direction setting unit sets the reference direction based on a position specified by an operator on the display area of the display unit, a robot system.

Images