Robot system equipped with a display unit

Abstract

The robot system of the present invention includes: a display control unit that controls an image displayed in a display area of ​​the display unit; and a reference direction setting unit that sets a reference direction for defining the direction of the image. The display control unit controls the direction of the image on the display unit based on the axial position of each joint axis of the robot and the reference direction, in a manner that forms a specific posture relationship with the reference direction.

Description

Technical Field

[0001] This invention relates to a robot system having a display unit. Prior Technology

[0002] A robotic system comprises a robot and tools mounted on the robot. The robot can perform specific tasks while changing its position and posture. The robot includes multiple constituent components such as an arm. In the prior art, it is well known that robot systems have display devices mounted on the constituent components of the robot's body. For example, Japanese Patent Publication No. 2018-529488 discloses a holding device for medical use, which has a display unit at the joint axis, including a ring that serves as an LED (light emitting diode) component, and the display unit displays the direction of movement, etc.

[0003] Furthermore, in the prior art, cameras are known to be able to obtain images that are not tilted even when the image is taken in a tilted state (e.g., Japanese Patent Application Publication No. 4-81081). [Previous Technical Documents] [Patent Literature]

[0004] [Patent Document 1] Japanese Patent Publication No. 2018-529488 [Patent Document 2] Japanese Patent Application Publication No. 4-81081 Summary of the Invention

[0005] [The problem that the invention aims to solve]

[0006] A robot changes its position and posture to move its work tools. When a display unit is integrated into the robot's components, the position and posture of the display unit change along with the components when the robot is driven. As a result, the orientation of the image displayed on the display unit changes for the operator, sometimes making it difficult for the operator to observe the image. [Technical means to solve the problem]

[0007] The robot system of the present invention comprises: a robot including a plurality of joint axes; and a display unit disposed on the body of the robot. The robot system includes: a display control unit that controls an image displayed in a display area of ​​the display unit; and a reference direction setting unit that sets a reference direction, which serves as a reference direction when the direction of the image displayed in the display unit is specified. The display control unit controls the direction of the image in the display area of ​​the display unit to display the image in a manner that forms a specific posture relationship with the reference direction, based on the axial positions of each joint axis of the robot and the reference direction. [Effects of the Invention]

[0008] According to the present invention, a robot system is provided which has a display unit installed on the body of the robot, and the image displayed on the display unit can be easily observed when the position and posture of the robot change. Simple Explanation of the Diagram

[0009] Figure 1 is a schematic diagram of the robot system according to the implementation method. Figure 2 is a schematic diagram illustrating the joint axes of the robot according to the embodiment. Figure 3 is a block diagram of the robot system according to the implementation method. Figure 4 is a schematic diagram of a robot illustrating the planar display unit mounted on the robot. Figure 5 is a schematic diagram of a robot illustrating the curved display section mounted on the robot. Figure 6 shows the first image displayed in the display section. Figure 7 shows the second image displayed in the display section. Figure 8 shows the third image displayed in the display section. Figure 9 is a flowchart of the control of the display unit displaying the image in the embodiment. Figure 10 is a top view illustrating the display coordinate system, reference point, reference plane, and display phase direction. Figure 11 is a schematic diagram illustrating the first control for setting the reference direction. Figure 12 is a schematic diagram illustrating the second control for setting the reference direction. Figure 13 is a first schematic diagram of a robot illustrating the third control of setting the reference direction. Figure 14 is a second schematic diagram of a robot illustrating the third control of setting the reference direction. Figure 15 is a third schematic diagram of a robot illustrating the third control of setting the reference direction. Figure 16 is a schematic diagram illustrating the fourth control for setting the reference direction. Figure 17 is a schematic diagram illustrating the fifth control for setting the reference direction. Figure 18 is a schematic diagram illustrating the first control for setting the display phase direction. Figure 19 is a schematic diagram illustrating the second control for setting the display phase direction. Figure 20 is a schematic diagram of a robot illustrating the third control that sets the display phase direction. Figure 21 is a schematic diagram illustrating the third control for setting the display phase direction. Figure 22 is a schematic diagram illustrating the fourth control for setting the display phase direction. Figure 23 is a schematic diagram showing the display phase direction of the display unit when the robot is in the first position and posture. Figure 24 is a schematic diagram showing the display phase direction when the robot performs display control driven by the first position and posture. Figure 25 is a schematic diagram showing the display phase direction when the robot is driven from position 1 and posture without display control. Figure 26 is a schematic diagram of the robot in position 2 and posture. Figure 27 is a schematic top view showing the display phase direction of the display unit when the robot is in the second position and posture. Figure 28 is a schematic top view showing the display phase direction when the robot performs display control driven by the second position and posture. Figure 29 is a schematic top view showing the display phase direction of the robot when it is driven from the second position and posture without display control. Figure 30 is a schematic diagram of the robot in position 3 and posture. Figure 31 is a schematic front view of the display phase direction of the display unit when the robot performs display control driven by the third position and posture. Figure 32 is a schematic front view of the display phase direction of the display unit when the robot is driven from the third position and posture without display control. Figure 33 shows the fourth image displayed in the display section. Figure 34 shows the fifth image displayed in the display section. Figure 35 shows the sixth image displayed in the display section. Figure 36 shows the 7th image displayed in the display section. Figure 37 shows the 8th image displayed in the display section. Figure 38 shows the 9th image displayed in the display section. Figure 39 shows the 10th image displayed in the display section. Figure 40 shows the 11th image displayed in the display section. Implementation

[0010] The robot system of this embodiment will be described with reference to Figures 1 to 40. The robot system of this embodiment includes a robot with a plurality of joint axes and a display device for displaying specific information. The display unit of the display device is disposed on the body of the robot.

[0011] (Robot System) Figure 1 shows a schematic diagram of the first robot system of this embodiment. The robot system 6 of this embodiment has the function of transporting workpieces. The robot system 6 includes a robot device, which includes a hand 2 as a working tool (end effector) and a robot 1 for changing the position and posture of the hand 2. The robot system 6 includes a control device 4 for controlling the robot 1 and the hand 2.

[0012] The robot 1 of this embodiment includes a base portion 14 fixed to a mounting surface and a rotating base 13 supported by the base portion 14. The rotating base 13 is formed in a manner that allows rotation relative to the base portion 14. The robot 1 includes an upper arm 11 and a lower arm 12. The lower arm 12 is rotatably supported on the rotating base 13 via a joint. The upper arm 11 is rotatably supported on the lower arm 12 via a joint. Furthermore, the upper arm 11 rotates about a rotation axis parallel to the direction in which the upper arm 11 extends. The robot 1 includes a wrist portion 15 connected to the end of the upper arm 11. The wrist portion 15 is rotatably supported on the upper arm 11 via a joint. The wrist portion 15 includes a flange 16 that rotates about a rotation axis along the direction in which the wrist portion 15 extends. A hand portion 2 is fixed to the flange 16.

[0013] Figure 2 is a schematic diagram of a robot used to explain the joint axes of the robot according to this embodiment. In this invention, a joint axis refers to the axis of the joint portion that connects the links constituting the robot, and is the part that changes the positional or angular relationship between the links. By changing the position of each axis (the position is an angle in the case of a rotary axis, and a displacement length in the case of a linear axis), the positional relationship between the links can be changed. As a result, at least one of the position and posture of the robot's front end can be changed. Furthermore, an actuator for moving the position of the joint axis can be provided at a location different from the part that constitutes the joint axis.

[0014] Referring to Figures 1 and 2, the robot 1 of this embodiment includes six joint axes. Starting from the base 59 side, which serves as the base portion 14 of the robot 1, the six joint axes are, in sequence, J1 axis 51 as the first joint axis, J2 axis 52 as the second joint axis, J3 axis 53 as the third joint axis, J4 axis 54 as the fourth joint axis, J5 axis 55 as the fifth joint axis, and J6 axis 56 as the sixth joint axis.

[0015] As shown on the right side of Figure 2, axes J1 51, J4 54, and J6 56 constitute a rotation axis R1 that rotates about the connecting rod that connects the joint axes. Also, axes J2 52, J3 53, and J5 55 constitute a rotation axis R2 that rotates about a direction orthogonal to the connecting rod that connects the axes.

[0016] In this embodiment, the hand 2 holds or releases the workpiece. The hand 2 holds the workpiece by closing its opposing claws. The working tool is not limited to the hand that holds the workpiece. Any working tool can be installed on the robot depending on the operation performed by the robot system. For example, when performing arc welding on the robot system, a welding gun can be installed on the robot.

[0017] Figure 2 is a simplified diagram illustrating the structure of the axes of robot 1. Furthermore, if the origin of a joint axis is set as the origin of the coordinate system (joint coordinate system) for each axis, and is also set as the point connecting links, then the position of the axis origin is represented by its position in the coordinate system set in space. Furthermore, hereinafter, the coordinate system used as a reference for setting the position and posture of the robot's joint axes, links, front end, and working tools mounted on the front end in space will be referred to as the robot's reference coordinate system 87. The reference coordinate system 87 is a coordinate system in which the position of the origin and the direction of the coordinate axes are fixed in the space where the robot is located.

[0018] In this embodiment, when the joint axis is a rotational axis, the position of the joint axis refers to the angle of the rotational axis. Moving the position of the joint axis means rotating the rotational axis to change its position. When the joint axis is a linear axis, the position of the joint axis refers to its position in the direction of movement of the linear axis. Moving the position of the joint axis means moving the linear axis to change its position. When described as the origin position of the joint axis, it refers to the origin position of the coordinate system set for each axis in a spatially defined coordinate system. The spatially defined coordinate system is a coordinate system used to represent at least one of the positions and orientations of the robot 1's front end (working tool), the flange 16 for mounting the front end to the robot 1, and the coordinate system set for each axis, on an orthogonal coordinate system fixed relative to space.

[0019] Furthermore, in order to represent at least one of the position and posture of robot 1 on the reference coordinate system 87 for the robot set in space, the coordinate system set for the robot device is called the tool coordinate system. The origin of the tool coordinate system, the point of translational movement, and the center point of rotational movement are set as control points. In this embodiment, all six axes of robot 1 are rotational axes, but robot 1 may also include linear axes. Furthermore, the robot 1 in this embodiment is a vertical multi-joint robot composed of six axes. However, as long as the position of each axis can be controlled and the orthogonal position can be controlled, it can also be any robot with other forms.

[0020] Furthermore, when the robot is mounted on a transfer axis or other drive device that changes the position and posture of the robot, the robot's reference coordinate system can also be set for space in the following way: that is, the position and posture of the robot or a part of the robot are also defined by taking into account the drive of such drive devices.

[0021] Figure 3 is a block diagram showing the robot system of this embodiment. Referring to Figures 1 to 3, the robot 1 includes a robot drive unit that changes the position and posture of the robot 1. The robot drive unit includes a robot drive motor 19, which serves as an actuator to drive components such as the arm. The robot drive unit drives the actuators disposed on the joint axes in a manner that moves the rotational position of each joint axis of the robot 1.

[0022] Furthermore, the robot drive unit only needs to be able to drive the robot's joint axes to move, and can be constructed using any principle or power source. Also, when the joint axis is a linear axis, the robot drive unit drives the actuator mounted on the joint axis to move the position on the linear axis. The hand 2 has a hand drive unit that drives the hand 2. The hand drive unit includes a pressure pump and valves for driving the claw of the hand 2.

[0023] The control unit 4 includes an arithmetic processing unit (computer) with a CPU (Central Processing Unit) as its processor. The arithmetic processing unit includes RAM (Random Access Memory) and ROM (Read Only Memory) connected to the CPU via a bus. The motion program 41 contains instructions for driving the robot 1 and the hand 2. The robot system 6 is driven by the motion program 41 to transport workpieces.

[0024] The control device 4 includes a memory unit 42 for storing information. The memory unit 42 stores information related to the control of the robot 1 and the hand 2. The motion program 41 is stored in the memory unit 42. The memory unit 42 may contain a non-temporary memory medium. For example, the memory unit 42 may contain a memory medium capable of storing information, such as volatile memory, non-volatile memory, magnetic memory medium, or optical memory medium.

[0025] The control device 4 includes a motion command unit 43 that sends out motion commands. The motion command unit 43 is equivalent to a processor driven by the motion program 41. The processor functions as the motion command unit 43 by reading the motion program 41 and implementing the control specified by the motion program 41. The robot drive unit is driven based on the motion commands of the robot 1 output from the motion command unit 43 to change the rotational position of the robot drive motor 19 on the joint axis of the robot 1.

[0026] The motion command unit 43 sends motion commands to drive the robot 1 to the robot drive circuit 45. The robot drive circuit 45 includes circuitry for driving the robot drive motor 19. The robot drive circuit 45 supplies power to the robot drive motor 19 based on the motion commands. Furthermore, the motion command unit 43 sends motion commands to drive the hand 2 to the hand drive circuit 44. The hand drive circuit 44 includes circuitry for driving the hand drive unit. The hand drive circuit 44 supplies power to the hand drive unit based on the motion commands. In this embodiment, the control device 4 is provided separately from the robot 1, but it is not limited to this configuration. The control device can also be installed inside the robot.

[0027] Robot 1 is equipped with a rotational position detector 18, which outputs the rotational position, or axis position data, of the joint axes of robot 1. The rotational position detector 18 may include, for example, an encoder. The position and posture of robot 1 can be detected by the output of the rotational position detector 18. In this embodiment, the rotational position detector 18 is mounted on a robot drive motor 19, which is configured corresponding to each joint axis.

[0028] The robot system 6 of this embodiment includes a display device for displaying arbitrary information. The display device includes at least one display unit 60 disposed on the body portion of the robot 1. In FIG1, as an example of the display unit 60, a display unit 61a is mounted at the joint between the rotating base 13 and the lower arm 12. The display device includes a processing unit 21, which generates the image displayed on the display unit 60 or processes commands from the display unit 60. In this embodiment, the processing unit 21 includes a control device 4.

[0029] The processing unit 21 includes a reference direction setting unit 22, which sets the reference direction, i.e., the direction of the image displayed on the display unit 60, as a reference. The processing unit 21 also includes a coordinate system setting unit 23, which sets a coordinate system, i.e., a display coordinate system, for the display area of ​​the display unit 60.

[0030] The processing unit 21 includes a coordinate system calculation unit 24, which calculates the position and orientation of the display coordinate system based on the axis positions of each joint axis of the robot 1. The processing unit 21 includes a display phase calculation unit 25, which calculates the display phase direction, which serves as the reference for the display direction on the display area of ​​the display unit 60. The processing unit 21 includes an offset angle setting unit 26, which sets the offset angle of the image displayed on the display unit. The processing unit 21 includes an instruction processing unit 29, which processes commands input through the operation of the display unit 60. The processing unit 21 includes a display control unit 27, which controls the image displayed in the display area of ​​the display unit 60.

[0031] The processing unit 21 is equivalent to a processor driven by the action program 41. The processor functions as the processing unit 21 by reading the action program 41 and implementing the controls specified by the action program 41. Furthermore, each unit within the processing unit 21—including the reference direction setting unit 22, coordinate system setting unit 23, coordinate system calculation unit 24, display phase calculation unit 25, offset angle setting unit 26, display control unit 27, and instruction processing unit 29—is equivalent to a processor driven by the action program 41. Each of these units functions as a separate unit by implementing the controls specified by the action program 41 through the processor.

[0032] (Display section) The robot system 6 of this embodiment has one or more display units 60 on the robot body, including the links, joint axes, and arm tip. The display unit 60 is mounted on a component of the robot whose position and posture are changed by the drive of a robot drive unit. The position and posture of the display unit 60 change due to the robot's joint axis drive. Furthermore, sometimes the robot is mounted on a drive device that changes its position and posture. This invention can also be applied to situations where the position and posture of the display unit 60 change due to the drive device. The display unit 60 of the display device can be any display panel capable of displaying text or images. For example, a liquid crystal display panel or an organic EL (Electroluminescence) display panel can be used as the display unit 60.

[0033] The display section 60 is configured to be appropriately mounted on the designated portion, and is preferably thin. The display section 60 is preferably shaped so as not to protrude significantly from the robot's body. Furthermore, when the display section 60 is positioned at the end of the robot, it is preferably shaped to be integral with the robot's body and have a smooth outer surface.

[0034] Figure 4 shows a schematic diagram of a robot illustrating examples of various display units according to this embodiment. Display units 61a, 61b, 61c, 62, 63a, 63b, 64, and 65 are formed in a plate shape, each having a planar display area. Display units 61a and 61b are fixed to the joint with their display areas perpendicular to axes J2 52 and J3 53. Display unit 61c is fixed to the joint with its display area perpendicular to axis J5 55. Display units 61a, 61b, and 61c move together with the joint.

[0035] Display unit 62 is fixed to the rotating base 13 with its display area parallel to the J1 axis. Display unit 62 rotates together with the rotating base 13. Display units 63a and 63b are fixed to the upper arm 11 with their display areas parallel to the J4 axis 54. Display units 63a and 63b move together with the upper arm 11. Display unit 64 is arranged with its display area parallel to the J5 axis 55. Display unit 65 is fixed to the wrist 15 with its display area parallel to the J6 axis 56. Display units 64 and 65 move together with the wrist 15. Thus, displays 61a, 61b, 61c, 62, 63a, 63b, 64, and 65 are fixed to the components of robot 1 and move together with the components of robot 1. Furthermore, each display unit rotates about the joint axis of robot 1.

[0036] Figure 5 shows a schematic diagram of another robot illustrating various display units of this embodiment. Display units 66 and 67 include flexible display panels. Display units 66 and 67 are formed such that the display area extends circumferentially along the surface of the constituent members of robot 1. Display units 66 and 67 may include thin and flexible display panels, such as organic EL display panels. Display unit 66 is fixed by wrapping around the lower arm 12. Display unit 66 is formed such that the display area is parallel to the direction of extension of the lower arm 12. Display unit 67 is fixed by wrapping around the upper arm 11. Display unit 67 is formed such that the display area is parallel to the direction of extension of the upper arm 11. Display units 66 and 67 are fixed to the constituent members of robot 1 and move together with the constituent members of robot 1.

[0037] The display units 66 and 67 in this embodiment include one display panel, but are not limited to this form. A plurality of display panels may also be arranged around the constituent members of the robot 1 to surround the constituent members, thus forming a display unit. A single image can be displayed using a plurality of display panels.

[0038] Figure 6 shows the first image displayed in the display unit. Figure 7 shows the second image displayed in the display unit. In the following embodiments, the display unit 61a, which is arranged between the rotating base 13 and the lower arm 12, is used as an example for explanation. The display control unit 27 of the display device can display images on the display area of ​​the display unit 61a based on data sent from other devices. The information displayed in the display unit includes, for example, the robot's motion status, the name of the executed motion program, the execution status of the motion program, the alarm status, the robot's drive speed, the number of times the operation has been performed, and the number of times the retrieval failed. Here, the robot's drive speed or motion speed can also be the translational speed or rotational speed of the robot's front end or tool coordinate system. Furthermore, the robot's drive speed or motion speed can also be the rotational speed of each joint axis. In the following description, unless there are special changes, these can also be set to be the same.

[0039] Furthermore, the information displayed on the display unit includes the cause of the problem and the countermeasures, the content of the next task to be performed, the connection status of the network connecting to the robot, the load status of the actuators included in the robot drive unit that drives the robot, the current time, date, and the elapsed time of the task. Preferably, the information displayed on the display unit is information related to the robot's operation or settings, or any information sent to the robot from external devices, etc., that is useful to the operator by being displayed on the robot's body.

[0040] Image 71a shows information such as whether the robot's drive motor is running (State), the name of the executed motion program (Prog.), the drive speed around the joint axis (Speed), the number of times the program has been executed (Count), the time, and the date. Image 71b also shows information about driving the robot.

[0041] Figure 8 shows the third image displayed in the display unit. The display unit not only has the function of displaying information, but also the function of inputting specific information through operator operation. The display device can display an input image used to input pre-defined information on the display unit. The third image 72a is an input image used to set the drive speed of the joint axis with the display unit or the desired joint axis around the joint axis. Furthermore, sometimes a joint axis is driven together with other joint axes besides that one joint axis. In this case, when the drive speed around one joint axis is changed, the drive speed of all joint axes can be adjusted so that the position and posture of the robot's front end remain unchanged. Also, the movement speed of the robot's front end in the currently executing motion program can be changed in the input image displayed on the display unit.

[0042] The display device can also function as an input device for inputting information through operator actions corresponding to the input image. Furthermore, it can function as a teaching device for manually driving robots or creating motion programs. For example, the display device can have a touch panel-type display section capable of input operations. Alternatively, the display device can also have buttons for implementing emergency stop or other desired functions.

[0043] Thus, the display device of this embodiment includes two types of devices: a device that only has a display function, and a device that has both a display function and an input function. Referring to FIG3, the instructions input to the display unit 60 of the display device are sent to the instruction processing unit 29. The instruction processing unit 29 processes the input instructions. For example, the instruction processing unit 29 sends the robot's action instructions to the action instruction unit 43, or changes the setting values ​​stored in the memory unit 42 for driving the robot.

[0044] The display control unit 27 can display any input image on the display area. For example, it can display an image of a soft keyboard, an image of numbers only, or an image of a slider or other bar-shaped image to change the input value. Furthermore, the display control unit 27 can display an image of continuous input operation by rotating a specific image, an image that rotates like a telephone dial, or buttons related to screen changes.

[0045] As a method of information input, any input method can be used, such as inputting by touching the display area like a touch panel, or inputting non-contactly using a capacitive sensor. In the case of non-contact input, information can be input through operations in the space above the display area. As a device for detecting input operations, any sensing method can be used, such as a capacitive sensor, an optical sensor, an optical camera, an infrared camera, or an ultrasonic sensor.

[0046] Furthermore, in addition to serving as input devices for information input, contact or non-contact sensors can also function as contact sensors to detect contact with people or surrounding objects. That is, by switching operating modes, a sensor that detects input operations can also be used as a sensor to ensure safety.

[0047] Information input to the display device can be transmitted via wired or wireless communication to the robot's control device or other devices connected via a network. Alternatively, the input information can be stored and memorized in a teaching device that includes a display function. Information calculated or processed based on the input information can also be displayed on the display section of the display device.

[0048] The image used for inputting information by the operator is the input image, preferably configured to allow input of any information for setting or confirmation purposes. For example, the input image is preferably configured to allow input of changes to setting values, changes to the robot's drive speed, indications for robot standby, changes to the robot's actions, robot drive, or interruption of actions. Furthermore, the input image can be an image used for any input, such as inputting information about tools or workpieces for changing tools mounted on the robot's front end, setting the coordinate system, position teaching, changing to direct teaching mode, or changing to automatic operation mode. The functions that can operate the setting values ​​are preferably limited to frequently used or simple functions. Alternatively, the input image can also be configured to allow for changes in the types of input functions.

[0049] Furthermore, when information is input to the display device, the display unit can also display an image indicating that specific information has been received, which enables the operator's input operation. When the operator inputs specific information, the display unit can switch to an image that allows the operator to perform input operations. For example, pressing an input button for a specific time, entering a password, or moving a finger across a specific point on the display area. In this way, the display control unit can also display an image that prohibits input operations to prevent accidental input. Moreover, when the operator performs operations such as password input, an image that allows input operations can be displayed.

[0050] When inputting information onto a display device, at least one of the permissions for performing input operations and the permissions for displaying information can be changed through specific input operations. For example, by performing a specific input operation, the items that can be set can be changed, the robot's actions can be changed, the range of values ​​that can be set can be changed, or the displayed information can be changed. In this way, the display device can also change the instructions or information that can be input, or change the data or information that can be displayed.

[0051] The display section can also be detachably formed on the robot's body. In particular, a display device having a display section and a display device constituting a teaching device can also be detachably formed on the robot's body. In this case, the display device can be equipped with a battery. Alternatively, the display device is preferably powered while mounted on the robot's body. Furthermore, the device for transmitting information displayed on the display device, or the device for transmitting information input to the display device, can be formed using a wired or wireless connection.

[0052] Furthermore, the display coordinate system set for the display unit can be configured such that a pre-generated coordinate system is applied based on the installation location of the display unit at a specific position on the robot's body. Alternatively, the display coordinate system can be directly input after the display unit is installed, or the display coordinate system can be set using methods from prior art.

[0053] When a display device that can be attached to a robot functions as a teaching device, its use can be limited to situations where a teaching device is needed. Alternatively, when multiple displays are available, the teaching device can be installed and used only in necessary parts of the robot. Therefore, the operability of the robot can be improved, the cost of the robot system can be reduced, or the risk of damage to the teaching device due to collisions with surrounding devices during robot operation can be reduced.

[0054] (Basic controls for displaying images) Figure 9 shows a flowchart of the control of the display unit displaying the image in this embodiment. Referring to Figure 3, before the display control unit 27 displays the image, the coordinate system setting unit 23 sets the display coordinate system corresponding to the display unit, and the reference plane and reference point set in the display coordinate system. The coordinate system setting unit 23 sets the display coordinate system, reference plane, and reference point, for example, according to the operation of the operator. Furthermore, the reference direction setting unit 22 sets the direction that becomes the reference for display, i.e., the reference direction, through specific control. The memory unit 42 stores the display coordinate system, reference plane, reference point, and reference direction.

[0055] Referring to Figures 3 and 9, in step 111, the processing unit 21 obtains the display coordinate system, the reference plane, and the reference point from the memory unit 42. In step 112, the processing unit 21 obtains the reference direction from the memory unit 42.

[0056] In step 113, the coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system based on the position and orientation of the robot 1. Specifically, the coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system based on the output of the rotation position detector 18. It calculates the position and orientation of the reference plane shown in the display coordinate system and the position of the reference point.

[0057] In step 114, the display phase calculation unit 25 calculates the display phase direction on the reference plane based on the reference direction and the reference point. That is, the display phase calculation unit 25 sets the direction of the image in the display unit. In step 115, the display control unit 27 generates a reference plane image based on the display phase direction. Then, in step 116, the display control unit 27 displays an image on the display unit 60 based on the reference plane image. Next, these controls will be explained in detail.

[0058] (Coordinate System Setting Department) Figure 10 shows a schematic top view illustrating the display coordinate system, reference plane, reference point, and display phase direction. Figure 10 corresponds to the top view of the display area of ​​the display unit 61a. The coordinate system setting unit 23 of the processing unit 21 sets the display coordinate system 88, which is a coordinate system set for the display area of ​​the display unit. The coordinate system setting unit 23 sets a reference plane 82 and a reference point 83 for the display area of ​​the display unit. The reference plane 82 is the reference plane of the display area of ​​the display unit and is set on the display coordinate system. The reference point 83 is a point that serves as the reference for display on the display area of ​​the display unit and is set on the display coordinate system.

[0059] The display coordinate system can be set at any position and orientation for the display area of ​​the display unit. Using the display coordinate system, the position and orientation of the robot on a reference coordinate system, such as a reference coordinate system, are displayed on the display unit or display area provided on the robot's body. In the display unit 61a, the display coordinate system is set in such a way that the plane formed by the two coordinate axes is coplanar with the display area.

[0060] The reference plane is a plane used as a reference when representing the image output to the display area of ​​the display unit. Furthermore, the reference plane is a plane set on the display coordinate system. The position and orientation of the reference plane on the display coordinate system can be predetermined. In the display unit 61a, the reference plane is set as a plane encompassing the two coordinate axes of the display coordinate system. The reference point is the point that defines the center position of the image output to the display area of ​​the display unit. The orientation of the image is defined by rotating the image around the reference point on the reference plane.

[0061] The reference direction can be set to any direction independently of the display coordinate system. The reference point is set on a reference plane. The reference plane can be set independently of the direction of the coordinate axes of the display coordinate system. For example, the reference plane can also be tilted relative to the XY plane of the display coordinate system. However, when the display area of ​​the display unit is planar, it is preferable that the display area, the reference plane, and the plane formed by the two coordinate axes of the display coordinate system are the same plane. By setting it in this way, the amount of calculation can be reduced.

[0062] When the display area of ​​the display unit is curved rather than planar, a reference point can be set on the display area so that the tangent plane of the reference point is the reference plane. In this case, even if the XY plane on the display coordinate system and the reference plane are the same plane, the reference plane does not include the display area.

[0063] By defining the display phase direction on the reference plane by the display phase calculation unit 25, the display control unit 27 can generate an image displayed on the reference plane. Next, the display control unit 27 can generate an image displayed on the display unit by projecting the image on the reference plane onto the display area of ​​the display unit.

[0064] The coordinate system setting unit 23 can arbitrarily set the display coordinate system 88, the reference plane 82, and the reference point 83. For example, the operator can pre-define the display coordinate system 88, the reference plane 82, and the reference point 83 based on the shape of the display area of ​​the display unit and input them into the processing unit 21. Basically, these settings are pre-defined for the display unit and can be preset as standard settings before the display unit is installed on the robot. The operator can change the settings as needed. For example, after setting the standard values ​​for the display unit, the operator can also change the displayed standard values ​​in a manner suitable for the operation based on the robot's status displayed on the display unit.

[0065] Furthermore, the coordinate system setting unit 23 can automatically set the display coordinate system 88, the reference plane 82, and the reference point 83 for the display unit. For example, there may be a case where the display unit has a wide display area. As shown in the display units 66 and 67 of FIG5, there may be a case where the flexible display unit is wound around the robot's constituent components. In this case, the coordinate system setting unit 23 can automatically set the display coordinate system, the reference plane, and the reference point so that the image is displayed in the same direction regardless of the rotational position of the constituent components. For example, the coordinate system setting unit 23 can also automatically set the setting value to be displayed by moving it around the joint axis.

[0066] Furthermore, when forming display units 66 and 67, sometimes multiple display units are combined and arranged to surround the robot's structural components. In this case, a display coordinate system, reference plane, and reference point can be set for each display area of ​​a single display unit. Alternatively, multiple display units can be combined to form a single display unit in a specific part of the robot, and a display coordinate system, reference plane, and reference point can be set accordingly. In this case, control can be performed to assign images to each display unit for display.

[0067] The display control unit 27 controls the display position and direction (phase) of the display area of ​​the display unit to perform display based on the display reference point 83 on the display area of ​​the display unit and the reference direction (display phase direction) calculated by the display phase calculation unit 25 and which becomes the display direction on the display area of ​​the display unit (the direction shown by arrow 84). For example, the display control unit 27 can display an image with the direction shown by arrow 84 above the image.

[0068] In this embodiment, the display control unit 27 defines the center position of the display on the display area of ​​the display unit using the reference point 83. The display control unit 27 defines the display direction by rotating the image on the reference plane 82 around the reference point 83 as the center of rotation. In this embodiment, the origin and the reference point of the display coordinate system 88 are arranged in the same position, but they can also be arranged in different positions.

[0069] The display phase direction indicated by arrow 84 is set such that it exists on the reference plane 82 and passes through the reference point 83. When the display area of ​​the display unit is planar, the reference plane 82 is preferably located on a plane parallel to or the same as the display area. In this embodiment, the display area of ​​the display unit is planar, but it may also be curved or other shapes instead of planar.

[0070] When the display area of ​​the display unit is curved, the tangent plane of a specific point on the display area can be set as the reference plane. The specific point on the display area can be set as the origin of the display coordinate system, making the reference plane a plane formed by the two coordinate axes of the display coordinate system. The image displayed in the display area of ​​the display unit can be displayed based on the reference point and the display phase direction, or by projecting an image from the reference plane onto the display area. The image displayed in the display area can also be displayed using any method, such as displaying based on calculated information.

[0071] Furthermore, the display area (the portion of the display panel that displays the image) of the display unit 61a is circular, but it can also be any shape such as a regular polygon. In this case, the reference point can be positioned at the center of the display area. For example, the reference point can be set at the center of gravity of the shape of the display area.

[0072] (Coordinate system calculation section) The coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system 88 based on the axial positions of each joint axis of the robot 1. The coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system 88 on the reference coordinate system 87 and other reference coordinate systems based on the axial positions of each joint axis of the robot 1 (angles in the case of rotation axes), the relationship between the positions and orientations of the joint axes expressed in DH (Denavit Hartenberg) parameters, and the position and orientation of the coordinate system (joint coordinate system) of the joint axes.

[0073] Here, the relationship between the position and orientation of the joint coordinate system and the display unit (the position and orientation of the display coordinate system 88) is defined. Preferably, a joint coordinate system with a fixed relationship to the position and orientation of the display unit is selected, and the relationship between the position and orientation of the display coordinate system and the joint coordinate system is calculated in advance. In this embodiment, it is preferable to pre-set the relationship between the position and orientation of the joint coordinate system and the joint coordinate system that is closest to the display unit on the side facing the robot's platform 59 among the joint axes connected sequentially via links.

[0074] Furthermore, regardless of the robot's form, the position and orientation of the display coordinate system on a reference coordinate system can be calculated using methods generally accepted by the industry. For example, a simultaneous transformation matrix (the relationship between position and orientation) on the reference coordinate system is defined up to a joint axis whose relationship with the position and orientation of the display unit is fixed. Based on the homogeneous transformation matrix between this joint axis and the display unit, the position and orientation of the display coordinate system on the reference coordinate system can be determined.

[0075] When a robot is mounted on a transfer axis or other drive mechanism that alters its position and orientation, the position and orientation of the display coordinate system can be calculated by considering the amount of movement of these drive components. Furthermore, sometimes the position and orientation of the display coordinate system are defined as the relationship between the position and orientation of peripheral devices to be driven, which are separately located from the robot. In this case, the devices to be driven, which are separately located from the robot, can also be included to consider the changes in position and orientation of all driven components up to the display coordinate system, and the relative position and orientation of the display coordinate system can be calculated.

[0076] (Reference Direction Setting Unit) The reference direction setting unit 22 sets the direction of the image displayed on the specified display unit as the reference direction. The reference direction setting unit 22 sets the direction on the reference coordinate system 87 of the robot 1 as the reference direction. Alternatively, the reference direction can be set to a direction on a coordinate system suitable for setting the reference direction. Furthermore, the reference direction setting unit 22 can also use a coordinate system that is fixed relative to the space where the robot system is located to set the reference direction.

[0077] The reference orientation setting unit 22 can also use the reference coordinate system 87 to set the reference orientation based on the orientation on the robot's reference coordinate system 87. This reference coordinate system 87 is set for the robot 1 to represent the position and posture of the robot's joints or arm tips. Alternatively, the reference orientation setting unit 22 can use a coordinate system set for a specific part of the robot 1 to set the reference orientation. Furthermore, in this case, the reference orientation changes along with the driving or movement of the specific part of the robot.

[0078] Furthermore, the reference orientation setting unit 22 can also use the reference coordinate system of devices surrounding the robot, such as devices, jigs, other robots, or other drive devices arranged near the robot 1, to set the reference orientation. In this case, the reference orientation setting unit 22 can also set the reference orientation for the parts of the surrounding devices whose position and posture change due to the driving or movement of the surrounding devices. Moreover, in this case, the reference orientation changes together with the driving or movement of the surrounding devices.

[0079] By setting a reference direction in the desired coordinate system using a reference direction setting unit, the orientation of the image displayed on the display unit can be set to be easily observable by the operator. Furthermore, in the input image used for input operations, an orientation that facilitates input can be set. In addition, various implementation methods can be considered for the reference direction setting method; embodiments are shown below.

[0080] Figure 11 shows a perspective view of the reference plane for explaining the first control of the reference direction setting unit in setting the reference direction. The reference direction setting unit 22 can also set the reference direction in the robot's reference coordinate system 87 in a manner that forms a specific posture relationship with respect to the normal direction of the reference point 83 of the reference plane 82 set for the display area. For example, the reference direction setting unit 22 can also use the direction obtained by a specific homogeneous transformation matrix relative to the normal direction as the reference direction. In this example, the reference plane 82 and the display area of ​​the display unit are on the same plane. Therefore, the reference direction setting unit 22 can also set the reference direction in the robot's reference coordinate system 87 in a manner that forms a specific posture relationship with respect to the normal direction of the display area.

[0081] That is, as shown by arrow 82a, the reference direction setting unit 22 calculates the normal direction of the reference plane 82 through the reference point 83. The reference direction setting unit 22 tilts the normal direction by the direction and angle of arrow 91 shown on the reference coordinate system 87. The direction shown by arrow 81 is a direction parallel to the reference direction. Moreover, the reference direction setting unit 22 sets the direction parallel to the direction shown by arrow 81 as the reference direction shown by arrow 80.

[0082] By defining the reference direction in this way, even when the positions of the joint axes of robot 1 change due to the driving of robot 1, the reference direction can be stably set. For example, as described below, a specific direction on the robot's reference coordinate system, such as the direction of gravity, can be set as the reference direction. In this case, if the normal direction of the display area of ​​the display unit is parallel to the reference direction, there is a problem that the display phase direction cannot be defined. However, in the control that sets the reference direction based on the normal direction of the reference plane, the above problem does not exist, and the reference direction can be stably set.

[0083] Alternatively, during control operations that set a specific direction on the robot's reference coordinate system as the reference direction, the posture relationship between the display unit and the reference direction may sometimes change. In this case, the reference plane on the display area of ​​the display unit may become orthogonal or nearly orthogonal to the reference direction. This posture relationship may cause larger changes in the displayed phase direction, or repeated large changes in the displayed phase direction. As a result, the calculation of the displayed phase direction may sometimes be unstable. In this case, control can be switched to set the reference direction based on the normal direction of the reference plane.

[0084] When the robot is in motion, the image is displayed in an orientation that maintains a specific posture relationship on the robot's reference coordinate system relative to the display area of ​​the display unit, so that the operator can easily observe the image displayed on the display unit. Alternatively, input images that are easy for the operator to input can be displayed.

[0085] Figure 12 shows a schematic top view illustrating the second control for setting the reference direction using the reference direction setting unit. The reference direction setting unit 22 can also set the reference direction on a coordinate system, such as the robot's reference coordinate system 87, based on the range of the display unit's position and the range of its posture. The position of the display unit can be the position of the origin or reference point of the display coordinate system. The posture of the display unit can be the posture of the display coordinate system. For example, when the position of the display unit changes due to the robot's movement, the reference direction can also be changed based on the position of the display unit on the reference coordinate system.

[0086] The range for setting the reference direction based on the position of the display unit can also be set as the range defined by the two coordinate axes of the robot's reference coordinate system. Furthermore, the range for determining the position of the display unit can also be set as the range defined by one or three coordinate axes of the robot's reference coordinate system.

[0087] In the example shown in Figure 12, four ranges 101a, 101b, 101c, and 101d are defined based on the two coordinate axes (X-axis and Y-axis) of the robot's reference coordinate system 87. When the display unit is positioned within range 101a, the reference direction setting unit 22 can set the reference direction indicated by arrow 80a. When the display unit is positioned within range 101b, the reference direction setting unit 22 can set the reference direction indicated by arrow 80b. Similarly, when the display unit is positioned within ranges 101c and 101d, the reference direction setting unit 22 can set the reference directions indicated by arrows 80c and 80d. The reference directions for each range 101a, 101b, 101c, and 101d can be set in the reference coordinate system 87.

[0088] By setting the reference direction in this way, during robot operation, the operator can easily observe the image displayed on the display unit based on the range of its position or posture. Furthermore, it facilitates input from the image. For example, when multiple operators are working around the robot, by changing the reference direction according to their working range, each operator can easily observe the displayed image.

[0089] Figure 13 shows a first schematic diagram of a robot for explaining the third control of setting the reference direction by the reference direction setting unit. Figure 14 shows a second schematic diagram of a robot for explaining the third control of setting the reference direction by the reference direction setting unit. Figure 15 shows a third schematic diagram of a robot for explaining the third control of setting the reference direction by the reference direction setting unit. In addition to showing the schematic diagrams of the robot, Figures 13 to 15 also show enlarged views of the image 71a displayed on the display unit 61a.

[0090] In Figure 13, robot 1 is positioned above the mounting surface 105. In Figure 14, robot 1 is positioned below the mounting surface 105. In Figure 14, robot 1 is suspended. In Figure 15, the mounting surface 105 is tilted. For each robot 1, the base portion 14 is fixed to the mounting surface 105. Arrow 90 indicates the direction of gravity (below the vertical direction).

[0091] The reference direction setting unit 22 can also set the reference direction to the direction of gravity, the opposite direction of gravity, or the direction after rotating the direction of gravity in a specific posture. That is, the reference direction setting unit 22 sets the direction specified in the reference coordinate system 87 as the reference direction. In the example shown in Figures 13 to 15, the reference direction is set to the vertically upward direction (the direction opposite to the direction shown by arrow 90). Each display unit 61a is arranged in various directions. However, in the direction of any display unit 61a, the display control unit 27 displays the image 71a with the vertically upward direction as the upper side of the image.

[0092] The third control is effective when the normal direction of the display area (reference plane) of the display unit forms an angle greater than a certain degree with the direction of gravity. The reference direction specified in this control is based on the direction of gravity, and therefore is a direction that is easily and intuitively distinguishable by the operator. For example, the display device can also be controlled in such a way that, regardless of the angle of the joint axis of the robot on which the display unit is installed, the image is displayed at a specific angle of 0 degrees or more relative to the horizontal direction. Furthermore, when the display unit includes a touch panel for teaching operation, it can also be controlled to display input images such as buttons at a specific angle relative to the horizontal direction.

[0093] By setting the reference direction based on the direction of gravity, operators can easily observe the images displayed on the screen without relying on the robot's setup or posture. Furthermore, operators can easily input data onto the input images.

[0094] Figure 16 shows a perspective view of the reference plane for explaining the fourth control of setting the reference direction by the reference direction setting unit. The angle between the reference direction and the normal direction of the reference plane changes due to the robot's actuation, and the angle between the reference direction and the normal direction of the reference plane is sometimes less than a certain threshold. For example, due to changes in the position and posture of robot 1, the angle θ1 between the reference direction indicated by arrow 80 and the normal direction of the reference plane 82 indicated by arrow 82a is sometimes less than a certain threshold.

[0095] In this situation, the reference orientation setting unit 22 can also maintain the reference orientation before the angle θ1 is less than a specific threshold. That is, when the position and posture of the robot change and the angle θ1 is less than a specific threshold, the reference orientation setting unit 22 can also maintain the previous reference orientation.

[0096] When the orientation relationship between the display area of ​​the display unit and the reference direction changes due to the robot's drive, and the reference direction is close to the normal direction of the reference plane 82, it is sometimes impossible to stably calculate the orientation (display phase direction) of the image in the display unit. In this case, by maintaining the reference direction without changing it, the orientation of the image can be specified to display the image.

[0097] Figure 17 shows a perspective view of the reference plane for explaining the fifth control of setting the reference direction by the reference direction setting unit. The reference direction setting unit 22 can also set the reference direction based on a position specified by the operator on the display area of ​​the display unit. In this example, the reference plane 82 is set to be on the same plane as the display area of ​​the display unit.

[0098] The operator changes the position and posture of robot 1 by using reference plane 82 to achieve the desired posture. The display control unit 27 displays reference point 83 on the display area of ​​the display unit. In the case where the display unit includes a touch panel, the operator designates point 102 by pressing the desired point on the display area. The reference direction setting unit 22 calculates the direction from reference point 83 (indicated by arrow 81) towards point 102. The reference direction setting unit 22 sets the direction parallel to arrow 81 as the reference direction indicated by arrow 80. The reference direction can be set in the robot's reference coordinate system.

[0099] In the fifth control, the reference direction is set to the direction within the plane of the display area of ​​the display unit. Here, when the desired reference direction intersects with the plane of the display area of ​​the display unit, the operator can change the position and posture of the robot to change the direction of the display area of ​​the display unit.

[0100] In the fifth control, a reference direction can be easily set to align the image orientation with a direction convenient for the operator, based on the robot's state. This control allows for easy setting of a reference direction that makes it easy for the operator to observe the display. Furthermore, it facilitates easy input by the operator onto the input image.

[0101] Furthermore, depending on the shape of the display area, the display area and the reference plane are sometimes not on the same plane. For example, the display area of ​​the display unit is sometimes formed as a curved surface. In this case, the reference direction setting unit can project a designated point specified by the operator onto the reference plane in a predetermined direction. Moreover, the reference direction can also be set based on the point projected onto the reference plane.

[0102] In the above embodiments, the direction from the reference point toward the designated point is set as the reference direction, but it is not limited to this form. The operator can set any direction based on the designated point specified in the display area as the reference direction. For example, in the first control of setting the reference direction, when setting the posture of the reference direction relative to the normal direction of the reference plane, the control of specifying the designated point on the display area can also be performed.

[0103] (Displays the phase calculation section) Figure 18 shows a perspective view of the reference plane for explaining the first control of calculating the display phase direction. The display phase calculation unit 25 calculates the reference direction of the display direction on the display area of ​​the display unit, i.e., the display phase direction (the direction shown by arrow 84), by projecting the reference direction indicated by arrow 80 onto the reference plane 82, which is the display area of ​​the display unit. Figure 18 shows the case where the reference direction indicated by arrow 80 is orthogonal to the normal direction of the reference plane 82 indicated by arrow 82a. In this case, the display phase calculation unit 25 can project the reference direction in any direction as shown by arrow 92. The display phase calculation unit 25 projects the reference direction onto the reference plane 82. Furthermore, the display phase direction indicated by arrow 84 can be defined by a reference point 83.

[0104] Figure 19 shows a perspective view of the reference plane for explaining the second control that calculates the display phase direction. Figure 19 shows a case where the reference direction indicated by arrow 80 is not orthogonal to the normal direction of the reference plane 82 indicated by arrow 82a and is tilted. The display phase calculation unit 25 calculates a direction parallel to the reference direction passing through the reference point 83 as shown by arrow 81, based on the reference direction indicated by arrow 80.

[0105] The display phase calculation unit 25 projects a line parallel to the reference direction passing through the reference point 83 towards the normal direction of the display area, as shown by arrow 92. The display phase calculation unit 25 also projects a line parallel to the reference direction passing through the reference point 83 towards the reference plane 82. Through this control, the display phase direction, which serves as the reference for the display phase on the display coordinate system, can be defined, as shown by arrow 84.

[0106] Alternatively, the display phase calculation unit 25 calculates the direction in which the reference direction indicated by arrow 80 is projected onto the reference plane 82 in the direction normal to the display area. Furthermore, the display phase calculation unit 25 calculates the direction passing through the reference point 83 in a manner parallel to the projection direction. The display phase calculation unit 25 can define this direction as the display phase direction.

[0107] Figure 20 shows a schematic diagram of a robot for explaining the third control for calculating the display phase direction. Figure 21 shows a top view of the reference plane for explaining the third control for calculating the display phase direction. Referring to Figures 3, 20, and 21, the display phase calculation unit 25 projects a reference direction onto the reference plane 82, which serves as the reference for the display area of ​​the display unit 61a. Furthermore, the display phase calculation unit 25 can also calculate the display phase direction by rotating a specific angle, i.e., an offset angle, around the reference point 83, which serves as the reference point for the display on the display area of ​​the display unit.

[0108] Here, the reference direction, as indicated by arrow 80, is defined as the vertically upward direction. The phase calculation unit 25 calculates the direction parallel to the reference direction of the reference plane 82 by projecting the reference direction onto the reference plane 82. As indicated by arrow 81, the phase calculation unit 25 calculates the direction parallel to the reference direction passing through the reference point 83.

[0109] The processing unit 21 of this embodiment includes an offset angle setting unit 26 that sets a specific angle, or offset angle, centered on a reference point on the display area. The offset angle setting unit 26 sets the offset angle according to the operator's operation. For example, the operator inputs the offset angle θ2 using a teaching operation panel. Furthermore, the offset angle setting unit 26 can set the offset angle θ2 relative to the direction indicated by the arrow 81 that projects the reference direction onto the reference plane.

[0110] Next, the display phase calculation unit 25 rotates the direction indicated by arrow 81 around the reference point 83 by a specific rotation angle, namely an offset angle θ2. The display phase calculation unit 25 rotates the direction of the projection reference direction within the reference plane 82. Furthermore, the display phase calculation unit 25 calculates the display phase direction indicated by arrow 84. The display control unit 27 controls the orientation of the image on the display area of ​​the display unit 61a based on the display phase direction calculated by the display phase calculation unit 25 and the reference point 83.

[0111] As shown in Figure 20, the image 71a displayed in the display unit 61a is tilted by an offset angle θ2 relative to the direction in which the reference direction is projected onto the reference plane. This control allows for easy orientation shifting of the image displayed in the display area of ​​the display unit for the operator to view. The image can be displayed in a manner that has a specific phase relationship with a specific direction based on the reference direction. When the operator observes the display unit from a specific direction, information is displayed corresponding to that specific direction, making it easy for the operator to observe the information. Furthermore, when the display device has an input function, the operator can easily perform input operations on the input image.

[0112] Figure 22 shows a top view of the reference plane for explaining the fourth control of setting the display phase direction. In this example, the reference plane 82 and the display area of ​​the display unit are arranged in the same plane. The offset angle setting unit 26 is configured so that the offset angle θ2 can be set by an operator on the display area of ​​the display unit.

[0113] The display control unit 27 displays the direction in which the current reference direction is projected onto the reference plane 82, as shown by arrow 81. The operator presses the display area of ​​the display unit to specify the designated point 103. The offset angle setting unit 26 calculates the offset angle θ2 relative to the direction shown by arrow 81. The display phase calculation unit 25 sets the display phase direction based on the offset angle θ2. The display control unit 27 controls the orientation of the image on the display area of ​​the display unit 61a based on the display phase direction calculated by the display phase calculation unit 25 and the reference point 83.

[0114] Furthermore, when the display area of ​​the display unit is curved, the operator can also set the offset angle based on the point obtained by projecting the point of contact on the display area onto the reference plane. For example, the offset angle can be calculated based on the point obtained by projecting a specified point designated by the operator onto the normal direction of the reference plane. Alternatively, it is not necessary to implement control to rotate the image based on the offset angle.

[0115] (Display Control Department) The display control unit 27 controls the image displayed in the display area of ​​the display unit. Furthermore, when an input image is displayed on the display unit, the display control unit 27 controls the orientation of the input image for display. Based on the axis positions and reference directions of the robot's joint axes, the display control unit 27 controls the orientation of the image on the display area of ​​the display unit to display an image that forms a specific posture relationship with the reference direction.

[0116] Furthermore, the display control unit 27 controls the position and orientation of the image on the display area of ​​the display unit to display based on the axis position of each axis of the robot's joint axis, the reference direction, the reference point on the display coordinate system, and the display phase direction on the display coordinate system calculated by the display phase calculation unit 25.

[0117] Furthermore, robots are sometimes mounted on devices with transfer axes or other drive mechanisms that change the robot's position and posture. In such cases, the processing unit 21 preferably displays images based on the axis positions of the robot's joint axes and the amount of movement of these drive components.

[0118] The display control unit 27 controls the orientation of the image on the display area of ​​the display unit. When the posture of the display unit changes due to the robot's driving, the image on the display area of ​​the display unit can be displayed in the desired orientation. Alternatively, an input image can be displayed in the desired orientation for input operations.

[0119] Next, a specific example of the image displayed by the display control unit 27 will be described. Figure 23 shows a schematic side view of the robot in its first position and posture. Here, regarding the image displayed on the display unit 61a, the case where the reference direction indicated by arrow 80 is the vertical upward direction will be explained. This reference direction can be set, for example, in the first or third control of setting the reference direction in this embodiment. On the display unit 61a, the display phase direction indicated by arrow 84 is in the same direction as the reference direction indicated by arrow 80. That is, the display phase direction indicated by arrow 84 is upward in the vertical direction.

[0120] Figure 24 shows a schematic diagram of the robot when it is driven from the first position and posture. Figure 25 shows a schematic diagram of a comparative example robot when it is driven from the first position and posture. Here, the axial position of the J2 axis 52 between the rotating base 13 and the lower arm 12 changes. As shown by arrow 93, the lower arm 12 rotates and its posture changes.

[0121] Figure 24 shows the state of the image on the display unit 61a when the processing unit 21 of this embodiment performs display control. The display control unit 27 controls the direction of the image on the display area of ​​the display unit 61a based on the axis position and reference direction of each joint axis of the robot 1, in a manner that displays an image with a specific posture relationship to the reference direction. The display phase direction indicated by arrow 84 is maintained parallel to the reference direction. That is, the display phase direction is maintained in a vertically upward direction.

[0122] In the comparative example shown in Figure 25, the display control of this embodiment is not implemented. In display unit 61a, the display phase direction indicated by arrow 85 rotates together with the rotation of the lower arm 12. As a result, the image displayed in display unit 61a is tilted. It is difficult for operators to observe the image displayed in display unit 61a.

[0123] In contrast, referring to FIG24, by implementing display control through the processing unit 21 of this embodiment, even if the axial position of the joint axis changes, the display phase direction indicated by arrow 84 will still be upward in the vertical direction. Therefore, the state of easy observation of the image displayed in the display unit 61a is maintained.

[0124] Figure 26 shows a schematic side view of the robot in its second position and posture. Figure 27 shows a schematic top view of the robot in its second position and posture. In this example, the display area (reference plane) of the display unit 63b is parallel to the horizontal direction. For the display unit 63b, the reference direction indicated by arrow 80 is set to the horizontal direction. This reference direction can be set in the first control, second control, or third control for setting the reference direction in this embodiment.

[0125] In display unit 63b, the display phase direction indicated by arrow 84 faces the horizontal direction. Furthermore, as indicated by arrow 94, the operator observes display unit 63b from above the robot 1. The display phase direction indicated by arrow 84 is the direction from which the operator can easily observe the image.

[0126] Figure 28 shows a schematic diagram of the robot when it is driven from the second position and posture. Figure 29 shows a schematic diagram of a comparative example robot when it is driven from the second position and posture. In this example, as indicated by arrow 95, the rotating base 13 rotates about axis J1 51.

[0127] In the example shown in Figure 28, display control is implemented using the processing unit 21. The display control unit 27 controls the orientation of the image on the display area of ​​the display unit to display the image in a specific posture relationship with the reference direction, based on the axis position and reference direction of each joint axis of the robot. The display phase direction indicated by arrow 84 is parallel to the reference direction indicated by arrow 80.

[0128] In the comparative example shown in Figure 29, the display control of this embodiment is not implemented. The display phase direction indicated by arrow 85 on the display unit 63b changes direction along with the rotation of the upper arm 11. As a result, when the operator observes the display unit 63b, the image displayed on the display unit 63b is tilted. It is difficult for the operator to observe the image displayed on the display unit 63b.

[0129] Referring to FIG28, by implementing display control using the processing unit 21 of this embodiment, even if the axis position of the joint axis changes, the display phase direction indicated by arrow 84 remains parallel to the reference direction. The image displayed in the display unit 63b is displayed in the same direction even when the robot is driven. Therefore, it is possible to maintain a state where the operator can easily observe the image displayed in the display unit 63b.

[0130] Figure 30 shows a schematic side view of the robot in its third position and posture. The upper arm 11, to which the display unit 63b is fixed, is tilted relative to the horizontal direction. For the display unit 63b, the reference direction indicated by arrow 80 is set in a horizontal direction. This reference direction can be set in the second or third control of setting the reference direction in this embodiment. In this example, the normal direction of the display area (reference plane) of the display unit 63b is not orthogonal to the reference direction. Furthermore, the operator observes the display unit 63b in the direction indicated by arrow 96.

[0131] Figure 31 illustrates the display phase direction of the robot when driven from the third position and posture. Referring to Figures 30 and 31, in this example, as indicated by arrow 95, the rotating base 13 rotates around the J1 axis 51. In the example shown in Figure 31, display control is implemented using the processing unit 21. The display control unit 27 controls the orientation of the image on the display area of ​​the display unit to display an image that has a specific posture relationship with the reference direction, based on the axial position and reference direction of each axis of the robot's joints.

[0132] Even when the upper arm 11 rotates as shown by arrows 95a and 95b, the display phase direction indicated by arrow 84 remains in the direction that projects the reference direction onto the display area (reference plane) of the display unit 63b. The display phase direction indicated by arrow 84 is the direction in which the operator can easily observe the image. When the operator observes the display unit 63b in the direction indicated by arrow 96, the image is displayed in the same direction.

[0133] Figure 32 illustrates the display phase direction of a comparative example of the robot during its third position and posture actuation. In this comparative example, the display control of this embodiment is not implemented. The display phase direction indicated by arrow 85 on display unit 63b changes direction along with the rotation of the upper arm 11 indicated by arrows 95a and 95b. As a result, when the operator observes display unit 63b, the image displayed on display unit 63b is tilted. It is difficult for the operator to observe the image displayed on display unit 63b.

[0134] Referring to FIG31, the display control is implemented by the processing unit 21 of this embodiment, so that even if the axis position of the joint axis changes, the display phase direction remains in a direction that is easy for the operator to observe the image. Therefore, the state in which the image displayed on the display unit 63b can be easily observed by the operator can be maintained.

[0135] In the above embodiments, a display coordinate system, a reference plane, and a reference point are set for the display area of ​​the display unit to calculate the display phase direction, but this is not limited to this configuration. The processing unit may also omit the display coordinate system, reference plane, and reference point. The processing unit can calculate the display phase direction based on the reference direction through arbitrary control. For example, when the reference direction is parallel to the display area of ​​the display unit, the processing unit may also set the reference direction as the display phase direction.

[0136] Furthermore, the display control unit can also perform image processing on the image generated on the reference plane. For example, it can perform arbitrary image conversion operations such as image magnification, image reduction, or specific image conversion. Moreover, the display control unit can display the converted image on the display unit. For example, as described above, when the display area of ​​the display unit is curved, by projecting the image on the reference plane onto the display area of ​​the display unit, an image can be generated that is displayed in the display area.

[0137] (Example of an image displayed in the display section) Next, an example of an image displayed on the display area of ​​the display unit via the display control unit will be described. Here, an example of an image displayed in the display unit 61a will be described.

[0138] Figure 33 shows the fourth image displayed in the display unit. Figure 34 shows the fifth image displayed in the display unit. The fourth image 71c is an image showing the robot's driving state. The fifth image 72b is an input image for inputting the driving speed of the joint axis with the display unit or the desired joint axis around the joint axis. Furthermore, sometimes a joint axis is driven together with other joint axes besides that one joint axis. In this case, when the driving speed around one joint axis is changed, the driving speed of all joint axes can also be adjusted so that the position and posture of the robot's front end do not change. Also, the movement speed of the robot's front end in the executed motion program can be changed by the input image displayed in the display unit. The display control unit 27 can display information or input images in the display area of ​​the display unit using at least one of the following: text, graphics, symbols, colors, and patterns.

[0139] Furthermore, the display control unit 27 can also display the direction of the image by means of at least one of text, graphics, symbols, colors, and patterns. In the fourth image 71c and the fifth image 72b, an image 73 is displayed that represents the upper side of the image.

[0140] In this way, the display control unit generates images in the display area in a manner that clearly indicates the image orientation, thereby making it easy for operators to identify the orientation of the image in the display area of ​​the display unit. Furthermore, when displaying input images on the display unit, it can improve the operability for operators.

[0141] Figure 35 shows the sixth image displayed in the display unit. Figure 36 shows the seventh image displayed in the display unit. In the sixth image 71d, the word "upper" is displayed to indicate the upper side of the image. In the seventh image 71e, the upper side of the image is indicated by a recessed portion of the frame surrounding the information indicating the operating status.

[0142] Figure 37 shows the eighth image displayed in the display unit. Figure 38 shows the ninth image displayed in the display unit. In the eighth image 71f and the ninth image 71g, a pattern indicating the upper side is displayed at the top. Furthermore, in the ninth image 71g, the pattern indicating the upper side is displayed in a color different from the background color of the display area. In this way, the display control unit 27 can generate images in various ways to represent the image direction.

[0143] The above embodiment describes an image representing the upper side, but is not limited to this form. The display control unit can display the image in a manner that clearly indicates its orientation. For example, the display control unit can also display text representing the lower side of the image.

[0144] Figure 39 shows the 10th image displayed in the display unit. Figure 40 shows the 11th image displayed in the display unit. The display control unit 27 can also perform control by changing the size of a specific portion of the image on the display area of ​​the display unit 61a according to the moving speed of the display unit 61a driven by the accompanying robot. For example, the display control unit 27 can also display a smaller specific image when the moving speed of the display unit 61a is small, and display a larger specific image when the moving speed of the display unit 61a is large.

[0145] In the tenth images 71ha and 71hb of Figure 39, the number of times the operation was performed is displayed on the display unit 61a. The display control unit 27 can change the size of the image showing the number of operations, as shown by arrow 97, based on the moving speed of the display unit 61a. For example, the display control unit 27 can calculate the moving speed of the display coordinate system as the moving speed of the display unit 61a. The display control unit 27 obtains the position and time of the origin of the display coordinate system at each predetermined time interval. Based on the position and time of the origin of the display coordinate system, the display control unit 27 can calculate the moving speed of the origin of the display coordinate system.

[0146] When the moving speed of the display unit 61a is less than a certain threshold, the display control unit 27 can display a smaller image of the number of executions, as shown in image 71ha. When the moving speed of the display unit 61a is above the certain threshold, the display control unit 27 can display a larger image of the number of executions, as shown in image 71hb. By implementing this control, the desired portion of the image is displayed larger when the robot's driving speed is higher, making it easier for the operator to observe the displayed image.

[0147] Figure 40, images 72ca and 72cb (11th images) are input images used for operators to perform input operations. In images 72ca and 72cb, specific setting values ​​can be changed by pressing the "+" or "-" button. The display control unit 27 can change the size of the image representing the setting value, as shown by arrow 98, according to the movement speed of the display unit 61a. Furthermore, the display control unit 27 can change the size of the buttons.

[0148] When the movement speed of the display unit 61a is less than a certain threshold, as shown in image 72ca, the display control unit 27 can display the image of the set value in a larger size and the image of the button in a smaller size. When the movement speed of the display unit 61a is greater than or equal to the certain threshold, as shown in image 72ca, the display control unit 27 can display the image of the set value in a smaller size and the image of the button in a larger size. Through this control, even if the robot's driving speed increases, the operator can still press the button stably because the button image is enlarged.

[0149] In the robot system of this embodiment, when the display device has an input function, the operator can perform input operations during robot operation. By implementing control that changes the image size according to the movement speed of the display unit, the operator can easily observe the image or easily perform input operations. For example, when the movement speed of the display unit increases, the button image is displayed larger, thereby making it easier for the operator to perform input operations. Furthermore, when input operations can be performed from the display device during robot operation, for safety reasons, it is preferable to set a limit on the movement speed of the robot or the drive speed of the joint axes that allows input operations. For example, it is preferable that input operations can be performed when the robot's movement speed or the drive speed of the joint axes is below a certain speed. Also, the setting value for enabling input operations can be changed according to the robot's movement speed or the drive speed of the joint axes.

[0150] In each of the above controls, the order of steps can be appropriately changed without altering the function or effect. The above embodiments can be appropriately combined. In the above figures, identical or equivalent parts are labeled with the same symbol. Furthermore, the above embodiments are illustrative and not limiting of the inventors. Also, the embodiments include modifications to the embodiments shown in the claims.

[0151] 1: Robot 2: Hands 4: Control device 6: Robotic System 11: Upper arm 12: Lower arm 13: Rotating base 14: Base section 15: Wrist 16: Flange 18: Rotational Position Detector 19: Robot drive motor 21: Processing Department 22: Reference Direction Setting Unit 23: Coordinate System Setting Department 24: Coordinate System Calculation Department 25: Display phase calculation unit 26: Offset Angle Setting Unit 27: Display Control Unit 29: Instruction Processing Unit 41: Action Programming 42: Memory Department 43: Action Command Section 44: Hand drive circuit 45: Robot drive circuit 51: J1 axis 52: J2 axis 53: J3 axis 54: J4 axis 55: J5 axis 56: J6 axis 59:pedestal 60, 61a, 61b, 61c, 62, 63a, 63b, 64, 65, 66, 67: Display section 71a,71b,71c,71d,71e,71f,71g,71ha,71hb: Images 72a, 72b, 72ca, 72cb: Images 80, 80a, 80b, 80c, 80d, 81, 82a, 84, 91, 92, 93, 94, 95, 95a, 95b, 96, 97, 98: Arrows 82: Reference Plane 83: Benchmark Point 87: Reference Coordinate System 88: Display Coordinate System 101a~101d: Range 102, 103: Designated points 105: Setting surface 111,112,113,114,115,116: Steps R1, R2: Rotation axes X: axis Y: axis θ1: Angle θ2: Offset angle

Claims

1. A robot system, characterized by comprising: a robot including a plurality of joint axes; a display unit disposed on the body portion of the robot; a display control unit controlling an image displayed in a display area of ​​the display unit; a reference direction setting unit setting a reference direction, which is a reference direction used when the direction of the image displayed in the display unit is specified; and a coordinate system setting unit setting a coordinate system, i.e., a display coordinate system, for the display area of ​​the display unit, a reference plane serving as a reference for the display area of ​​the display unit and set on the display coordinate system, and a reference point serving as a reference for display in the display area of ​​the display unit and set on the display coordinate system. The coordinate system calculation unit calculates the position and orientation of the display coordinate system based on the axial positions of each joint axis of the robot; and the display phase calculation unit calculates the reference direction, i.e., the display phase direction, which becomes the display direction on the display area of ​​the display unit by projecting the reference direction onto the reference plane on the display coordinate system; and the display control unit controls the direction of the image on the display area of ​​the display unit for display based on the display phase direction calculated by the display phase calculation unit and the reference point.

2. A robot system, characterized by comprising: a robot including a plurality of joint axes; a display unit disposed on the body portion of the robot; a display control unit controlling an image displayed in a display area of ​​the display unit; a reference direction setting unit setting a reference direction, which is a reference direction used when the direction of the image displayed in the display unit is specified; and a coordinate system setting unit setting a coordinate system, i.e., a display coordinate system, for the display area of ​​the display unit, a reference plane serving as a reference for the display area of ​​the display unit and set on the display coordinate system, and a reference point serving as a reference for display in the display area of ​​the display unit and set on the display coordinate system. The coordinate system calculation unit calculates the position and orientation of the display coordinate system based on the axial positions of each joint axis of the robot; and the display phase calculation unit calculates the direction that serves as the reference for the display direction on the display area of ​​the display unit, i.e., the display phase direction, by projecting a reference direction onto a reference plane on the display coordinate system and rotating the reference direction around the reference point by a specific angle, i.e., an offset angle; and the display control unit controls the orientation of the image on the display area of ​​the display unit for display based on the display phase direction calculated by the display phase calculation unit and the reference point.

3. The robot system of claim 1 or 2, wherein the display unit has an input function, that is, displays an input image for inputting information, and inputs information by means of an operation corresponding to the input image, and the display control unit controls the direction of the input image to display the input image.

4. The robot system of claim 1 or 2, wherein the aforementioned reference direction setting unit sets the reference direction on the reference coordinate system of the robot in such a way that it forms a specific posture relationship with the normal direction of the reference plane.

5. The robot system of claim 1 or 2, wherein the reference direction setting unit sets the reference direction in the reference coordinate system of the robot according to the range where the display unit is located.

6. The robot system of claim 1 or 2, wherein the aforementioned reference direction setting unit sets the gravity direction, the direction opposite to the gravity direction, or the direction that causes the gravity direction to rotate in a specific posture as the reference direction.

7. The robot system of request item 1 or 2, wherein the reference direction setting unit sets the reference direction based on the position specified by the operator on the display area of ​​the display unit.

8. In the robot system of claim 1 or 2, the aforementioned reference direction setting unit maintains the reference direction before the angle between the reference direction and the normal direction of the reference plane changes due to the drive of the robot and the angle between the reference direction and the normal direction is less than a certain threshold.

9. The robot system of claim 2, which includes an offset angle setting unit, wherein the offset angle setting unit sets a specific angle, i.e., an offset angle, centered on the aforementioned reference point based on a position specified by the operator on the display area of ​​the aforementioned display unit.

10. The robot system of claim 3, wherein when information is input on the aforementioned display unit, the display unit displays an image that accepts input of specific information to enable input operations by the operator, and switches to an image that the operator can perform input operations by inputting specific information.

11. The robot system of claim 3, wherein the aforementioned display unit is configured such that at least one of the permissions for performing input operations and the permissions for displaying information can be changed by a specific input operation of the operator.

12. The robot system of any one of claims 1, 2, and 9, wherein the display control unit displays an image on the display area of ​​the display unit in a manner that indicates the direction of the image by means of at least one of text, graphics, symbols, colors, and patterns.

13. A robot system according to any one of claims 1, 2, and 9, wherein the display control unit controls the display by changing the size of an image in a specific portion of the display area of ​​the display unit according to the moving speed of the display unit driven by the robot.

14. The robot system of any one of claims 1, 2, and 9, wherein the aforementioned display unit can be attached to or detached from the main body of the robot.

15. A robot system, characterized by comprising: a robot including a plurality of joint axes; a display unit disposed on the body portion of the robot; a display control unit controlling an image displayed in a display area of ​​the display unit; a reference direction setting unit setting a reference direction, which is used as a reference direction when the direction of the image displayed in the display unit is specified; and the display control unit controls the direction of the image on the display area of ​​the display unit to display the image based on the axis position of each joint axis of the robot and the reference direction, in a manner that forms a specific posture relationship with the reference direction; the reference direction setting unit sets the reference direction based on a position specified by an operator on the display area of ​​the display unit.

Images