Robot system provided with a robot having a display section

Abstract

A robot system includes a display control section that controls an image displayed in a display region of a display section, and a reference direction setting section that sets a reference direction for determining an orientation of the image. The display control section controls the orientation of the image of the display section based on an axis position of each joint axis of the robot and the reference direction, to perform display in a predetermined posture relationship with respect to the reference direction.

Description

Technical Field

[0001] This invention relates to a robot system equipped with a robot having a display unit. Background Technology

[0002] The robot system includes a robot and working tools installed on the robot. The robot can perform predetermined tasks while changing its position and posture. The robot consists of multiple components such as an arm. In the prior art, robot systems with display devices installed on the components of the robot's main body are known. For example, Japanese Patent Application Publication No. 2018-529488 discloses a holding device for medical use, which includes a display unit composed of a ring of LED (Light Emitting Diode) components arranged on the joint axis, the display unit indicating the direction of movement, etc.

[0003] Furthermore, in the prior art, it is known that cameras can obtain images without tilting even when the camera is tilted (e.g., Japanese Patent Application Publication No. 4-81081).

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Publication No. 2018-529488

[0007] Patent Document 2: Japanese Patent Application Publication No. 4-81081 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] Robots change position and posture to move their work tools. When a display unit is integrated into the robot's components, the position and posture of the display unit change along with the components when the robot is driven. As a result, for the operator, the orientation of the image displayed on the display unit sometimes changes, making it difficult to observe the image.

[0010] Methods for solving problems

[0011] The robot system disclosed herein includes a robot comprising multiple joint axes and a display unit disposed on the main body of the robot. The robot system includes: a display control unit that controls an image displayed in a display area of ​​the display unit; and a reference direction setting unit that sets a reference direction, i.e., the orientation of the image displayed in the display unit, based on the axial positions of each joint axis of the robot and the reference direction, to display an image in a predetermined posture relative to the reference direction.

[0012] Invention Effects

[0013] According to the present disclosure, a robot system can be provided which has a display unit installed on the main body of the robot, and the image displayed on the display unit can be easily observed when the position and posture of the robot change. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the robot system in the implementation method.

[0015] Figure 2 This is a schematic diagram illustrating the joint axes of the robot in the implementation method.

[0016] Figure 3 This is a block diagram of the robot system in the implementation method.

[0017] Figure 4 This is a schematic diagram of a robot illustrating the flat display unit installed on the robot.

[0018] Figure 5 This is a schematic diagram of a robot illustrating the curved display section installed on the robot.

[0019] Figure 6 It is the first image displayed on the display screen.

[0020] Figure 7 It is the second image displayed on the display unit.

[0021] Figure 8 It is the third image displayed on the display unit.

[0022] Figure 9 This is a flowchart of the control for displaying images on the display unit in the implementation method.

[0023] Figure 10 It is a planar diagram that illustrates and displays the coordinate system, reference point, reference plane, and phase direction.

[0024] Figure 11 This is a schematic diagram illustrating the first control for setting the reference direction.

[0025] Figure 12 This is a schematic diagram illustrating the second control for setting the reference direction.

[0026] Figure 13 This is a first schematic diagram illustrating a robot with a third control that sets a reference direction.

[0027] Figure 14 This is a second schematic diagram illustrating a robot with a third control that sets a reference direction.

[0028] Figure 15 This is a third schematic diagram illustrating the robot under third control that sets the reference direction.

[0029] Figure 16 This is a schematic diagram illustrating the fourth control that sets the reference direction.

[0030] Figure 17 This is a schematic diagram illustrating the fifth control that sets the reference direction.

[0031] Figure 18 This is a schematic diagram illustrating the first control for setting the display phase direction.

[0032] Figure 19 This is a schematic diagram illustrating the setting of the second control for the phase direction of the display.

[0033] Figure 20 This is a schematic diagram illustrating a robot controlled by a third controller that sets and displays the phase direction.

[0034] Figure 21 This is a schematic diagram illustrating the setting of the third control for the phase direction of the display.

[0035] Figure 22 This is a schematic diagram illustrating the fourth control for setting the display phase direction.

[0036] Figure 23 This is a schematic diagram showing the display phase direction of the display unit when the robot is in its first position and posture.

[0037] Figure 24 This is a schematic diagram showing the phase direction of the robot when it is driven from its first position and posture and is under displayed control.

[0038] Figure 25 This is a schematic diagram showing the phase direction when the robot is driven from a first position and posture without any displayed control.

[0039] Figure 26 This is a schematic diagram of the robot in its second position and pose.

[0040] Figure 27 This is a schematic plan view showing the display phase direction of the display unit when the robot is in its second position and posture.

[0041] Figure 28 It is a schematic plan view showing the phase direction when the robot is driven from a second position and posture and is under displayed control.

[0042] Figure 29 It is a schematic plan view showing the phase direction of the robot when it is driven from a second position and posture without explicit control.

[0043] Figure 30 This is a schematic diagram of the robot in its third position and pose.

[0044] Figure 31 This is a schematic front view of the display phase direction of the display unit when the robot is driven from a third position and posture and is controlled by the display.

[0045] Figure 32 This is a schematic front view of the display phase direction of the display unit when the robot is driven from a third position and posture, without any displayed control.

[0046] Figure 33 It is the fourth image displayed on the display unit.

[0047] Figure 34 It is the fifth image displayed on the display unit.

[0048] Figure 35 It is the sixth image displayed on the display unit.

[0049] Figure 36 It is the seventh image displayed on the display unit.

[0050] Figure 37 This is the eighth image displayed on the display unit.

[0051] Figure 38 It is the ninth image displayed on the display unit.

[0052] Figure 39 This is the tenth image displayed on the display screen.

[0053] Figure 40 It is the eleventh image displayed on the display panel. Detailed Implementation

[0054] Reference Figures 1 to 40 The robot system described in this embodiment will now be explained. The robot system of this embodiment includes a robot with multiple joint axes and a display device for displaying predetermined information. The display unit of the display device is disposed on the main body of the robot.

[0055] (Robot System)

[0056] Figure 1 This is a schematic diagram of the first robot system of this embodiment. The robot system 6 of this embodiment has the function of transporting workpieces. The robot system 6 includes a robot device comprising a manipulator 2 as a working tool (end-effector) and a robot 1 for changing the position and posture of the manipulator 2. The robot system 6 includes a control device 4 for controlling the robot 1 and the manipulator 2.

[0057] The robot 1 of this embodiment includes a base portion 14 fixed to a mounting surface and a rotating base 13 supported on the base portion 14. The rotating base 13 is configured to rotate relative to the base portion 14. The robot 1 includes an upper arm 11 and a lower arm 12. The lower arm 12 is rotatably supported on the rotating base 13 via a joint. The upper arm 11 is rotatably supported on the lower arm 12 via a joint. Furthermore, the upper arm 11 rotates about a rotation axis parallel to the extending direction of the upper arm 11. The robot 1 includes a wrist 15 connected to the end of the upper arm 11. The wrist 15 is rotatably supported on the upper arm 11 via a joint. The wrist 15 includes a flange 16 that rotates about a rotation axis along the extending direction of the wrist 15. A robotic arm 2 is fixed to the flange 16.

[0058] Figure 2 This is a schematic diagram of a robot illustrating the joint axes of the robot according to this embodiment. In this invention, a joint axis is an axis connecting the links constituting the robot and the joint portions of the links, and is the part that changes the positional or angular relationship between the links. By changing the position of each axis (the position is an angle in the case of a rotary axis, and the displacement length in the case of a linear axis), the positional relationship between the links can be changed. As a result, at least one of the position and posture of the robot's front end can be changed. Alternatively, an actuator for moving the position of the joint axis can be provided at a location different from the part that constitutes the joint axis.

[0059] Reference Figure 1 and Figure 2 The robot 1 in this embodiment consists of 6 joint axes. These 6 joint axes, starting from the base 59 side of the base portion 14 of the robot 1, are as follows: the first joint axis is J1 axis 51, the second joint axis is J2 axis 52, the third joint axis is J3 axis 53, the fourth joint axis is J4 axis 54, the fifth joint axis is J5 axis 55, and the sixth joint axis is J6 axis 56.

[0060] like Figure 2 As shown on the right, axes J1 51, J4 54, and J6 56 constitute a rotation axis R1 that rotates about the connecting rod that connects the joint axis and the joint axis. Additionally, axes J2 52, J3 53, and J5 55 constitute a rotation axis R2 that rotates about a direction orthogonal to the connecting rod that connects the joint axis and the joint axis.

[0061] In this embodiment, the robotic arm 2 grasps or releases a workpiece. The robotic arm 2 grasps the workpiece by closing its opposing claws. The working tool is not limited to a robotic arm that grasps a workpiece. Any working tool can be mounted on the robot depending on the task being performed by the robot system. For example, in the case of arc welding performed by the robot system, a welding torch can be mounted on the robot.

[0062] Figure 2This is a simplified explanatory diagram illustrating the structure of the axes of robot 1. Furthermore, if the origin of a joint axis is set as the origin of the coordinate system (joint coordinate system) for each axis and the point connecting links, then the position of the axis origin is represented by its position in a spatial coordinate system. Additionally, the coordinate system set in space that serves as the reference for representing the position and posture of the robot's joint axes, links, front end, and working tools mounted on the front end will be referred to as the robot's reference coordinate system 87. The reference coordinate system 87 is a coordinate system in space where the position of the origin and the orientation of the coordinate axes are fixed.

[0063] In this embodiment, when the joint axis is a rotational axis, the position of the joint axis is the angle of the rotational axis. Moving the position of the joint axis means rotating the rotational axis and changing its position. Conversely, when the joint axis is a linear axis, the position of the joint axis is the position in the direction of movement of the linear axis. Moving the position of the joint axis means moving the linear axis and changing its position. Furthermore, when described as the position of the origin of the joint axis, it refers to the position of the origin of the coordinate system set for each axis in a coordinate system set in space. The coordinate system set in space is used to represent at least one of the positions and postures of the front end (working tool) of the robot 1, the flange 16 on which the front end is mounted, or the coordinate system set for each axis, etc., on an orthogonal coordinate system fixed relative to space.

[0064] Furthermore, in order to represent at least one of the position and posture of robot 1 on the reference coordinate system 87 set for the robot in space, the coordinate system set for the robot device is designated as the tool coordinate system. The origin, the point of translational movement, and the center point of rotational movement of the tool coordinate system are designated as control points. In this embodiment, all six axes of robot 1 are rotational axes, but robot 1 may also include linear axes. In addition, robot 1 in this embodiment is a vertical multi-joint robot composed of six axes. However, any robot with other forms is possible as long as it is capable of controlling the position of each axis and controlling orthogonal positions.

[0065] Furthermore, when the robot is mounted on a travel axis or other drive device that changes the robot's position and posture, the robot's reference coordinate system can be set in space by determining the position and posture of the robot or parts of the robot that also take into account the drive of these drive devices.

[0066] Figure 3 A block diagram illustrating the robot system of this embodiment. (Refer to...) Figures 1 to 3The robot 1 includes a robot drive unit that changes the position and orientation of the robot 1. The robot drive unit includes a robot drive motor 19, which serves as an actuator, such as a drive arm. The robot drive unit drives actuators disposed on the joint axes to move the rotational position of each joint axis of the robot 1.

[0067] Furthermore, the robot drive unit can be constructed from any principle or power source, as long as it can drive the robot's joint axis to cause displacement. Additionally, when the joint axis is a linear axis, the robot drive unit drives the actuator disposed on the joint axis in a manner that moves the position on the linear axis. The robotic arm 2 includes a robotic arm drive unit for driving the robotic arm 2. The robotic arm drive unit includes a pressure pump and valves for driving the gripper of the robotic arm 2.

[0068] The control unit 4 includes an arithmetic processing unit (computer) with a CPU (Central Processing Unit) as a processor. The arithmetic processing unit has RAM (Random Access Memory) and ROM (Read Only Memory) connected to the CPU via a bus. The motion program 41 contains instruction statements to drive the robot 1 and the manipulator 2. The robot system 6 is driven by the motion program 41 to transport workpieces.

[0069] The control device 4 includes a storage unit 42 for storing information. The storage unit 42 stores information related to the control of the robot 1 and the manipulator 2. The action program 41 is stored in the storage unit 42. The storage unit 42 can be constructed from a non-transitory storage medium. For example, the storage unit 42 can be constructed from a storage medium capable of storing information, such as volatile memory, non-volatile memory, magnetic storage medium, or optical storage medium.

[0070] The control device 4 includes a motion command unit 43 that sends motion commands. The motion command unit 43 is equivalent to a processor that drives the robot according to the motion program 41. The processor reads the motion program 41 and implements the control determined by the motion program 41, thereby functioning as the motion command unit 43. Based on the motion commands of the robot 1 output from the motion command unit 43, the robot drive unit is driven to change the rotational position of the robot drive motor 19 on the joint axis of the robot 1.

[0071] The motion command unit 43 sends motion commands to the robot drive circuit 45 for driving the robot 1. The robot drive circuit 45 includes circuitry for driving the robot drive motor 19. The robot drive circuit 45 supplies power to the robot drive motor 19 based on the motion commands. Additionally, the motion command unit 43 sends motion commands for driving the robot arm 2 to the robot arm drive circuit 44. The robot arm drive circuit 44 includes circuitry for driving the robot arm drive unit. The robot arm drive circuit 44 supplies power to the robot arm drive unit based on the motion commands. In this embodiment, the control device 4 is separately provided from the robot 1, but it is not limited to this method. The control device may also be configured inside the robot.

[0072] The robot 1 includes a rotational position detector 18 that outputs rotational position data of the robot 1 on the joint axes, i.e., axis position data. The rotational position detector 18 is, for example, an encoder. The position and posture of the robot 1 can be detected by the output of the rotational position detector 18. In this embodiment, the rotational position detector 18 is mounted on a robot drive motor 19 arranged corresponding to each joint axis.

[0073] The robot system 6 of this embodiment includes a display device for displaying arbitrary information. The display device includes at least one display section 60 disposed on the main body of the robot 1. Figure 1 In this embodiment, as an example of a display unit 60, a display unit 61a is mounted at the joint between the rotating base 13 and the lower arm 12. The display device includes a processing unit 21 that generates images displayed on the display unit 60 or processes commands from the display unit 60. In this embodiment, the processing unit 21 is configured as a control device 4.

[0074] The processing unit 21 includes a reference direction setting unit 22, which sets a reference direction, i.e., a reference direction, when determining the orientation of the image displayed on the display unit 60. The processing unit 21 also includes a coordinate system setting unit 23, which sets a coordinate system, i.e., a display coordinate system, for the display area of ​​the display unit 60.

[0075] The processing unit 21 includes a coordinate system calculation unit 24 that calculates the position and orientation of the display coordinate system based on the axis positions of each joint axis of the robot 1. The processing unit 21 includes a display phase calculation unit 25 that calculates the display phase direction as a reference for the display orientation on the display area of ​​the display unit 60. The processing unit 21 includes an offset angle setting unit 26 that sets the offset angle of the image displayed on the display unit. The processing unit 21 includes an instruction processing unit 29 that processes instructions input through the operation of the display unit 60. The processing unit 21 includes a display control unit 27 that controls the image displayed in the display area of ​​the display unit 60.

[0076] The processing unit 21 is equivalent to a processor driven by the action program 41. The processor reads the action program 41 and implements the control determined by the action program 41, thereby functioning as the processing unit 21. Furthermore, each of the units included in the processing unit 21—the reference direction setting unit 22, the coordinate system setting unit 23, the coordinate system calculation unit 24, the display phase calculation unit 25, the offset angle setting unit 26, the display control unit 27, and the instruction processing unit 29—is equivalent to a processor driven by the action program 41. The processor functions as each unit by implementing the control determined in the action program 41.

[0077] (Display section)

[0078] The robot system 6 of this embodiment has one or more display units 60 in the main body of the robot, such as the links, joint axes, and arm tip. The display units 60 are mounted on components of the robot whose position and posture are changed by a drive unit. The display units 60 are driven by the robot's joint axes to change position and posture. Alternatively, the robot may be equipped with a drive device for changing position and posture. This invention can also be applied when the position and posture of the display units 60 are changed by the drive device. The display units 60 of the display device can be any display panel capable of displaying characters or images. For example, a liquid crystal display panel or an organic EL (Electroluminescence) display panel can be used as the display unit 60.

[0079] The display section 60 is configured to be appropriately mounted on the part to be installed, and is preferably thin. The display section 60 preferably has a shape that does not protrude significantly from the main body of the robot. In addition, when the display section 60 is disposed at the end of the robot, it preferably has a shape that is integral with the main body of the robot and has a smooth outer surface.

[0080] Figure 4 This is a schematic diagram of a robot illustrating examples of various display units in this embodiment. Display units 61a, 61b, 61c, 62, 63a, 63b, 64, and 65 are formed in a plate shape, each having a planar display area. Display units 61a and 61b are fixed to the joint with their display areas perpendicular to axes J2 52 and J3 53. Display unit 61c is fixed to the joint with its display area perpendicular to axis J5 55. Display units 61a, 61b, and 61c move together with the joint.

[0081] Display unit 62 is fixed to the rotating base 13 with its display area parallel to the J1 axis. Display unit 62 rotates together with the rotating base 13. Display units 63a and 63b are fixed to the upper arm 11 with their display areas parallel to the J4 axis 54. Display units 63a and 63b move together with the upper arm 11. Display unit 64 is arranged with its display area parallel to the J5 axis 55. Display unit 65 is fixed to the wrist 15 with its display area parallel to the J6 axis 56. Display units 64 and 65 move together with the wrist 15. Thus, displays 61a, 61b, 61c, 62, 63a, 63b, 64, and 65 are fixed to the components of robot 1 and move together with the components of robot 1. In addition, each display unit rotates about the joint axis of robot 1.

[0082] Figure 5 This is a schematic diagram of another robot illustrating various display units of this embodiment. Display units 66 and 67 are constructed from flexible display panels. Display units 66 and 67 are formed such that the display area extends circumferentially along the surface of the component of robot 1. Display units 66 and 67 can be constructed from thin and flexible display panels, such as organic EL display panels. Display unit 66 is fixed by wrapping around the lower arm 12. Display unit 66 is formed such that the display area is parallel to the extending direction of the lower arm 12. Display unit 67 is fixed by wrapping around the upper arm 11. Display unit 67 is formed such that the display area is parallel to the extending direction of the upper arm 11. Display units 66 and 67 are fixed to the component of robot 1 and move together with the component of robot 1.

[0083] The display units 66 and 67 in this embodiment are composed of a single display panel, but are not limited to this method. Multiple display panels may also be arranged around the constituent parts of the robot 1 to form a display unit that surrounds the constituent parts. An image may be displayed on multiple display panels.

[0084] Figure 6 This refers to the first image displayed on the display screen. Figure 7This represents the second image displayed on the display unit. In the following embodiment, the display unit 61a, which is disposed between the rotating base 13 and the lower arm 12, will be described as an example. The display control unit 27 of the display device can display an image on the display area of ​​the display unit 61a based on data sent from other devices. The information displayed on the display unit includes, for example, the robot's motion status, the name of the currently executed motion program, the execution status of the motion program, the alarm status, the robot's drive speed, the number of times the job has been executed, and the number of times the retrieval failed. Here, the robot's drive speed or motion speed may also be the translational speed or rotational speed of the robot's front end or tool coordinate system, etc. Furthermore, the robot's drive speed or motion speed may also be the rotational speed of each joint axis, etc. In the following description, this will continue unless otherwise specified.

[0085] In addition, the information displayed on the display unit includes the cause of the problem and the method of countermeasures, the content of the next task to be performed, the connection status of the network to which the robot is connected, the load status of the actuators included in the robot drive unit that drives the robot, the current time, date and time, and the elapsed time of the task. The information displayed on the display unit is preferably information related to the robot's operation or settings, or any information sent to the robot from external devices, etc., that is useful to the operator when displayed on the main body of the robot.

[0086] In the first image 71a, information is displayed regarding whether the robot's drive motor is operating (State), the name of the currently executing motion program (Prog.), the drive speed around the joint axis (Speed), the number of times the program has been executed (Count), the time, and the date. The second image 71b also displays information about driving the robot.

[0087] Figure 8 This refers to the third image displayed on the display unit. The display unit not only has a display function for showing information but also an input function for inputting predetermined information through operator operation. The display device can display an input image for inputting predetermined information on the display unit. The third image 72a is an input image used to set the drive speed around the joint axis where the display unit is located or a desired joint axis. Additionally, sometimes other joint axes besides one joint axis are driven together with one joint axis. In this case, when changing the drive speed around one joint axis, the drive speeds of all joint axes can be adjusted to keep the position and posture of the robot's front end constant. Furthermore, the movement speed of the robot's front end in the currently executing motion program can be changed in the input image displayed on the display unit.

[0088] The display device can also function as an input device for inputting information through operator actions corresponding to an input image. Furthermore, the display device can function as a teaching pendant for manually driving a robot or generating motion programs. For example, the display device can have a touch panel for input operations as its display section. Alternatively, the display device can also have buttons for implementing emergency stop or other desired functions.

[0089] Thus, the display device in this embodiment includes both a device with only display function and a device with both display and input functions. (Refer to...) Figure 3 The commands input to the display unit 60 of the display device are sent to the command processing unit 29. The command processing unit 29 processes the input commands. For example, the command processing unit 29 sends motion commands for the robot to the motion command unit 43, or changes the settings stored in the storage unit 42 for driving the robot.

[0090] The display control unit 27 can display any input image on the display area. For example, it can display an image of a software keyboard, an image of numbers only, or an image of a bar-shaped image such as a slider to change the value to be input. In addition, the display control unit 27 can display an image of continuous input operation by rotating a predetermined image, an image of rotating like a dial on an analog telephone, or buttons related to changing the screen.

[0091] As a method of information input, any input method can be used, such as input through contact with the display area like a touch panel, or non-contact input using a capacitive sensor. In the case of non-contact input, information can be input through operations in the space above the display area. As a device for detecting input operations, any sensing method can be used, such as a capacitive sensor, an optical sensor, an optical camera, an infrared camera, or an ultrasonic sensor.

[0092] Furthermore, in addition to input devices for inputting information, contact or non-contact sensors can also be used as contact sensors to detect contact with people or surrounding objects. That is, by switching operating modes, sensors that detect input operations can also be used as sensors to ensure safety.

[0093] Information input to the display device is transmitted to the robot's control device or other devices such as network-connected devices via wired or wireless communication. Alternatively, the input information can be saved and stored in a teaching pendant that includes a display function. The display unit of the display device can display information calculated or processed based on the input information.

[0094] The image used for operator input, i.e., the input image, is preferably configured to allow input of any information for setting or confirmation purposes. For example, the input image is preferably configured to allow input of changes to setpoints, changes to the robot's drive speed, instructions to put the robot into standby mode, changes to the robot's actions, or interruptions to the robot's drive or actions. Furthermore, the input image can also be any image used for inputting information about tools or workpieces used for changing tools mounted on the robot's front end, setting coordinate systems, position teaching, changing to direct teaching mode, or changing to automatic operation mode. The function capable of operating setpoints is preferably limited to frequently used or simple functions. Alternatively, the input image can also be configured to allow for changes in the type of input function.

[0095] Furthermore, when information is input into the display device, the display unit can also display an image that accepts specific information for the operator to input. The display unit can also switch to an image that allows the operator to input specific information. For example, the operator can press a button designated for input after a predetermined time, enter a password, or move a finger across a predetermined point on the display area. Thus, to prevent erroneous input operations, the display control unit can also display an image that prohibits input operations. Moreover, it can also display an image that allows input operations, such as when the operator enters a password.

[0096] When information is input into the display device, at least one of the permissions for performing input operations and the permissions for displaying information can be changed through predetermined input operations. For example, by performing predetermined input operations, it is possible to change the settable items, change the robot's actions, change the range of settable values, or change the displayed information. In this way, the display device can also change the input instructions or information, or change the data or information that can be displayed.

[0097] The display unit can also be configured to be detachable from the main body of the robot. In particular, the display device having the display unit and the display device constituting the teaching pendant can also be configured to be detachable from the main body of the robot. In this case, the display device may be equipped with a battery. Alternatively, it is preferable that the display device is powered when mounted on the main body of the robot. Furthermore, the device for transmitting information displayed on the display device, or the device for transmitting information input to the display device, can be configured to be connected via wired or wireless connection.

[0098] Furthermore, the display coordinate system set for the display unit can be configured such that a coordinate system pre-generated based on the predetermined position where the display unit is mounted on the main body of the robot is applied. Alternatively, the display coordinate system can be directly input after the display unit is mounted, or it can be set using existing methods.

[0099] When a display device that can be attached to and detached from the robot functions as a teaching pendant, it can be used only when a teaching pendant is required. Alternatively, when multiple displays are provided, the teaching pendant can be used only in necessary parts of the robot. Therefore, the operability of the robot can be improved, the cost of the robot system can be reduced, or the risk of the teaching pendant being damaged by collisions with surrounding devices during robot movement can be decreased.

[0100] (Basic controls for displaying images)

[0101] Figure 9 A flowchart illustrating the control of image display on the display unit in this embodiment. (Refer to...) Figure 3 Before the display control unit 27 displays the image, the coordinate system setting unit 23 sets the display coordinate system corresponding to the display unit, as well as the reference plane and reference point set in the display coordinate system. The coordinate system setting unit 23 sets the display coordinate system, reference plane, and reference point, for example, based on the operator's operation. Additionally, the reference direction setting unit 22 sets the direction that becomes the reference for display, i.e., the reference direction, through predetermined control. The storage unit 42 stores the display coordinate system, reference plane, reference point, and reference direction.

[0102] Reference Figure 3 as well as Figure 9 In step 111, the processing unit 21 obtains the display coordinate system, reference plane, and reference point from the storage unit 42. In step 112, the processing unit 21 obtains the reference direction from the storage unit 42.

[0103] In step 113, the coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system based on the position and orientation of the robot 1. Specifically, the coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system based on the output of the rotation position detector 18. It calculates the position and orientation of the reference plane represented by the display coordinate system, as well as the position of the reference point.

[0104] In step 114, the display phase calculation unit 25 calculates the display phase direction in the reference plane based on the reference direction and the reference point. That is, the display phase calculation unit 25 sets the orientation of the image in the display unit. In step 115, the display control unit 27 generates an image in the reference plane based on the display phase direction. Then, in step 116, the display control unit 27 displays the image on the display unit 60 based on the image in the reference plane. Next, these controls will be described in detail.

[0105] (Coordinate System Setting Department) Figure 10 This indicates and explains the coordinate system, reference plane, reference point, and a rough plan view showing the phase direction. Figure 10 This is a plan view corresponding to the display area of ​​the display unit 61a. The coordinate system setting unit 23 of the processing unit 21 sets the coordinate system, namely the display coordinate system 88, for the display area of ​​the display unit. The coordinate system setting unit 23 sets the following for the display area of ​​the display unit: a reference plane 82, which is a reference plane for the display area of ​​the display unit, and is set on the display coordinate system; and a reference point 83, which is a reference point for display on the display area of ​​the display unit, and is set on the display coordinate system.

[0106] The display coordinate system can be set at any position and orientation relative to the display area of ​​the display unit. Using the display coordinate system, the position and orientation of the robot, such as its reference coordinate system, are displayed on the display unit or display area located on the main body of the robot. In the display unit 61a, the display coordinate system is set such that the plane formed by the two coordinate axes is coplanar with the display area.

[0107] A reference plane is a plane used as a reference when representing an image output to the display area of ​​the display unit. Furthermore, the reference plane is a plane set in the display coordinate system. The position and orientation of the reference plane in the display coordinate system can be predetermined. In the display unit 61a, the reference plane is set as a plane encompassing the two coordinate axes of the display coordinate system. A reference point is a point that determines the center position of the image output to the display area of ​​the display unit. The orientation of the image is determined by rotating the image around the reference point in the reference plane.

[0108] The reference direction can be set independently of the display coordinate system to any direction. The reference point is set on the reference plane. The reference plane can be set independently of the direction of the coordinate axes of the display coordinate system. For example, the reference plane can also be tilted relative to the XY plane of the display coordinate system. However, when the display area of ​​the display unit is planar, the display area, the reference plane, and the plane formed by the two coordinate axes of the display coordinate system are preferably the same plane. By setting it in this way, the amount of calculation can be reduced.

[0109] When the display area of ​​the display unit is curved rather than planar, a reference point can be set on the display area, and the tangent plane at the reference point can be used as the reference plane. In this case, even if the XY plane in the display coordinate system and the reference plane are the same plane, the reference plane does not include the display area.

[0110] By determining the display phase direction on the reference plane using the display phase calculation unit 25, the display control unit 27 can generate an image to be displayed on the reference plane. Next, the display control unit 27 can generate an image to be displayed on the display unit by projecting the image on the reference plane onto the display area of ​​the display unit.

[0111] The coordinate system setting unit 23 can arbitrarily set the display coordinate system 88, the reference plane 82, and the reference point 83. For example, the operator can pre-determine the display coordinate system 88, the reference plane 82, and the reference point 83 based on the shape of the display area of ​​the display unit and input them into the processing unit 21. Basically, these setting values ​​are determined for the display unit and can be predetermined as standard settings before the display unit is installed on the robot. The operator can change the setting values ​​as needed. For example, after setting standard values ​​for the display unit, the operator can also change the standard display values ​​according to the state of the robot displayed on the display unit to facilitate operation.

[0112] Furthermore, the coordinate system setting unit 23 can automatically set the display coordinate system 88, the reference plane 82, and the reference point 83 for the display unit. For example, sometimes the display unit has a large display area. Figure 5 As shown in display units 66 and 67, there are cases where the flexible display unit is wound around the robot's constituent parts. In this case, the coordinate system setting unit 23 can also automatically set the display coordinate system, reference plane, and reference point so that the image can be displayed in the same direction regardless of the rotational position of the constituent parts. For example, the coordinate system setting unit 23 can also automatically set the displayed setting value by moving it around the joint axis.

[0113] Furthermore, when display units 66 and 67 are formed, multiple display units are sometimes combined and arranged to surround the component parts of the robot. In this case, a display coordinate system, a reference plane, and a reference point can be set for each display area of ​​a single display unit. Alternatively, multiple display units can be combined to form a single display unit in a predetermined part of the robot, and a display coordinate system, a reference plane, and a reference point can be set. In this case, it is also possible to control the display of images on each display unit.

[0114] The display control unit 27 controls the position and orientation (phase) of the display in the display area of ​​the display unit to perform the display based on the reference point 83 on the display area of ​​the display unit and the display phase direction (direction shown by arrow 84), which is the reference direction for the orientation of the display on the display area of ​​the display unit calculated by the display phase calculation unit 25. For example, the display control unit 27 can display an image with the direction shown by arrow 84 above the image.

[0115] In this embodiment, the display control unit 27 determines the center position of the display on the display area of ​​the display unit using the reference point 83. The display control unit 27 determines the orientation of the display by rotating the image in the reference plane 82 with the reference point 83 as the center of rotation. In this embodiment, the origin and the reference point of the display coordinate system 88 are arranged in the same position, but they can also be arranged in different positions.

[0116] The display phase direction indicated by arrow 84 exists on the reference plane 82 and is set to pass through the reference point 83. When the display area of ​​the display unit is planar, the reference plane 82 is preferably parallel to the display area or exists on the same plane as the display area. In this embodiment, the display area of ​​the display unit is planar, but it may also be curved or other shapes.

[0117] When the display area of ​​the display unit is curved, a tangent plane at a predetermined point on the display area can be used as a reference plane. By using the predetermined point on the display area as the origin of the display coordinate system, the reference plane can be made into a plane formed by the two coordinate axes of the display coordinate system. The image displayed in the display area of ​​the display unit can be displayed based on the reference point and the display phase direction, or by projecting an image from the reference plane onto the display area. The image displayed in the display area can be displayed using any method, such as displaying based on calculated information.

[0118] Furthermore, the display area (the portion of the display panel that displays the image) of the display unit 61a is circular, but it can also be any shape such as a regular polygon. In this case, the reference point can be positioned at the center of the display area. For example, the reference point can be set at the centroid of the shape of the display area.

[0119] (Coordinate System Calculation Department)

[0120] The coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system 88 based on the axis positions of each joint axis of the robot 1. The coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system 88 on a reference coordinate system, such as the reference coordinate system 87, based on the axis positions of each joint axis of the robot 1 (angles in the case of rotation axes), the relationship between the positions and orientations of the joint axes as expressed by DH (Denavit Hartenberg) parameters, and the position and orientation of the joint coordinate system (joint coordinate system).

[0121] Here, the relationship between the joint coordinate system and the position and orientation of the display unit (the position and orientation of the display coordinate system 88) is determined. It is particularly preferable to select a joint coordinate system whose relationship with the position and orientation of the display unit is fixed, and the relationship between the position and orientation of the display coordinate system and the joint coordinate system is calculated in advance. In this embodiment, for joint axes connected sequentially via links, it is preferable to pre-set the relationship between the position and orientation of the joint coordinate system set from the display unit toward the robot's base 59 side and the nearest joint axis.

[0122] Furthermore, regardless of the type of robot, the position and orientation of the display coordinate system on a reference coordinate system can be calculated using methods widely recognized by those skilled in the art. For example, a homogeneous transformation matrix (relationship between position and orientation) in a reference coordinate system is determined up to a joint axis whose relationship with the position and orientation of the display unit is fixed. Based on this homogeneous transformation matrix between the joint axis and the display unit, the position and orientation of the display coordinate system on the reference coordinate system can be determined.

[0123] When a robot is mounted on a travel axis or other drive mechanism that alters its position and orientation, the position and orientation of the display coordinate system can be calculated by considering the movement of these drive components. Additionally, sometimes the position and orientation of the display coordinate system are defined as the relationship between the relative position and orientation of peripheral devices driven separately from the robot. In this case, drive mechanisms separately from the robot can also be included, and the relative position and orientation of the display coordinate system can be calculated by considering the changes in position and orientation of all drive components up to the display coordinate system.

[0124] (Reference Direction Setting Unit)

[0125] The reference direction setting unit 22 sets the direction, i.e., the reference direction, as a reference when determining the orientation of the image displayed on the display unit. The reference direction setting unit 22 sets the direction on the reference coordinate system 87 of the robot 1 as the reference direction. Alternatively, the reference direction can also be a direction on a coordinate system that is convenient for setting the reference direction. In addition, the reference direction setting unit 22 can also use a coordinate system that is set to be fixed relative to the space where the robot system is located to set the reference direction.

[0126] The reference orientation setting unit 22 can also use a reference coordinate system 87 set for the robot 1 to represent the position and posture of the robot 1's joints or arm tips, and set the reference orientation based on the robot's orientation in the reference coordinate system 87. Alternatively, the reference orientation setting unit 22 can use a coordinate system set for a predetermined part of the robot 1 to set the reference orientation. In this case, the reference orientation changes along with the driving or movement of the predetermined part of the robot.

[0127] Alternatively, the reference orientation setting unit 22 can also use the reference coordinate system of devices surrounding the robot, such as apparatuses, grippers, other robots, or other drive devices located near the robot 1, to set the reference orientation. In this case, the reference orientation setting unit 22 can also set the reference orientation in the portion where the position and posture change by driving or moving the surrounding devices. Furthermore, in this case, the reference orientation changes with the driving or moving of the surrounding devices.

[0128] The reference direction setting unit sets a reference direction in the desired coordinate system, thereby enabling the operator to easily observe the image displayed on the display unit. Furthermore, in the input image during input operations, an easy-to-input direction can be set. Moreover, various implementation methods for setting the reference direction are also possible; embodiments are shown below.

[0129] Figure 11 This is a perspective view illustrating the reference plane for the first control of the reference direction setting unit in setting the reference direction. The reference direction setting unit 22 can also set the reference direction in the robot's reference coordinate system 87 in a manner that forms a predetermined posture relative to the normal direction at a reference point 83 of the reference plane 82 set for the display area. For example, the reference direction setting unit 22 can also use the direction obtained relative to the normal direction through a predetermined homogeneous transformation matrix as the reference direction. In this example, the reference plane 82 and the display area of ​​the display unit are on the same plane. Therefore, the reference direction setting unit 22 can also set the reference direction in the robot's reference coordinate system 87 in a manner that forms a predetermined posture relative to the normal direction of the display area.

[0130] That is, as shown by arrow 82a, the reference direction setting unit 22 calculates the normal direction of the reference plane 82 by passing through the reference point 83. The reference direction setting unit 22 tilts the normal direction by the orientation and angle of arrow 91, which is represented by the reference coordinate system 87. The direction shown by arrow 81 becomes a direction parallel to the reference direction. Then, the reference direction setting unit 22 sets the direction parallel to the direction shown by arrow 81 as the reference direction shown by arrow 80.

[0131] By determining the reference direction in this way, the reference direction can be stably set even when the positions of each axis of the joint axis of robot 1 change due to the driving of robot 1. For example, as described later, a predetermined direction in the robot's reference coordinate system, such as the direction of gravity, can be set as the reference direction. In this case, if the normal direction of the display area of ​​the display unit is parallel to the reference direction, there is a problem that the display phase direction cannot be determined. However, in the control that sets the reference direction based on the normal direction of the reference plane, this problem does not exist, and the reference direction can be stably set.

[0132] Alternatively, during the control process of setting a predetermined direction in the robot's reference coordinate system as the reference direction, there may be changes in the posture relationship between the display unit and the reference direction. In this case, the reference plane on the display area of ​​the display unit may be orthogonal to or nearly orthogonal to the reference direction. Depending on this posture relationship, sometimes the change in the display phase direction becomes larger, or large changes in the display phase direction occur frequently. As a result, the calculation of the display phase direction may sometimes become unstable. In this case, it is also possible to switch to control that sets the reference direction from the normal direction of the reference plane.

[0133] When the robot is driven, the image is oriented to maintain a predetermined posture in the robot's reference coordinate system relative to the display area of ​​the display unit, thus making the image displayed on the display unit easily visible to the operator. Alternatively, an input image can be displayed to facilitate the operator's input operations.

[0134] Figure 12 This is a schematic plan view illustrating the second control for setting the reference direction by the reference direction setting unit. The reference direction setting unit 22 can also set the reference direction based on the range of the display unit's position and the range of its posture, within a coordinate system such as the robot's reference coordinate system 87. The position of the display unit can be the location of the origin or reference point of the display coordinate system. The posture of the display unit can be the posture of the display coordinate system. For example, when the position of the display unit changes with the robot's movement, the reference direction can be changed based on the position of the display unit in the reference coordinate system.

[0135] The range for setting the reference direction based on the location of the display unit can also be determined by the two coordinate axes of the robot's reference coordinate system. Alternatively, the range for determining the location of the display unit can also be defined by one or three coordinate axes of the robot's reference coordinate system.

[0136] exist Figure 12 In the example shown, four ranges 101a, 101b, 101c, and 101d are defined based on the two coordinate axes (X-axis and Y-axis) of the robot's reference coordinate system 87. When the display unit is positioned within range 101a, the reference direction setting unit 22 can set the reference direction indicated by arrow 80a. When the display unit is positioned within range 101b, the reference direction setting unit 22 can set the reference direction indicated by arrow 80b. Similarly, when the display unit is positioned within ranges 101c and 101d, the reference direction setting unit 22 can set the reference directions indicated by arrows 80c and 80d. The reference directions in each range 101a, 101b, 101c, and 101d can be set in the reference coordinate system 87.

[0137] By setting the reference direction in this way, the operator can easily observe the image displayed on the screen based on the range of the display unit's location or posture during robot operation. Furthermore, it facilitates input for the operator when entering images. For example, when multiple operators are working around the robot, the reference direction can be changed according to the operator's work area, making it easy for each operator to see the display.

[0138] Figure 13 This is a first schematic diagram of a robot controlled by a third controller that sets the reference direction using a reference direction setting unit. Figure 14 This is a second schematic diagram illustrating the robot controlled by the third control unit, which sets the reference direction. Figure 15 This is a third schematic diagram illustrating the third control robot that sets the reference direction using the reference direction setting unit. Figures 13 to 15 In addition to a schematic diagram of the robot, an enlarged view of image 71a displayed on display unit 61a is also shown.

[0139] exist Figure 13 In this configuration, robot 1 is positioned on the upper side of the mounting surface 105. Figure 14 In this configuration, robot 1 is positioned below the mounting surface 105. Figure 14 In the middle, Robot 1 is suspended. Figure 15 In the middle, the mounting surface 105 is inclined. In any robot 1, the base part 14 is fixed to the mounting surface 105. Arrow 90 indicates the direction of gravity (vertically downward).

[0140] The reference direction setting unit 22 can also set the reference direction to the direction of gravity, the opposite direction of gravity, or the direction in which gravity rotates at a predetermined posture. That is, the reference direction setting unit 22 can also set the direction determined by the reference coordinate system 87 as the reference direction. Figures 13 to 15 In the example shown, the reference direction is set to vertically upward (the direction opposite to that indicated by arrow 90). Each display unit 61a is configured in various orientations. However, in any orientation of the display unit 61a, the display control unit 27 displays the image 71a with its vertically upward orientation positioned above the image.

[0141] The third control is effective when the normal direction of the display area (reference plane) of the display unit forms an angle of a certain degree or greater with the direction of gravity. The reference direction determined in this control is based on the direction of gravity, and therefore is a direction that is easily and intuitively understood by the operator. For example, the display device can also be controlled to display images at a predetermined angle of 0 degrees or more relative to the horizontal direction, regardless of the angle of the joint axis of the robot on which the display unit is provided. In addition, when the display unit is composed of a touch panel for teaching operation, it can also be controlled to display input images such as buttons at a predetermined angle relative to the horizontal direction.

[0142] By setting the reference orientation based on the direction of gravity, the image displayed on the screen becomes easier for the operator to observe, regardless of the robot's settings or posture. Furthermore, it facilitates input for the operator within the input image.

[0143] Figure 16 This is a perspective view illustrating the reference plane for the fourth control unit that sets the reference direction. The angle between the reference direction and the normal direction of the reference plane varies depending on the robot's actuation, and the angle between the reference direction and the normal direction of the reference plane is sometimes less than a predetermined threshold. For example, due to changes in the position and posture of robot 1, the angle θ1 between the reference direction indicated by arrow 80 and the normal direction of the reference plane 82 indicated by arrow 82a is sometimes less than the predetermined threshold.

[0144] In this case, the reference orientation setting unit 22 may also maintain the reference orientation before the angle θ1 is less than a predetermined threshold. That is, the reference orientation setting unit 22 may maintain the previous reference orientation even when the robot's position and posture change and the angle θ1 is less than the predetermined threshold.

[0145] When the relationship between the display area of ​​the display unit and the orientation of the reference direction changes due to the robot's drive, and the reference direction is close to the normal direction of the reference plane 82, it is sometimes impossible to stably calculate the orientation (display phase direction) of the image in the display unit. In such cases, by continuing without changing the reference direction, the orientation of the image can be determined and the image can be displayed.

[0146] Figure 17 This is a perspective view illustrating the reference plane of the fifth control unit for setting the reference direction using the reference direction setting unit. The reference direction setting unit 22 can also set the reference direction based on a position specified by the operator on the display area of ​​the display unit. In this example, the reference plane 82 and the display area of ​​the display unit are set to be the same plane.

[0147] The operator changes the position and posture of robot 1 to bring reference plane 82 into the desired posture. Display control unit 27 displays reference point 83 on the display area of ​​the display unit. If the display unit is a touch panel, the operator designates a specified point 102 by pressing the desired point on the display area. Reference direction setting unit 22 calculates the direction from reference point 83 (indicated by arrow 81) toward specified point 102. Reference direction setting unit 22 sets the direction parallel to arrow 81 as the reference direction indicated by arrow 80. The reference direction can be set in the robot's reference coordinate system.

[0148] In the fifth control, the reference direction is set as the direction within the plane of the display area of ​​the display unit. Here, when the desired reference direction is the direction intersecting the plane of the display area of ​​the display unit, the operator can change the position and posture of the robot to change the orientation of the display area of ​​the display unit.

[0149] In the fifth control, a reference direction can be easily set according to the robot's state to ensure that the image orientation aligns with the operator's preferred orientation. This control allows for easy setting of a reference direction that makes it easy for the operator to see the display. Furthermore, it facilitates easy input for the operator into the input image.

[0150] Furthermore, depending on factors such as the shape of the display area, the display area and the reference plane are sometimes not on the same plane. For example, sometimes the display area of ​​the display unit is formed as a curved surface. In this case, the reference direction setting unit can project a specified point designated by the operator onto the reference plane in a predetermined direction. Moreover, the reference direction can also be set based on the point projected onto the reference plane.

[0151] In the above embodiments, the direction from the reference point toward the specified point is set as the reference direction, but this method is not limited to. Any direction based on the specified point specified by the operator in the display area can be set as the reference direction. For example, in the first control for setting the reference direction, when setting the posture of the reference direction relative to the normal direction of the reference plane, control can also be performed to specify the specified point on the display area.

[0152] (Displays phase calculation unit)

[0153] Figure 18 This is a perspective view illustrating the reference plane for calculating the first control of the display phase direction. The display phase calculation unit 25 calculates the direction that serves as the reference for the display direction on the display area of ​​the display unit, namely the display phase direction (the direction indicated by arrow 84), by projecting the reference direction indicated by arrow 80 onto the reference plane 82, which is the reference plane of the display area of ​​the display unit. Figure 18This indicates the case where the reference direction indicated by arrow 80 is orthogonal to the normal direction of the reference plane 82 indicated by arrow 82a. In this case, the display phase calculation unit 25 can project the reference direction to any direction as shown by arrow 92. The display phase calculation unit 25 projects the reference direction toward the reference plane 82. Furthermore, the display phase direction indicated by arrow 84 can be determined by means of the reference point 83.

[0154] Figure 19 This is a three-dimensional view of the reference plane for calculating and displaying the phase direction of the second control. Figure 19 This indicates a case where the reference direction indicated by arrow 80 is not perpendicular to the normal direction of the reference plane 82 indicated by arrow 82a, but is tilted. The phase calculation unit 25 calculates the direction parallel to the reference direction passing through the reference point 83, as shown by arrow 81, based on the reference direction indicated by arrow 80.

[0155] The display phase calculation unit 25 projects a direction parallel to the reference direction through the reference point 83 onto the normal direction of the display area, as shown by arrow 92. The display phase calculation unit 25 projects a direction parallel to the reference direction through the reference point 83 toward the reference plane 82. Through this control, as shown by arrow 84, the display phase direction, which serves as the reference for the phase of the display in the display coordinate system, can be determined.

[0156] Alternatively, the display phase calculation unit 25 calculates the direction in which the reference direction indicated by arrow 80 is projected onto the reference plane 82 towards the normal direction of the display area. Then, the display phase calculation unit 25 calculates the direction passing through the reference point 83 in a manner parallel to the projection direction. The display phase calculation unit 25 can determine this direction as the display phase direction.

[0157] Figure 20 This is a schematic diagram illustrating the calculation and display of the phase direction of a third-controlled robot. Figure 21 This is a plan view illustrating the calculation and display of the third control reference plane for the phase direction. (Refer to...) Figure 3 , Figure 20 and Figure 21 The display phase calculation unit 25 projects the reference direction onto the reference plane 82 of the display area of ​​the display unit 61a. Furthermore, the display phase calculation unit 25 can also calculate the display phase direction by rotating around a predetermined angle, i.e., an offset angle, with the reference point 83, which serves as the reference for the display on the display area of ​​the display unit, as the center.

[0158] Here, the reference direction is determined to be vertically upward, as indicated by arrow 80. The display phase calculation unit 25 calculates the direction parallel to the reference direction in the reference plane 82 by projecting the reference direction onto the reference plane 82. As indicated by arrow 81, the display phase calculation unit 25 calculates the direction parallel to the reference direction passing through the reference point 83.

[0159] The processing unit 21 in this embodiment includes an offset angle setting unit 26 that sets a predetermined angle, or offset angle, centered on a reference point on the display area. The offset angle setting unit 26 sets the offset angle according to the operator's operation. For example, the operator inputs the offset angle θ2 using a teach pendant. Furthermore, the offset angle setting unit 26 can set the offset angle θ2 relative to the direction indicated by the arrow 81 that projects the reference direction onto the reference plane.

[0160] Next, the display phase calculation unit 25 rotates around the reference point 83 by a predetermined rotation angle, i.e., an offset angle θ2, in the direction indicated by arrow 81. The display phase calculation unit 25 rotates within the reference plane 82 in the direction of the projection reference direction. Then, the display phase calculation unit 25 calculates the display phase direction indicated by arrow 84. The display control unit 27 controls the orientation of the image on the display area of ​​the display unit 61a for display based on the display phase direction calculated by the display phase calculation unit 25 and the reference point 83.

[0161] like Figure 20 As shown, the image 71a displayed on the display unit 61a is tilted by an offset angle θ2 relative to the direction in which the reference direction is projected onto the reference plane. This control allows the orientation of the image displayed on the display area of ​​the display unit to be easily shifted relative to the operator. The image can be displayed in a manner that establishes a predetermined phase relationship with respect to a predetermined direction based on the reference direction. When the operator observes the display unit from a specific direction, information is easily seen by displaying information according to that specific direction. Furthermore, if the display device has an input function, the operator can easily perform input operations on the input image.

[0162] Figure 22 This is a plan view illustrating the reference plane for setting the fourth control of the display phase direction. In this example, the reference plane 82 and the display area of ​​the display unit are configured to be on the same plane. The offset angle setting unit 26 is configured to allow the operator to set the offset angle θ2 by operating the display area of ​​the display unit.

[0163] As indicated by arrow 81, the display control unit 27 displays the direction in which the current reference direction is projected onto the reference plane 82. The operator presses the display area of ​​the display unit to specify the designated point 103. The offset angle setting unit 26 calculates the offset angle θ2 for the direction indicated by arrow 81. The display phase calculation unit 25 sets the display phase direction based on the offset angle θ2. The display control unit 27 controls the orientation of the image on the display area of ​​the display unit 61a for display based on the display phase direction calculated by the display phase calculation unit 25 and the reference point 83.

[0164] Furthermore, when the display area of ​​the display unit is curved, the offset angle can be set based on the point on the reference plane where the point the operator touches in the display area is projected. For example, the offset angle can be calculated based on the point on the normal direction of the reference plane where a specified point is projected by the operator. Alternatively, control of image rotation based on such an offset angle may not be implemented.

[0165] (Display Control Department)

[0166] The display control unit 27 controls the image displayed on the display area of ​​the display unit. Additionally, when the display unit displays an input image, the display control unit 27 controls the orientation of the input image and displays it. Based on the axial positions of each axis of the robot's joints and the reference direction, the display control unit 27 controls the orientation of the image on the display area of ​​the display unit and displays it, so as to display an image that forms a predetermined posture relative to the reference direction.

[0167] In addition, the display control unit 27 controls the position and orientation of the image on the display area of ​​the display unit and displays it based on the axis position of each axis of the robot's joint axis, the reference direction, the reference point on the display coordinate system, and the display phase direction on the display coordinate system calculated by the display phase calculation unit 25.

[0168] Furthermore, sometimes the robot is mounted on a device with a travel axis or other drive mechanism that alters the robot's position and posture. In this case, the processing unit 21 preferably displays an image based on the axial position of each axis of the robot's joints and the amount of movement of these drive components.

[0169] The display control unit 27 controls the orientation of the image on the display area of ​​the display unit, and can display the image in the display area of ​​the display unit in a desired orientation when the posture of the display unit changes due to the drive of the robot. Alternatively, it can display an input image in a desired orientation to perform an input operation.

[0170] Next, a specific example of the image displayed by the display control unit 27 will be described. Figure 23This is a schematic side view of the robot in its first position and posture. Here, regarding the image of display unit 61a, the reference direction indicated by arrow 80 is shown as vertically upward. Such a reference direction can be set, for example, by the first or third control of setting the reference direction in this embodiment. In display unit 61a, the display phase direction indicated by arrow 84 faces the same direction as the reference direction indicated by arrow 80. That is, the display phase direction indicated by arrow 84 faces upward in the vertical direction.

[0171] Figure 24 A schematic diagram showing the robot from its initial position and when it is being posed. Figure 25 This is a schematic diagram of a robot representing a comparative example of a robot driven from a first position and in a pose. Here, the axial position of the J2 axis 52 between the rotating base 13 and the lower arm 12 changes. As indicated by arrow 93, the lower arm 12 rotates and its pose changes.

[0172] exist Figure 24 The image on display unit 61a is shown in the state when the display control implemented by the processing unit 21 of this embodiment is executed. The display control unit 27 controls the orientation of the image on the display area of ​​display unit 61a based on the axial position of each axis of the robot 1's joints and the reference direction, so as to display an image that forms a predetermined posture relative to the reference direction. The display phase direction indicated by arrow 84 remains parallel to the reference direction. That is, the display phase direction remains vertically upward.

[0173] exist Figure 25 In the comparative example shown, the display control of this embodiment was not implemented. In the display unit 61a, the display phase direction indicated by arrow 85 rotates together with the rotation of the lower arm 12. As a result, the image displayed on the display unit 61a is tilted. It is difficult for the operator to see the image displayed on the display unit 61a.

[0174] In contrast, refer to Figure 24 Because of the display control performed by the processing unit 21 in this embodiment, even if the axial position of the joint axis changes, the display phase direction indicated by arrow 84 is still upward in the vertical direction. Therefore, the image displayed on the display unit 61a remains in a state that is easy to observe.

[0175] Figure 26 A schematic side view of the robot in its second position and posture. Figure 27This is a schematic plan view of the robot in its second position and posture. In this example, the display area (reference plane) of the display unit 63b is parallel to the horizontal direction. Regarding the display unit 63b, the reference direction indicated by arrow 80 is set to the horizontal direction. This reference direction can be set by the first, second, or third control of setting the reference direction in this embodiment.

[0176] In display unit 63b, the display phase direction indicated by arrow 84 faces the horizontal direction. Furthermore, as indicated by arrow 94, the operator observes display unit 63b from above the robot 1. The display phase direction indicated by arrow 84 is the direction from which the operator can easily observe the image.

[0177] Figure 28 A schematic diagram showing the robot in its second position and when it is posed. Figure 29 A schematic diagram of a robot showing a comparative example of a robot driven from a second position and a pose. In this example, as indicated by arrow 95, the rotating base 13 rotates about axis J1 51.

[0178] exist Figure 28 In the example shown, display control is implemented by the processing unit 21. The display control unit 27 controls the orientation of the image on the display area of ​​the display unit and displays it in a manner that forms a predetermined posture relative to the reference direction, based on the axis positions of each axis of the robot's joints and the reference direction. The display phase direction indicated by arrow 84 is parallel to the reference direction indicated by arrow 80.

[0179] exist Figure 29 In the comparative example shown, the display control of this embodiment was not implemented. The orientation of the display phase direction indicated by arrow 85 in display unit 63b changes with the rotation of upper arm 11. As a result, when the operator observes display unit 63b, the image displayed on display unit 63b is tilted. It is difficult for the operator to observe the image displayed on display unit 63b.

[0180] Reference Figure 28 By implementing the display control through the processing unit 21 of this embodiment, even if the axis position of the joint axis changes, the display phase direction indicated by arrow 84 remains parallel to the reference direction. The image displayed on the display unit 63b is displayed in the same orientation even when the robot is driven. Therefore, it is possible to maintain a state in which the operator can easily observe the image displayed on the display unit 63b.

[0181] Figure 30This is a schematic side view of the robot in its third position and posture. The upper arm 11, to which the display unit 63b is fixed, is tilted relative to the horizontal direction. Regarding the display unit 63b, the reference direction indicated by arrow 80 is set to the horizontal direction. This reference direction can be set by the second or third control of setting the reference direction in this embodiment. In this example, the normal direction of the display area (reference plane) of the display unit 63b is not orthogonal to the reference direction. Furthermore, the operator observes the display unit 63b in the direction indicated by arrow 96.

[0182] Figure 31 This diagram illustrates the phase direction of the robot when it is in its third position and during pose-driven actions. (See reference...) Figure 30 and Figure 31 In this example, as indicated by arrow 95, the rotating base 13 rotates about axis J1 51. Figure 31 In the example shown, display control is implemented based on the processing unit 21. The display control unit 27 controls the orientation of the image on the display area of ​​the display unit and displays the image based on the axis position and reference direction of each axis of the robot's joints, so as to display the image in a predetermined posture relative to the reference direction.

[0183] Even if the upper arm 11 rotates as shown by arrows 95a and 95b, the display phase direction indicated by arrow 84 remains the direction that projects the reference direction onto the display area (reference plane) of the display unit 63b. The display phase direction indicated by arrow 84 becomes the direction in which the operator can easily observe the image. When the operator observes the display unit 63b in the direction indicated by arrow 96, the image is displayed in the same orientation.

[0184] Figure 32 This diagram illustrates the display phase direction of a comparative example of the robot when driven from the third position and in a posture. In the comparative example, the display control of this embodiment is not implemented. The display phase direction indicated by arrow 85 in display unit 63b changes direction with the rotation of the upper arm 11 indicated by arrows 95a and 95b. As a result, when the operator observes display unit 63b, the image displayed on display unit 63b appears tilted. It is difficult for the operator to see the image displayed on display unit 63b.

[0185] Reference Figure 31 By implementing the display control through the processing unit 21 of this embodiment, even if the axial position of the joint axis changes, the display phase direction remains in a direction that is easy for the operator to observe the image. Therefore, it is possible to maintain a state in which the operator can easily observe the image displayed on the display unit 63b.

[0186] In the above embodiments, a display coordinate system, a reference plane, and a reference point are set for the display area of ​​the display unit to calculate the display phase direction, but this method is not limited to this. The processing unit may also choose not to set a display coordinate system, a reference plane, and a reference point. The processing unit can calculate the display phase direction based on the reference direction through arbitrary control. For example, when the reference direction is parallel to the display area of ​​the display unit, the processing unit may also set the reference direction as the display phase direction.

[0187] Furthermore, the display control unit can also perform image processing on the image generated in the reference plane. For example, it can perform arbitrary image conversion operations, such as image magnification, image reduction, or predetermined image conversion. Moreover, the display control unit can display the converted image on the display unit. For example, as described above, when the display area of ​​the display unit is curved, an image displayed in the display area can be generated by projecting the image on the reference plane onto the display area of ​​the display unit.

[0188] (Example of an image displayed on the display screen)

[0189] Next, an example of an image displayed by the display control unit on the display area of ​​the display unit will be described. Here, an example of an image displayed on the display unit 61a will be described.

[0190] Figure 33 This indicates the fourth image displayed on the display unit. Figure 34 The fifth image displayed on the display unit. The fourth image 71c is an image showing the robot's driving state. The fifth image 72b is an input image showing the driving speed around the joint axis where the display unit is located or the desired joint axis. Additionally, sometimes other joint axes besides one joint axis are driven together with one joint axis. In this case, when changing the driving speed around one joint axis, the driving speeds of all joint axes can be adjusted to keep the position and posture of the robot's front end constant. Furthermore, the movement speed of the robot's front end in the currently executing motion program can be changed in the input image displayed on the display unit. The display control unit 27 can display information or input images in the display area of ​​the display unit using at least one of characters, graphics, symbols, colors, and patterns.

[0191] Furthermore, the display control unit 27 can also display the orientation of the image by using at least one of characters, graphics, symbols, colors, and patterns. In the fourth image 71c and the fifth image 72b, an image 73 is displayed that represents the upper side of the image.

[0192] In this way, the display control unit generates an image in the display area in a manner that allows the orientation of the image to be known, making it easier for the operator to recognize the orientation of the image in the display area of the display unit. Additionally, when the input image is displayed on the display unit, the operability of the operator can be improved.

[0193] Figure 35 Represents the sixth image displayed on the display unit. Figure 36 Represents the seventh image displayed on the display unit. In the sixth image 71d, the character "up" indicating the upper side of the image is displayed. In the seventh image 71e, a part of the frame surrounding the information indicating the driving state is recessed, thereby indicating the upper side of the image.

[0194] simulation Figure 37 Represents the eighth image displayed on the display unit. Figure 38 Represents the ninth image displayed on the display unit. In the eighth image 71f and the ninth image 71g, a pattern indicating the upper side is displayed at the upper part. And, in the ninth image 71g, the pattern indicating the upper side is displayed in a color different from the color of the background of the display area. In this way, the display control unit 27 can generate an image in a manner that represents the orientation of the image in various ways.

[0195] Regarding the above-described embodiment, the image indicating the upper side has been described, but it is not limited to this method. The display control unit only needs to display in a manner that allows the orientation of the image to be known. For example, the display control unit can also display characters indicating the lower side of the image, etc.

[0196] Figure 39 Represents the tenth image displayed on the display unit. Figure 40 Represents the eleventh image displayed on the display unit. The display control unit 27 can also perform control to change the size of a predetermined part of the image on the display area of the display unit 61a according to the moving speed of the display unit 61a of the display device accompanying the driving of the robot. For example, the display control unit 27 can display a predetermined image small when the moving speed of the display unit 61a is small, and display a predetermined image large when the moving speed of the display unit 61a is large.

[0197] In Figure 39 In the tenth images 71ha and 71hb, the number of times of operation execution is displayed on the display unit 61a. The display control unit 27 can change the size of the image of the number of times of execution as shown by the arrow 97 according to the moving speed of the display unit 61a. For example, the display control unit 27 can calculate the moving speed of the display coordinate system as the moving speed of the display unit 61a. The display control unit 27 acquires the position and time of the origin of the display coordinate system at every predetermined time interval. The display control unit 27 can calculate the moving speed of the origin of the display coordinate system based on the position and time of the origin of the display coordinate system.

[0198] When the moving speed of the display unit 61a is less than a predetermined threshold, the display control unit 27 can display the image of the number of executions at a smaller size, as shown in image 71ha. When the moving speed of the display unit 61a is greater than or equal to the predetermined threshold, the display control unit 27 can display the image of the number of executions at a larger size, as shown in image 71hb. By implementing this control, the desired portion of the image is displayed at a larger size when the robot's driving speed is high, making it easier for the operator to see the displayed image.

[0199] Figure 40 The eleventh images 72ca and 72cb are input images for the operator to perform input operations. In the eleventh images 72ca and 72cb, a predetermined setting value can be changed by pressing the "+" or "-" button. The display control unit 27 can change the size of the image representing the setting value according to the movement speed of the display unit 61a, as shown by arrow 98. Additionally, the display control unit 27 can change the size of the buttons.

[0200] When the moving speed of the display unit 61a is less than a predetermined threshold, as shown in image 72ca, the display control unit 27 can display a larger image of the set value and a smaller image of the button. When the moving speed of the display unit 61a is greater than or equal to the predetermined threshold, as shown in image 72ca, the display control unit 27 can display a smaller image of the set value and a larger image of the button. Through this control, even if the robot's driving speed increases, the operator can press the button stably because the button image also becomes larger.

[0201] In the robot system of this embodiment, when the display device has an input function, the operator can perform input operations during robot operation. By implementing control that changes the image size according to the movement speed of the display unit, the operator can easily see the image or easily perform input operations. For example, by displaying a larger button image when the movement speed of the display unit increases, the operator can easily perform input operations. Furthermore, during robot operation, when input operations from the display device are possible, for safety reasons, it is preferable to limit the robot's movement speed or the drive speed of the joint axes that allows input operations. For example, it is preferable that input operations can be performed when the robot's movement speed or the drive speed of the joint axes is below a predetermined speed. Alternatively, the setting value for enabling input operations can be changed according to the robot's movement speed or the drive speed of the joint axes.

[0202] In each of the above-described controls, the order of steps can be appropriately changed without altering the function or effect. The above-described embodiments can be appropriately combined. In the above figures, identical or equivalent parts are labeled with the same symbols. Furthermore, the above-described embodiments are illustrative and do not limit the invention. Additionally, the embodiments include modifications to the embodiments shown within the scope of the claimed patent protection.

[0203] Symbol Explanation

[0204] 1 robot;

[0205] 2 robotic arms;

[0206] 4. Control devices;

[0207] 6. Robotic system;

[0208] 11. Upper arm;

[0209] 12. Lower arm;

[0210] 13. Rotating base;

[0211] 14. Base section;

[0212] 15. Wrist;

[0213] 16 flanges;

[0214] 21 Processing Department;

[0215] 22. Reference direction setting unit;

[0216] 23. Coordinate System Setting Department;

[0217] 24-coordinate system calculation unit;

[0218] 25 Displays the phase calculation unit;

[0219] 26. Offset angle setting unit;

[0220] 27. Display Control Unit;

[0221] 42. Storage Unit;

[0222] 43. Action Command Section;

[0223] 51J1 axis;

[0224] 52J2 axis;

[0225] 53J3 axis;

[0226] 54J4 axis;

[0227] 55J5 axis;

[0228] 56J6 axis;

[0229] 59 pedestals;

[0230] R1 and R2 are rotating axes;

[0231] Display units 60, 61a, 61b, 61c, 62, 63a, 63b, 64, 65, 66, and 67;

[0232] Images 71a, 71b, 71c, 71d, 71e, 71f, 71g, 71ha, and 71hb;

[0233] Images 72a, 72b, 72ca, and 72cb;

[0234] 82 reference plane;

[0235] 83 benchmark points;

[0236] 87. Reference coordinate system;

[0237] 88 displays the coordinate system;

[0238] The range is 101a to 101d;

[0239] Points 102 and 103 are designated;

[0240] θ1 angle;

[0241] θ2 offset angle.

Claims

1. A robot system, characterized in that, have: A robot, which includes multiple joint axes; A display unit is located in the main body of the robot; A display control unit controls the image displayed in the display area of ​​the display unit; as well as The reference direction setting unit is set to a reference direction, i.e., a reference direction, when determining the orientation of the image displayed on the display unit. The display control unit controls the orientation of the image in the display area of ​​the display unit and displays it based on the axial position of each joint axis of the robot and the reference direction, in a manner that forms a predetermined posture relationship with respect to the reference direction. The robot system has the following features: The coordinate system setting unit sets a display coordinate system, a reference plane, and a reference point. The display coordinate system is a coordinate system set for the display area of ​​the display unit. The reference plane is a reference plane for the display area of ​​the display unit and is set on the display coordinate system. The reference point is a point that serves as the reference for display on the display area of ​​the display unit and is set on the display coordinate system. The coordinate system calculation unit calculates the position and posture of the display coordinate system based on the axis positions of each joint axis of the robot; as well as The display phase calculation unit calculates the display phase direction by projecting a reference direction onto a reference plane on the display coordinate system. This display phase direction serves as the reference direction for the orientation of the display on the display area of ​​the display unit. The display control unit controls the orientation of the image on the display area of ​​the display unit and displays it based on the display phase direction calculated by the display phase calculation unit and the reference point.

2. A robot system, characterized in that, have: A robot, which includes multiple joint axes; A display unit is located in the main body of the robot; A display control unit controls the image displayed in the display area of ​​the display unit; as well as The reference direction setting unit is set to a reference direction, i.e., a reference direction, when determining the orientation of the image displayed on the display unit. The display control unit controls the orientation of the image in the display area of ​​the display unit and displays it based on the axial position of each joint axis of the robot and the reference direction, in a manner that forms a predetermined posture relationship with respect to the reference direction. The robot system has the following features: The coordinate system setting unit sets a display coordinate system, a reference plane, and a reference point. The display coordinate system is a coordinate system set for the display area of ​​the display unit. The reference plane is a reference plane for the display area of ​​the display unit and is set on the display coordinate system. The reference point is a point that serves as the reference for display on the display area of ​​the display unit and is set on the display coordinate system. The coordinate system calculation unit calculates the position and posture of the display coordinate system based on the axis positions of each joint axis of the robot; as well as The display phase calculation unit projects a reference direction onto a reference plane in the display coordinate system, and rotates around a reference point by an offset angle of a predetermined angle. This calculates the reference direction for the orientation of the display on the display area of ​​the display unit, i.e., the display phase direction. The display control unit controls the orientation of the image on the display area of ​​the display unit and displays it based on the display phase direction calculated by the display phase calculation unit and the reference point.

3. The robot system according to claim 1 or 2, characterized in that, The display unit displays an input image for inputting information and has an input function that allows information to be input through operations corresponding to the input image. The display control unit controls the orientation of the input image to display the input image.

4. The robot system according to claim 1 or 2, characterized in that, The reference direction setting unit sets the reference direction on the robot's reference coordinate system in a manner that forms a predetermined posture relationship with the normal direction of the reference plane.

5. The robot system according to claim 1 or 2, characterized in that, The reference direction setting unit sets the reference direction in the robot's reference coordinate system according to the range where the display unit is located.

6. The robot system according to claim 1 or 2, characterized in that, The reference direction setting unit sets the gravity direction, the direction opposite to the gravity direction, or the direction after rotating the gravity direction in a predetermined posture as the reference direction.

7. The robot system according to claim 1 or 2, characterized in that, The reference direction setting unit sets the reference direction based on the position specified by the operator on the display area of ​​the display unit.

8. The robot system according to claim 1 or 2, characterized in that, When the angle between the reference direction and the normal direction of the reference plane changes due to the robot's drive, and the angle between the reference direction and the normal direction is less than a predetermined threshold, the reference direction setting unit maintains the reference direction without changing it until the angle between the reference direction and the normal direction is less than the predetermined threshold.

9. The robot system according to claim 2, characterized in that, The robot system includes an offset angle setting unit, which sets a predetermined angle, i.e., an offset angle, centered on a reference point based on the position specified by the operator on the display area of ​​the display unit.

10. The robot system according to claim 3, characterized in that, When information is input into the display unit, the display unit displays an image that accepts input of specific information for the operator to perform input operations. When the operator inputs specific information, the display unit switches to an image that the operator can perform input operations.

11. The robot system according to claim 3, characterized in that, The display unit is configured to allow the operator to change at least one of the permissions for performing input operations and the permissions for displaying information through predetermined input operations.

12. The robot system according to claim 1 or 2, characterized in that, The display control unit displays the image on the display area of ​​the display unit in a manner that indicates the orientation of the image by means of at least one of characters, graphics, symbols, colors, and patterns.

13. The robot system according to claim 1 or 2, characterized in that, The display control unit controls the size of the image on a predetermined portion of the display area of ​​the display unit to vary according to the moving speed of the display unit driven by the robot.

14. The robot system according to claim 1 or 2, characterized in that, The display unit can be attached to or detached from the main body of the robot.

Images