Visual recognition device, visual recognition method, and visual recognition program

Abstract

To provide a visual recognition device and so on by which an operator can visually recognize robot operation information regardless of relative position or posture with a robot.SOLUTION: A visual recognition device 5 of a robot arm 100 comprises: an operation information acquisition part 51 which acquires of operation information of a robot arm 100 comprising a cutting tool 20 for processing a work-piece W; and a display device 52 which is separate from the robot arm 100 and can be visually recognized by an operator for the robot arm 100 simultaneously with the robot arm 100, and which displays the operation information acquired by the operation information acquisition part 51. A display device 52A can be installed by the operator so as to be located on a visual line of the operator. The display device 52A is a transmission type display.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a robot vision device and the like. [Background technology]

[0002] Industrial robots are being introduced to perform various tasks in place of humans in industrial sites such as factories, logistics warehouses, construction sites, and hospitals. Conventional industrial robots are often large and powerful, and for safety reasons, they must be operated in isolated spaces that humans cannot enter. On the other hand, in recent years, collaborative robots that work alongside humans in the same space have also been increasingly introduced. Compared to conventional industrial robots, collaborative robots are small and can be installed in narrow spaces, and because they have low power output, they do not require large-scale systems to ensure safety.

[0003] Since humans can approach collaborative robots, they can directly view, for example, the end effector at the tip of the robot arm that performs various processes on the workpiece, the robot hand, and / or the workpiece, and can confirm that the intended processing is being performed correctly. In Patent Document 1, the robot arm is further provided with a display unit that displays information that humans cannot directly see, such as the force and torque applied to the end effector. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-30058 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, since the display unit is provided on the robot arm, there is a possibility that the human may not be able to see the display unit depending on the posture of the robot arm and the position and / or posture of the human relative to the robot arm.

[0006] The present invention has been made in consideration of these circumstances, and aims to provide a visualization device or the like that allows an operator to visually check the operation information of a robot regardless of the position or posture relative to the robot. Note that the present invention is not limited to collaborative robots, and can be applied to robot arms and the like as general industrial robots. [Means for solving the problem]

[0007] In order to solve the above problem, a visualizing device according to one embodiment of the present invention comprises an operation information acquisition unit that acquires operation information of a robot having a processing unit that processes a workpiece, and a display device that is separate from the robot and can be viewed by the robot operator simultaneously with the robot, and that displays the operation information acquired by the operation information acquisition unit.

[0008] In this aspect, the display device that displays the robot's operational information is separate from the robot, so the robot operator can visually recognize the operational information regardless of the relative position or posture of the robot.

[0009] Another aspect of the present invention is a visualization method, which includes: an operation information acquisition step of acquiring operation information of a robot having a processing section for processing a workpiece; and a display step of displaying the operation information acquired in the operation information acquisition step on a display device separate from the robot that can be viewed by an operator of the robot simultaneously with the robot.

[0010] Any combination of the above components and any conversion of these expressions into methods, devices, systems, recording media, computer programs, etc. are also encompassed by the present invention. [Effects of the Invention]

[0011] According to the present invention, an operator can visually recognize the operation information of the robot regardless of the position or posture relative to the robot. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 2 is a perspective view showing the appearance of the robot arm. [Figure 2] 1 is a schematic diagram showing the configuration of a connecting device that constitutes each joint of a robot arm. [Figure 3] 1 is a schematic diagram showing the configuration of a connecting device that constitutes each joint of a robot arm. [Figure 4] Schematic diagram showing the operation of pressing the cutting tool against the workpiece at the start of cutting. [Figure 5] FIG. 2 is a schematic enlarged view of a cutting tool pressed against a workpiece. [Figure 6] FIG. 2 is a functional block diagram of a visual recognition device for a robot arm. [Figure 7] 10 shows an example of a display on a display device. [Figure 8] 10 shows an example of a display on a display device. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, a mode for carrying out the present invention (hereinafter also referred to as an embodiment) will be described in detail with reference to the drawings. In the description and / or drawings, identical or equivalent components, members, processes, etc. will be assigned the same reference numerals, and redundant explanations will be omitted. The scale and shape of each part shown in the drawings are set for convenience to simplify the explanation, and should not be interpreted as limiting unless otherwise specified. The embodiment is an example and does not limit the scope of the present invention in any way. All features and combinations thereof described in the embodiment are not necessarily essential to the present invention.

[0014] FIG. 1 is a perspective view showing the appearance of a robot arm 100 as an example of an industrial robot or a collaborative robot. This robot arm 100 is a vertically articulated robot arm with a serial link mechanism. Robots to which the present invention can be applied are not limited to robot arms in the narrow sense, but may be robot arms in the broad sense having joints (corresponding to joints in the human body) that connect multiple links (corresponding to bones in the human body) so that they can move relative to one another, or any other robot. Furthermore, a parallel link mechanism may be used instead of the serial link mechanism, and a horizontally articulated type may be used instead of the vertically articulated type.

[0015] The robot arm 100 has seven joints or axes, which are, in order from the side closest to the base 10, a first joint 11, a second joint 12, a third joint 13, a fourth joint 14, a fifth joint 15, a sixth joint 16, and a seventh joint 17. Each joint corresponds to a joint in the human body, with the first joint 11 corresponding to the waist, the second joint 12 corresponding to the shoulder, the third joint 13 corresponding to the upper arm (twist), the fourth joint 14 corresponding to the elbow, the fifth joint 15 corresponding to the forearm (twist), the sixth joint 16 corresponding to the wrist, and the seventh joint 17 corresponding to the fingertips (twist). The direction of each axis can be designed as appropriate depending on the purpose and use of the robot arm 100, but in this embodiment, the base 10 is placed on a horizontal plane, so the first joint 11 is oriented vertically (perpendicular to the base 10, which is a horizontal plane), the second joint 12 is oriented horizontally (parallel to the base 10, which is a horizontal plane), the third joint 13 is oriented vertically to the second joint 12, the fourth joint 14 is oriented horizontally, the fifth joint 15 is oriented vertically to the fourth joint 14, the sixth joint 16 is oriented horizontally, and the seventh joint 17 is oriented vertically to the sixth joint 16.

[0016] An end effector or robot hand having a shape and function appropriate for the task is attached to the seventh joint 17 at the tip of the robot arm 100. Various types of end effectors are available, such as a grapple for grasping an object, a shovel for scooping an object, a fork for supporting an object from below and transporting it, a hook for hanging and transporting it, and a crane for lifting and transporting it. In this embodiment, an example is described in which a cutting tool is used as the end effector to perform cutting processes such as chamfering and deburring on a workpiece W as a processing object or a workpiece to be processed. Note that this embodiment, which will be described in detail below, is not limited to cutting processes, but can also be applied to processes such as applying a liquid by pressing a roller as a processing section against the processing object, and assembly processes such as fitting by pressing an assembly tool as a processing section against the processing object.

[0017] 2 and 3 schematically show the configuration of a connecting device 30 that constitutes each of the joints 11 to 17 of the robot arm 100. The connecting device 30 in Fig. 2 is applicable to joints that perform a "bending" motion, such as the second joint 12, the fourth joint 14, and the sixth joint 16 in Fig. 1, and the connecting device 30 in Fig. 3 is applicable to joints that perform a "twisting" motion, such as the first joint 11, the third joint 13, the fifth joint 15, and the seventh joint 17 in Fig. 1.

[0018] 2, the connecting device 30 connects a first link 41 as a first member and a second link 42 as a second member so as to allow relative movement. The connecting device 30 corresponds to a joint in the human body, and the first link 41 and the second link 42 connected to each other by the connecting device 30 correspond to bones in the human body.

[0019] The first link 41 and the second link 42 move relative to each other in various ways depending on the manner of connection by the connecting device 30. In this embodiment, an example will be described in which the first link 41 and the second link 42 rotate relative to each other around a rotation axis A that is perpendicular to the extension direction of the first link 41 and the second link 42.

[0020] The relative movement between the first link 41 and the second link 42 is not limited to rotational movement but may be translational movement. For example, the first link 41 and the second link 42 may be configured to translate relatively in a direction perpendicular to the extension direction of the first link 41 and the second link (direction perpendicular to the plane of the paper in FIG. 2), or the first link 41 and the second link 42 may be configured to translate relatively in a direction parallel to the extension direction of the first link 41 and the second link (up and down direction in FIG. 2).

[0021] The coupling device 30 includes a housing 31, a control board 32, a motor 33, a reducer 34, an elastic member 35, and an output flange 36. The housing 31 has a rotationally symmetric shape about the rotation axis A, and accommodates the components 32 to 36 of the coupling device 30 therein. On the outer periphery of the housing 31, a first mounting portion 311 to which the first link 41 is attached and a second mounting portion 312 to which the second link 42 is attached are provided.

[0022] The first link 41, which is fixed to the housing 31 at the first attachment portion 311, is rotatable integrally with the housing 31 relative to the second link 42 around the rotation axis A. The second attachment portion 312 is an opening on the bottom side (left side in FIG. 2) that connects to the internal space of the housing 31 that houses the components 32 to 36. The second link 42 is attached to the coupling device 30 via the output flange 36 provided in this opening.

[0023] The control board 32 controls the coupling device 30 under the control of a central control device (not shown) that controls the entire robot arm 100. For example, the control board 32 generates drive commands for the motor 33, performs adaptive control based on measurement data from an output shaft encoder (not shown), detects torque based on elastic deformation of the elastic member 35, and performs adaptive control based on the detected torque. The motor 33 is an actuator that generates power to rotate the second link 42 around the rotation axis A in response to a drive command from the control board 32. The reducer 34 reduces the rotational speed of the motor 33 using gears or the like, and generates torque proportional to the reduction ratio.

[0024] The elastic member 35 is provided in series between the motor 33 and reducer 34 as a power source and the second link 42 as a load rotationally driven by the power, and constitutes a series elastic actuator (SEA) in the coupling device 30. Even if a human working together with the robot arm 100 as a collaborative robot collides with the robot arm 100, the elastic deformation of the elastic member 35 absorbs the impact, improving safety. Furthermore, the elastic member 35 can store and release the power generated by the motor 33 and reducer 34 and the external force applied to the second link 42 as elastic energy, thereby achieving efficient movement similar to that of human muscles. In particular, when a cutting tool for chamfering, deburring, or the like is pressed against the workpiece W as described below, the elastic deformation of the elastic member 35 absorbs the impact of contact between the cutting tool and the workpiece W, while the elastic force allows the cutting tool to be pressed against the workpiece W efficiently.

[0025] The elastic member 35 is a member that imparts elasticity to the connecting device 30, and is formed of any elastic body such as a spring or rubber. Furthermore, in addition to or instead of the elastic member 35, a resistance imparting member that imparts resistance to the relative rotation of the first link 41 and the second link 42 of the connecting device 30 may be provided. Examples of resistance imparting members include those that impart resistance through mechanical friction and those that impart resistance through the viscosity of a viscous fluid such as oil or grease. The elasticity of the elastic member 35 and the resistance imparted by the resistance imparting member may be variable by the control board 32.

[0026] The elastic member 35 also functions as a torque sensor that detects torque due to an external force. Torque due to an external force causes elastic deformation of the elastic member 35, and torque can be detected based on the amount of elastic deformation. To measure the amount of elastic deformation of the elastic member 35, various displacement sensors, such as magnetostrictive sensors, strain gauges, piezoelectric elements, polarizing elements, and capacitance sensors, can be attached to the surface of the elastic member 35. The amount of elastic deformation measured by the displacement sensor is converted into torque by a computing device or the like mounted on the control board 32. The connecting device 30 may also be provided with a force sensor that detects force applied to the connecting device 30 by the elastic member 35 and / or other members. Hereinafter, force sensors and torque sensors are collectively referred to as force sensors.

[0027] In this way, a force sensor capable of detecting the applied force and / or torque may be attached to the connecting device 30. Based on the force and / or torque applied to the connecting device 30 constituting each of the joints 11 to 17, the force and / or torque that the end effector or robot hand serving as a processing unit or machining unit attached to the tip of the robot arm 100 receives from the workpiece W serving as a processed object or workpiece can be calculated. In other words, the force sensor attached to the connecting device 30 can indirectly detect the force and / or torque that the end effector receives from the workpiece W. In addition to or instead of these force sensors, a force sensor may be attached to the end effector to directly detect the force and / or torque that the end effector receives from the workpiece W.

[0028] In the above configuration, the reducer 34, elastic member 35, and resistance-imparting member each constitute a flexibility-imparting portion that imparts flexibility to the connecting device 30 as a joint. Here, flexibility refers to the ease with which a joint bends, and a joint is said to be flexible when it can bend in response to an external force. For example, the reducer 34 generates torque proportional to the reduction ratio, so by lowering the reduction ratio, a highly flexible state in which the joint can bend easily in response to an external force can be achieved. Furthermore, the elastic member 35 and resistance-imparting member can be said to impart flexibility to the joint because they allow the joint to bend while generating elastic force or resistance that resists external forces. Note that it is sufficient to provide a flexibility-imparting portion in at least one joint; it is not necessary to provide a flexibility-imparting portion in all seven joints 11 to 17 in FIG. 1.

[0029] The output shaft encoder (not shown) is a rotary encoder that measures the rotational position of the second link 42 about the rotation axis A relative to the first link 41. The output flange 36 transmits the torque generated by the reducer 34 to the second link 42 via the elastic member 35, causing the second link 42 to rotate about the rotation axis A. A bearing 361 that smooths the rotation of the second link 42 relative to the housing 31 is provided around the output flange 36.

[0030] 3, the first link 41 and the second link 42 rotate relative to each other about a rotation axis B that is parallel to the extension direction of each link. A notch 37 provided on the housing 31 on the side of the second link 42 prevents the housing 31 from interfering with the rotation of the second link 42 around the rotation axis B caused by the motor 33.

[0031] Next, we will explain the cutting tool 20, which is attached to the seventh joint 17 (FIG. 1) at the tip of the robot arm 100 and serves as a processing unit or machining unit that performs cutting processes such as chamfering and deburring of the workpiece W. FIG. 4 schematically shows the operation of pressing the cutting tool 20 against the workpiece W at the start of cutting. The robot arm 100 in this figure has the same configuration as in FIG. 1, but is illustrated in a greatly simplified manner. For example, of the seven axes or joints 11 to 17, only the second joint 12, the fourth joint 14, and the seventh joint 17 are illustrated schematically in this figure, and the other four joints 11, 13, 15, and 16 are not shown.

[0032] The cutting tool 20 is driven to approach the workpiece W, which is an object to be processed or machined, within the plane of FIG. 4. In the example of FIG. 4, the second joint 12 is primarily driven to rotate clockwise, and the cutting tool 20 attached to the seventh joint 17 at the tip of the robot arm 100 comes into contact with the workpiece W. At this time, the elastic members 35 attached to the second joint 12 and / or the other joints 11, 13 to 17 elastically deform, thereby effectively absorbing the impact of contact between the cutting tool 20 and the workpiece W. Each elastic member 35 elastically deforms to improve the adhesion between the contacting cutting tool 20 and the workpiece W, thereby automatically fine-tuning the position and orientation of the cutting tool 20 relative to the workpiece W. In this way, even when the positioning accuracy of the cutting tool 20 relative to the workpiece W by a central control device (not shown) that controls the entire robot arm 100 is low (e.g., during direct teaching of the robot arm 100), the series elastic actuators attached to the joints 11 to 17 can reliably press the cutting tool 20 against the workpiece W. Furthermore, the cutting tool 20 pressed against the workpiece W by the elastic force of the elastic member 35 can effectively perform cutting processes such as chamfering and deburring on the portion of the workpiece W to be machined or processed.

[0033] In general, a robot arm 100 can realize translational and rotational movement of an end effector about each axis (X-axis, Y-axis, Z-axis, etc.) in three-dimensional space (e.g., XYZ space) using six axes or joints. In contrast, the robot arm 100 of this embodiment (FIG. 1) has seven axes or joints 11-17. That is, the robot arm 100 of this embodiment includes multiple redundant joints that impart redundant movement to a cutting tool 20 serving as an end effector. These redundant joints increase the degree of freedom of the posture that the robot arm 100 can assume, allowing the flexibility or compliance that each joint 11-17 imparts to the cutting tool 20 to be adjusted based on variations in the position and posture of each joint 11-17 relative to the cutting tool 20. Providing redundant axes in this manner makes it possible to realize a highly versatile robot arm 100 that can be adapted to various workpieces W with different shapes and materials.

[0034] FIG. 5 is a schematic enlarged view of a cutting tool 20 as a processing section pressed against a workpiece W. The cutting tool 20 is provided at the tip of the robot arm 100 and processes or processes the workpiece W as a workpiece or processed object. Specifically, the cutting tool 20 is provided at the tip of a link that has the seventh joint 17 of the robot arm 100 as its tip end and the sixth joint 16 as its base end or rear end. The cutting tool 20 includes one or more sharp blades or cutting edges, a grinding part such as a file, etc., to perform cutting processes such as chamfering and deburring on the workpiece W. The surface of each blade is approximately parallel to the plane of FIG. 5 . By moving the cutting tool 20 relative to the workpiece W in a processing direction or machining direction intersecting the plane of the plane of the paper (e.g., a direction perpendicular to the plane of the paper), cutting processes such as chamfering and deburring are performed on the portion of the workpiece W to be processed (the upper left corner in the example of FIG. 5 ).

[0035] In FIG. 5 , the force F and torque T superimposed on the cutting tool 20 visually represent, by arrows, the magnitude and direction of the force and torque, respectively, that the cutting tool 20 receives from the portion of the workpiece W being machined with the cutting tool 20. These are displayed in the line of sight of the operator of the robot arm 100 by a display device separate from the robot arm 100, which will be described later. The operator of the robot arm 100 can safely and efficiently operate the robot arm 100 while keeping both the robot arm 100 (particularly the cutting tool 20 as an end effector) and operational information of the robot arm 100, such as the force F and torque T, in view. In particular, by visualizing to the operator operational information that is not directly visible to humans, such as the force F and torque T applied to the cutting tool 20 as an end effector or processing unit, it is possible to effectively prevent the force F and / or torque T from becoming too large, which would damage the cutting tool 20 and / or the workpiece W, and the force F and / or torque T from becoming too small, which would reduce the efficiency of the cutting process.

[0036] Furthermore, since the display device that displays operational information of the robot arm 100, such as the force F and the torque T, is separate from the robot arm 100, the operator of the robot arm 100 can always visually recognize the operational information regardless of the relative position or posture of the robot arm 100 or obstacles between the robot arm 100 and the operator. In other words, as will be described later, the display device can be positioned between the robot arm 100 and the operator in the line of sight of the operator, so that the operational information, such as the force F and the torque T, displayed by the display device is always positioned in the foreground of the robot arm 100 in the operator's field of vision. Therefore, regardless of the position or posture of the robot arm 100 in the background, the operator can always visually recognize the operational information of the robot arm 100 in the foreground.

[0037] 5, when the operational information of the robot arm 100, such as the force F or the torque T, is represented by an image such as an arrow, the magnitude of each can be represented by the length, thickness, color, movement, etc. of the arrow. Furthermore, the operational information of the robot arm 100, such as the force F or the torque T, does not necessarily have to be superimposed on the cutting tool 20, which is the visually inspected part, but may be displayed in any manner at any position on the screen of a display device placed in the operator's line of sight. Furthermore, the display manner of the operational information of the robot arm 100, such as the force F or the torque T, is not limited to arrows, and the magnitude and direction of each may be displayed by numbers, letters, symbols, colors, etc.

[0038] FIG. 6 is a functional block diagram of the vision device 5 of the robot arm 100. The vision device 5 includes a motion information acquisition unit 51, a display device 52, an image acquisition unit 53, a viewing direction detection unit 54, and a motion detection unit 55. These functional blocks are realized by the cooperation of hardware resources, such as a computer's central processing unit, memory, input devices, output devices, and peripheral devices connected to the computer, and software executed using these resources. Regardless of the type of computer or its installation location, each of the above functional blocks may be realized by the hardware resources of a single computer or by a combination of hardware resources distributed across multiple computers. For example, some or all of the functional blocks of the vision device 5 may be realized in a distributed or centralized manner by computers or processors installed on the same premises or in the same building as the robot arm 100, or may be realized in a distributed or centralized manner by computers or processors installed on a different premises or in a different building than the robot arm 100.

[0039] The motion information acquisition unit 51 acquires motion information of each unit of the robot arm 100 equipped with the cutting tool 20 as a processing unit that processes a workpiece W as a processing object. As illustrated in FIG. 5, the motion information of the robot arm 100 includes at least one of a force F and a torque T that the cutting tool 20 receives from the workpiece W. The force F and / or the torque T applied to the cutting tool 20 may be acquired directly from a force sensor attached to the cutting tool 20, or may be acquired (calculated) indirectly from force sensors attached to the joints 11 to 17.

[0040] Other operational information of the robot arm 100 includes the speed and acceleration of the cutting tool 20 in the processing direction or machining direction (for example, the direction perpendicular to the paper surface of Figure 6) (or, more generally, the processing content, processing speed, processing efficiency, etc. by the processing unit), the force F and / or torque T applied to the cutting tool 20, or the current cutting amount and / or future cutting amount of the workpiece W estimated based on the operational settings of the robot arm 100 and / or cutting tool 20 (or, more generally, the past and / or future processing results by the processing unit), impedance matching ellipsoids and compliance ellipsoids that visually represent the flexibility or compliance imparted to the cutting tool 20 according to the position and posture of each joint 11 to 17 in a robot arm 100 with seven or more axes including redundant axes, the force and / or torque applied to each joint 11 to 17, the load on each moving part of the robot arm 100 such as the cutting tool 20 and each joint 11 to 17, power consumption, current operating history, and diagnostic and estimated results of abnormalities, etc. In this way, the operation information of the robot arm 100 is information about the past, present, and future operations of the robot arm 100, and includes any information that cannot be directly seen by the operator of the robot arm 100.

[0041] The display device 52 is a display device separate from the robot arm 100 that can be viewed by the operator of the robot arm 100 at the same time as the robot arm 100, and displays the operation information acquired by the operation information acquisition unit 51. Three types of display devices 52 are shown as examples in Figure 6.

[0042] A first example of the display device 52 is a transmissive display 52A. The transmissive display 52A is a glasses-type display that can be worn by an operator (indicated as "Operator (Local)" in FIG. 6, hereinafter also referred to as the local operator) who operates the robot arm 100 while visually viewing the actual robot arm 100 in an area where the robot arm 100 is installed, so as to be positioned in the line of sight of the local operator. The local operator wearing the transmissive display 52A can visually view the robot arm 100 through the transparent screen. At the same time, operation information acquired by the operation information acquisition unit 51 is displayed on the screen of the transmissive display 52A. For example, when the local operator operates or monitors the cutting tool 20 and / or the workpiece W while visually viewing the cutting tool 20 through the screen of the transmissive display 52A, an image representing operation information such as the force F and torque T applied to the cutting tool 20 is displayed on the screen of the transmissive display 52A, superimposed on the cutting tool 20 viewed by the local operator. FIG. 5 schematically illustrates the field of view of the local operator at this time. In FIG. 5, the force F and torque T in the foreground are displayed by the transmissive display 52A, and the cutting tool 20 and workpiece W in the background are viewed by the local operator through the screen of the transmissive display 52A.

[0043] A second example of the display device 52 is a mobile terminal 52B such as a tablet carried by the local operator when operating or monitoring the robot arm 100. The mobile terminal 52B may be a head-mounted display or the like that can be worn by the local operator so as to be positioned in the local operator's line of sight. On the screen of the mobile terminal 52B, an image of the robot arm 100 captured by a camera CAM installed to capture an image of the robot arm 100 or an image of the robot arm 100 captured by a camera (not shown) equipped in the mobile terminal 52B itself is displayed in the background. Then, operation information such as force F and torque T acquired by the operation information acquisition unit 51 is displayed in the foreground on the screen of the mobile terminal 52B. Therefore, the local operator can simultaneously view the robot arm 100 to be operated or monitored and the operation information on the screen of the mobile terminal 52B. For example, as shown in FIG. 5, an image in which the robot arm 100 in the background and the operation information in the foreground are combined is displayed on the screen of the mobile terminal 52B. In this way, since the mobile terminal 52B that displays the operation information of the robot arm 100 is separate from the robot arm 100, the on-site operator of the robot arm 100 can visually recognize the operation information regardless of the relative position or posture of the robot arm 100.

[0044] A third example of the display device 52 is a remote terminal 52C such as a monitor that is viewed by an operator (indicated as "Operator (Remote)" in FIG. 6, hereinafter also referred to as "remote operator") who operates the robot arm 100 while visually checking the robot arm 100 from a remote location outside the area where the robot arm 100 is installed, when operating or monitoring the robot arm 100. The remote terminal 52C may be a head-mounted display or the like that can be worn by the remote operator so as to be positioned in the remote operator's line of sight. On the screen of the remote terminal 52C, an image of the robot arm 100 acquired by the image acquisition unit 53 from a camera CAM that the remote operator can operate is displayed in the background. Then, operation information such as force F and torque T acquired by the operation information acquisition unit 51 is displayed in the foreground on the screen of the remote terminal 52C. Therefore, the remote operator can simultaneously view the robot arm 100 to be operated or monitored and the operation information on the screen of the remote terminal 52C. For example, as shown in FIG. 5, an image in which the robot arm 100 in the background and the operation information in the foreground are combined is displayed on the screen of the remote terminal 52C. In this way, since the remote terminal 52C that displays the operation information of the robot arm 100 is separate from the robot arm 100, the remote operator of the robot arm 100 can visually recognize the operation information regardless of the position or posture of the robot arm 100.

[0045] The viewing direction detection unit 54 detects the viewing direction of the operator of the robot arm 100. For example, the viewing direction detection unit 54 detects the line of sight and / or the direction of the operator's head. The viewing direction detection unit 54 is configured by, for example, an eye tracker or an inertial sensor mounted on the transmissive display 52A or a head-mounted display worn by the operator, an eye tracker or an inertial sensor mounted on another device worn or carried by the operator, or an image sensor that detects the operator's line of sight, position, posture, etc. Note that when a mobile terminal (e.g., 52B) such as a tablet carried by the operator is used as the display device 52, the viewing direction detection unit 54 may detect the direction of a camera included in the mobile terminal as the line of sight of the virtual operator, or may detect the position or posture of the mobile terminal as the direction of the virtual operator's head.

[0046] The operation information acquisition unit 51 acquires operation information about a part of the robot arm 100 that corresponds to the viewing direction detected by the viewing direction detection unit 54. The display device 52 displays operation information about the part when the operator is viewing the direction of the part. For example, when the cutting tool 20 and / or the workpiece W are in the viewing direction detected by the viewing direction detection unit 54, the operation information acquisition unit 51 acquires operation information about the cutting tool 20, and the display device 52 displays the operation information about the cutting tool 20 as shown in FIG. 5 .

[0047] Furthermore, when the base 10 of the robot arm 100 is in the viewing direction detected by the viewing direction detection unit 54, the operation information acquisition unit 51 acquires operation information regarding the entire robot arm 100, such as the overall load and power consumption of the robot arm 100, and the display device 52 (the bold rectangular frame in FIG. 7 represents the screen of the display device 52) may display the load, power consumption, and the like using numbers, graphs, or the like, as schematically shown in FIG. 8. Furthermore, when the cutting tool 20 and / or the workpiece W are out of the viewing direction detected by the viewing direction detection unit 54 but are reflected in the operator's field of view or on the edge of the screen of the display device 52, as schematically shown in FIG. 8, the display device 52 may display, for example, an elliptical image simply representing only the magnitude of the force F and / or the torque T, instead of the arrow images of the force F and the torque T in FIG. 5. The magnitude of the force F and / or the torque T can be represented, for example, by the color, size, movement, or the like of the elliptical image. As described above, according to this embodiment, the part of the robot arm 100 for displaying the operation information, the operation information to be displayed, and the display mode of the operation information can be changed depending on the viewing direction detected by the viewing direction detection unit 54.

[0048] The motion detection unit 55 detects the motion or gesture of the operator. The motion detection unit 55 is configured, for example, by an inertial sensor mounted on the transmissive display 52A or head-mounted display worn by the operator, an inertial sensor mounted on another device worn or carried by the operator, an image sensor that detects the position, posture, motion, etc. of the operator, etc. The display device 52 changes the display mode of the motion information in accordance with the motion detected by the motion detection unit 55. For example, when the motion detection unit 55 detects the motion of the operator spreading both arms, the display device 52 enlarges the motion information (for example, by lengthening or thickening the image of the arrow representing the force F and / or torque T). When the mobile terminal 52B or the remote terminal 52C with a non-transmissive screen is used as the display device 52, it is preferable that the display mode of the image of the robot arm 100 displayed in the background of the motion information also be changed in accordance with the motion detected by the motion detection unit 55.

[0049] The present invention has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of the respective components and treatment processes, and that such modifications are also within the scope of the present invention.

[0050] The functional configuration of each device described in the embodiments can be realized by hardware resources, software resources, or a combination of hardware and software resources. Examples of hardware resources include processors, ROMs, RAMs, and other LSIs. Examples of software resources include operating systems, applications, and other programs. [Explanation of symbols]

[0051] 5 Vision device, 20 Cutting tool, 30 Connecting device, 35 Elastic member, 41 First link, 42 ​​Second link, 51 Operation information acquisition unit, 52 Display device, 52A Transmissive display, 52B Portable terminal, 52C Remote terminal, 53 Image acquisition unit, 54 Viewing direction detection unit, 55 Operation detection unit, 100 Robot arm.

Claims

1. an operation information acquisition unit that acquires operation information of a robot having a processing unit that processes a workpiece; a display device separate from the robot that can be viewed by an operator of the robot at the same time as the robot, and that displays the motion information acquired by the motion information acquisition unit; Equipped with further comprising a viewing direction detection unit that detects the viewing direction of the operator, the motion information acquisition unit acquires motion information relating to a part of the robot according to the viewing direction; The display device is a visual confirmation device that displays the operation information related to the part when the operator is visually confirming the direction of the part.

2. The viewing device according to claim 1 , wherein the display device is wearable by an operator so as to be positioned in the operator's line of sight.

3. The viewing device according to claim 1 or 2, wherein the display device is a transmissive display.

4. The viewing device according to claim 1 , wherein the viewing direction detection unit detects the direction of the operator's line of sight and / or head.

5. The visual confirmation device according to claim 1 , wherein the operation information includes at least one of a force and a torque that the processing section receives from the object to be processed.

6. 6. The visualizing device according to claim 5, wherein the display device displays an image representing the magnitude and / or direction of at least one of the force and the torque acquired by the motion information acquiring unit, superimposed on the processing unit being viewed by the operator.

7. The visual recognition device according to claim 5 , wherein the motion information acquisition unit includes a force sensor attached to the processing unit and capable of detecting at least one of the force and the torque.

8. the robot is a robot arm including a plurality of links, a joint connecting the plurality of links to enable relative movement, and the processing unit provided at a tip end of at least one of the links, the motion information acquisition unit includes a force sensor attached to the joint and capable of detecting at least one of the force and the torque; A viewing device according to any one of claims 5 to 7.

9. further comprising a motion detection unit that detects a motion of the operator, the display device changes a display mode of the motion information in accordance with the motion detected by the motion detection unit. A viewing device according to any one of claims 1 to 8.

10. an operation information acquisition step of acquiring operation information of a robot including a processing unit that processes a workpiece; a display step of displaying the motion information acquired in the motion information acquisition step on a display device separate from the robot that can be viewed by an operator of the robot simultaneously with the robot; Equipped with further comprising a viewing direction detecting step of detecting a viewing direction of the operator, the motion information acquiring step acquires motion information relating to a part of the robot according to the viewing direction; The display step is a visual recognition method in which, when the operator is visually recognizing the direction of the part, the operation information related to the part is displayed.

11. an operation information acquisition step of acquiring operation information of a robot including a processing unit that processes a workpiece; a display step of displaying the motion information acquired in the motion information acquisition step on a display device separate from the robot that can be viewed by an operator of the robot simultaneously with the robot; on the computer, causing the computer to further execute a gaze direction detection step of detecting a gaze direction of the operator; the motion information acquiring step acquires motion information relating to a part of the robot according to the viewing direction; The display step is a viewing program that displays the motion information related to the part when the operator views the direction of the part.

Images