Robot system and control method of robot system

Abstract

To provide a robot system which allows a user to accurately recognize a workpiece imageable range imaged by an imaging part.SOLUTION: An appearance inspection system 100 includes: a root arm 12; an imaging part 20 which includes an irradiation part 21 irradiating a workpiece 200 with irradiation light and images the workpiece 200 irradiated with the irradiation light; a display part 45 and a display part 53 on which the workpiece 200 is displayed; and a processing part 41 and a processing part 51 for performing processing of moving the imaging part 20 or the workpiece 200 by the robot arm 12, acquiring the coordinate of the irradiation light in an image captured by the imaging part 20 relatively moved with respect to the workpiece 200, and making a virtual image A of the irradiation light generated by computer graphic overlap the image of the workpiece 200 and displaying the overlapping image on the display part 45 and the display part 53, on the basis of the acquired coordinate of the irradiation light and a control point TCP of the robot arm 12.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] This disclosure relates to a robot system and a method for controlling a robot system.

Background Art

[0002] Conventionally, a teaching device has been known (see, for example, Patent Document 1).

[0003] Patent Document 1 discloses an offline teaching device for a vision sensor. This offline teaching device includes a storage device that stores the shapes and dimensions of a workpiece, a robot that performs work on the workpiece, and a vision sensor that measures the workpiece, and a display that displays images of the workpiece, the robot, and the vision sensor based on the data stored in the storage device. On this display, the modeled workpiece, robot, and vision sensor are arranged, and teaching and adjustment of the vision sensor are performed offline. Specifically, this offline teaching device arranges the vision sensor on the display, images the measurement site of the workpiece with the arranged vision sensor, and generates an image of the vision sensor of the imaged measurement site. Thereby, the user can recognize the imageable range of the workpiece imaged by the vision sensor by visually recognizing the image of the measurement site imaged by the vision sensor.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the offline teaching device described in Patent Document 1, an image of the vision sensor that captures the measurement site of the workpiece is generated based on the modeled workpiece, robot, and vision sensor. Therefore, the modeled workpiece, robot, and vision sensor may include shape errors, position errors, etc. compared to the actual object. For this reason, in an offline teaching device as described in Patent Document 1, the user cannot accurately recognize which part of the workpiece can be imaged by the vision sensor. Therefore, it is desired that the user accurately recognize the imaging range of the workpiece imaged by the imaging unit.

[0006] This disclosure is made to solve the above problems, and one object of this disclosure is to provide a robot system and a control method for a robot system that enable a user to accurately recognize the imaging range of a workpiece imaged by an imaging unit.

Means for Solving the Problems

[0007] The robot system according to the first aspect of this disclosure includes a robot arm, an imaging unit that irradiates the workpiece with irradiation light and captures the workpiece irradiated with the irradiation light, a display unit on which the workpiece is displayed, and the imaging unit or the workpiece is moved by the robot arm to obtain the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece, and based on the obtained coordinates of the irradiation light and the control point of the robot arm, a process of superimposing a virtual image of the irradiation light generated by computer graphics on the image of the workpiece and displaying it on the display unit is performed. Here, superimposing the virtual image on the image of the workpiece and displaying it means a concept that includes superimposing the virtual image on the image of the actual workpiece and displaying it, and superimposing the virtual image on the image of the modeled workpiece and displaying it.

[0008] In the robot system according to the first aspect of this disclosure, as described above, the robot arm moves the imaging unit or the workpiece, acquires the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece, and based on the acquired coordinates of the irradiation light and the control point of the robot arm, a processing unit is provided that performs a process of superimposing a virtual image of the irradiation light generated by computer graphics on the image of the workpiece and displaying it on the display unit. As a result, based on the coordinates of the actual irradiation light and the control point of the actual robot arm, a virtual image of the irradiation light is displayed on the image of the workpiece. That is, since the shape and position are based on actual information, the errors in shape and position are small. For this reason, by the user visually recognizing the virtual image of the irradiation light on the image of the workpiece, it is possible to confirm how the irradiation light hits the workpiece, so that the imageable range of the workpiece imaged by the imaging unit can be accurately recognized.

[0009] The control method of the robot system according to the second aspect of this disclosure includes an irradiation unit that irradiates a workpiece with irradiation light by a robot arm, an imaging unit that images the workpiece irradiated with the irradiation light, or moving the workpiece, acquiring the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece, and based on the acquired coordinates of the irradiation light and the control point of the robot arm, superimposing a virtual image of the irradiation light generated by computer graphics on the image of the workpiece and displaying it on the display unit. Note that superimposing and displaying the virtual image on the image of the workpiece is a concept that includes the meaning of superimposing and displaying the virtual image on the image of the actual workpiece and the meaning of superimposing and displaying the virtual image on the image of the modeled workpiece.

[0010] In the method for controlling a robot system according to the second aspect of this disclosure, as described above, the coordinates of the irradiation light in the image captured by the imaging unit that moves relative to the workpiece are acquired, and based on the acquired coordinates of the irradiation light and the control point of the robot arm, a virtual image of the irradiation light generated by computer graphics is superimposed on the image of the workpiece and displayed on the display unit. As a result, a virtual image of the irradiation light is displayed on the image of the workpiece based on the coordinates of the actual irradiation light and the control point of the actual robot arm. That is, since the shape and position are based on actual information, the errors in shape and position are small. Therefore, by the user visually recognizing the virtual image of the irradiation light on the image of the workpiece, it is possible to confirm how the irradiation light hits the workpiece, so that it is possible to provide a method for controlling a robot system that can accurately recognize the imageable range of the workpiece imaged by the imaging unit.

Advantages of the Invention

[0011] According to the present disclosure, as described above, the user can accurately recognize the imageable range of the workpiece imaged by the imaging unit.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments embodying the present disclosure will be described with reference to the drawings.

[0014] [First Embodiment] The overall configuration of the appearance inspection system 100 according to the first embodiment will be described.

[0015] As shown in FIG. 1, the appearance inspection system 100 is an appearance inspection system for inspecting the appearance of the workpiece 200. The workpiece 200 is, for example, a product or part related to automobiles, agricultural machinery, pottery, or household electrical appliances. The workpiece 200 is not particularly limited. Also, the appearance inspection system 100 is an example of a robot system.

[0016] The appearance inspection system 100 includes a robot 10, an imaging unit 20, a robot controller 30, a wearable display device 40, and a fixed-type display device 50.

[0017] The robot 10 is, for example, a vertical articulated robot. The robot 10 includes a base portion 11 and a robot arm 12 connected to the base portion 11. The base portion 11 is fixed to an installation surface such as a floor, a wall, or a ceiling. Note that the base portion 11 may be attached to a movable cart. The robot arm 12 has a plurality of joints. Each of the plurality of joints has a servo motor as a drive source. Also, the tip of the robot arm 12 holds the imaging unit 20. The robot 10 moves the imaging unit 20 held at the tip of the robot arm 12 with respect to the fixed workpiece 200 by driving the plurality of joints of the robot arm 12.

[0018] The imaging unit 20 is attached to the tip of the robot arm 12. Also, as shown in FIG. 2, the imaging unit 20 includes an irradiation unit 21 that irradiates the workpiece 200 with irradiation light, and images the workpiece 200 irradiated with the irradiation light. Further, the imaging unit 20 includes an imaging element unit 22, and the imaging element unit 22 images the irradiation light irradiated by the irradiation unit 21 and the workpiece 200. For example, the imaging unit 20 is a line-type camera, and is moved along the surface of the workpiece 200 by the robot 10 to perform a scan imaging of the surface of the workpiece 200. Further, the imaging unit 20 includes a processing unit 23. The processing unit 23 performs image processing on the image captured by the imaging element unit 22. Also, the processing unit 23 controls the imaging timing by the imaging unit 20.

[0019] As shown in FIG. 2, the robot controller 30 controls the operation of the robot 10. The robot controller 30 includes a processing unit 31, a storage unit 32, and a signal output unit 33. The processing unit 31 includes a processor and performs various processes related to the operation of the robot 10. The storage unit 32 includes a non-volatile memory and stores various information such as a program for operating the robot 10. The signal output unit 33 includes, for example, an FPGA (Field Programmable Gate Array), and can output the coordinates of the TCP (Tool Center Point), which is the control point of the robot 10, at high speed. Also, the signal output unit 33 can output a plurality of coordinates of the robot 10. For example, the signal output unit 33 can output the coordinates of points in the vicinity of the control point TCP in addition to the coordinates of the control point TCP.

[0020] As shown in FIG. 3, the wearable display device 40 overlays and displays a virtual image A generated by computer graphics on the real image that the user U visually recognizes. That is, the wearable display device 40 is a display unit that displays mixed reality. Further, the wearable display device 40 is worn by the user U. Specifically, the wearable display device 40 includes a goggle-type display unit that is worn on the head of the user U. Further, as shown in FIG. 2, the wearable display device 40 includes a processing unit 41, a storage unit 42, an imaging unit 43, a sensor 44, and a display unit 45. The processing unit 41 includes a processor and performs various processes such as generating the virtual image A. The storage unit 42 includes a non-volatile memory and stores various information such as a program for generating the virtual image A. The imaging unit 43 images the real image. The sensor 44 includes an acceleration sensor, a gyroscope, and the like. The movement of the user U and the like are detected by the sensor 44. As shown in FIG. 4, the display unit 45 is a see-through type display unit, and the user U can visually recognize a real image such as the robot 10 or the workpiece 200 through the display unit 45. Further, a virtual image A generated by computer graphics is overlaid and displayed on the real image on the display unit 45. The virtual image A is, for example, a hologram. The details of the virtual image A will be described later.

[0021] Further, in the first embodiment, as shown in FIG. 5, the fixed display device 50 is fixedly arranged without being worn by the user U. Then, an image of the three-dimensional model M of the workpiece 200 is displayed on the fixed display device 50. The image of the three-dimensional model M of the workpiece 200 is, for example, an image of the workpiece 200 created using CAD (Computer Aided Design). As shown in FIG. 2, the fixed display device 50 includes a processing unit 51, a storage unit 52, and a display unit 53. The processing unit 51 includes a processor and performs various processes such as generating the virtual image A. The storage unit 52 includes a non-volatile memory and stores various information such as a program for generating the virtual image A. The display unit 53 is, for example, a liquid crystal display, an organic EL display, or the like.

[0022] (Function of displaying the imaging range) The appearance inspection system 100 has a function of displaying the imaging range of the work 200 to be inspected. Specifically, a virtual image A of the irradiation light irradiated from the irradiation unit 21 to the work 200 is generated by computer graphics, and the virtual image A of the irradiation light is displayed on the display unit 45 of the wearable display device 40 so as to overlap the real image visually recognized by the user U. Also, the virtual image A of the irradiation light is displayed on the display unit 53 of the fixed display device 50 so as to overlap the image of the three-dimensional model M of the work 200. Further, the function of displaying the imaging range is used, for example, when the user U teaches an operation to the robot 10 for inspecting the work 200. Hereinafter, a control method of the appearance inspection system 100 for displaying the imaging range will be described in detail.

[0023] As shown in FIG. 6, in step S1, the processing unit 31 of the robot controller 30 receives an operation of the teaching pendant by the user U. Then, based on the received operation, the processing unit 31 moves the robot arm 12 to relatively move the imaging unit 20 including the irradiation unit 21 with respect to the work 200.

[0024] In step S2, in the first embodiment, the processing unit 23 of the imaging unit 20 acquires the coordinates of the irradiation light in the image captured by the imaging unit 20 that is relatively moved with respect to the work 200. Specifically, while the imaging unit 20 is in the on state, the processing unit 23 acquires the image captured by the image sensor unit 22.

[0025] In step S3, the processing unit 23 acquires the coordinates of the irradiation light in the image captured by the imaging unit 20 that is relatively moved with respect to the work 200. First, the processing unit 23 executes preprocessing of the acquired image.

[0026] In step S4, in the first embodiment, the processing unit 41 of the wearable display device 40 and the processing unit 51 of the fixed display device 50 respectively superimpose a virtual image A of the irradiation light generated by computer graphics on the image of the workpiece 200 based on the coordinates of the acquired irradiation light and the control point TCP of the robot arm 12, and perform processing to display it on the display unit 45 of the wearable display device 40 and the display unit 53 of the fixed display device 50.

[0027] Also, in the first embodiment, the processing unit 41 of the wearable display device 40 and the processing unit 51 of the fixed display device 50 respectively superimpose a virtual image A of the irradiation light corresponding to the portion where the pixel value of the image of the irradiation light captured by the imaging unit 20 is equal to or greater than the threshold on the image of the workpiece 200, and perform processing to display it on the wearable display device 40 and the fixed display device 50.

[0028] The processing of the processing unit 41 of the wearable display device 40 will be described in detail. As shown in FIG. 1, a marker 210 for aligning the real coordinates and the coordinates of the wearable display device 40 is arranged near the workpiece 200. The marker 210 is, for example, a QR code (registered trademark). In a state where the user U wears the wearable display device 40 on the head, the imaging unit 43 of the wearable display device 40 captures the marker 210 by visually recognizing the marker 210. Then, the processing unit 41 of the wearable display device 40 aligns the real coordinates and the coordinates of the wearable display device 40 based on information such as the QR code (registered trademark) obtained from the marker 210 and the robot coordinate system of the robot 10. Note that the coordinates of the actual workpiece 200 are preset based on the robot coordinate system. Also, the coordinates of the actual workpiece 200 are input from the robot controller 30 to the wearable display device 40. Then, the processing unit 41 executes processing to display the virtual image A of the irradiation light on the wearable display device 40 so as to overlap the image of the actual workpiece 200. Note that as the position of the user U's head moves, the position of the virtual image A of the irradiation light displayed on the display unit 45 of the wearable display device 40 also changes.

[0029] The processing of the processing unit 51 of the fixed display device 50 will be described in detail. As shown in FIG. 5, in the first embodiment, the processing unit 51 of the fixed display device 50 performs a process of causing the virtual image A of the irradiation light to be displayed on the display unit 53 of the fixed display device 50 so as to overlap the image of the three-dimensional model M of the workpiece 200. As described above, the processing unit 51 of the fixed display device 50 overlaps and displays the virtual image A of the irradiation light on the image of the three-dimensional model M of the workpiece 200. Further, the image of the three-dimensional model M of the workpiece 200 displayed on the display unit 53 of the fixed display device 50 can be rotated and moved by the operation of the user U. Then, the virtual image A of the irradiation light moves as the image of the three-dimensional model M rotates and moves.

[0030] Next, in step S5, the processing unit 31 of the robot controller 30 determines whether or not the operation of moving the robot arm 12 by the user U has ended. If the result in step S6 is no, the operations from step S1 to step S4 are repeated. That is, in the first embodiment, the processing unit 31 of the robot controller 30 moves the imaging unit 20 by the robot arm 12, and the processing unit 23 of the imaging unit 20 sequentially acquires the coordinates of the irradiation light in the image captured by the imaging unit 20 that is relatively moved with respect to the workpiece 200. Then, based on the coordinates of the irradiation light sequentially acquired by the imaging unit 20, the processing unit 41 of the wearable display device 40 and the processing unit 51 of the fixed display device 50 perform a process of sequentially superimposing the virtual image A of the linear irradiation light on the image of the workpiece 200 and displaying it on the display unit 45 of the wearable display device 40 and the display unit 53 of the fixed display device 50. That is, on the display unit 45 of the wearable display device 40 and the display unit 53 of the fixed display device 50, a virtual image A of a single linear irradiation light whose position changes moment by moment is displayed. If the result in step S6 is yes, the control process of the appearance inspection system 100 for displaying the imaging range ends.

[0031] (Effect of the First Embodiment) The appearance inspection system 100 acquires the coordinates of the irradiation light in the image captured by the imaging unit 20 that is relatively moved with respect to the workpiece 200, and based on the acquired coordinates of the irradiation light and the control point TCP of the robot arm 12, superimposes the virtual image A of the irradiation light generated by computer graphics on the image of the workpiece 200 and performs a process of displaying it on the display unit 45 and the display unit 53. Thereby, based on the coordinates of the actual irradiation light and the control point TCP of the actual robot arm 12, the virtual image A of the irradiation light is displayed on the image of the workpiece 200. That is, since the shape and position are based on actual information, the errors in the shape and position are small. For this reason, when the user U visually recognizes the virtual image A of the irradiation light on the image of the workpiece 200, it is possible to confirm how the irradiation light hits the workpiece 200, so that the imaging range of the workpiece 200 imaged by the imaging unit 20 can be accurately recognized.

[0032] The processing unit 31 moves the imaging unit 20 by the robot arm 12. The processing unit 41 and the processing unit 51 sequentially acquire the coordinates of the irradiation light in the image captured by the imaging unit 20 that is relatively moved with respect to the workpiece 200, and based on the sequentially acquired coordinates of the irradiation light, sequentially superimpose the virtual image A of the linear irradiation light on the image of the workpiece 200 and perform a process of displaying it on the display unit 45 and the display unit 53. Thereby, the user U can recognize the inspectable range according to the current posture of the robot arm 12 by visually recognizing the virtual image A of the linear irradiation light.

[0033] The processing unit 41 of the wearable display device 40 performs a process of displaying the virtual image A of the irradiation light on the display unit 45 so as to overlap the image of the actual workpiece 200. Thereby, since the virtual image A of the irradiation light is displayed so as to overlap the image of the actual workpiece 200, the user U can accurately recognize the inspectable range with respect to the actual workpiece 200.

[0034] The display unit 45 of the wearable display device 40 superimposes and displays a virtual image A generated by computer graphics on the real image viewed by the user U. The processing unit 41 performs a process of causing the display unit 45 of the wearable display device 40 to display the virtual image A of the irradiation light so as to overlap the image of the real workpiece 200 viewed by the user U. As a result, since the wearable display device 40 is worn by the user U, the virtual image A of the irradiation light also moves in response to the movement of the user U. As a result, the user U can confirm the real workpiece 200 and the inspectable range from various angles.

[0035] The processing unit 51 of the fixed display device 50 performs a process of causing the display unit 53 to display the virtual image A of the irradiation light so as to overlap the image of the three-dimensional model M of the workpiece 200. As a result, in response to the rotation and movement of the three-dimensional model M of the workpiece 200 on the display unit 53, the virtual image A of the irradiation light also moves, so that the workpiece 200 and the inspectable range can be confirmed from various angles.

[0036] [Second Embodiment] The appearance inspection system 100 according to the second embodiment will be described. In the appearance inspection system 100 according to the second embodiment, a virtual image A1 of belt-shaped irradiation light is displayed.

[0037] Specifically, similar to the first embodiment, the processing unit 31 of the robot controller 30 moves the imaging unit 20 by the robot 10. Further, the processing unit 23 of the imaging unit 20 sequentially acquires the coordinates of the irradiation light in the image captured by the imaging unit 20 that is relatively moved with respect to the workpiece 200. And in the second embodiment, as shown in FIG. 7, the processing unit 41 of the wearable display device 40 performs a process of superimposing a virtual image A1 of a strip-shaped irradiation light on the image of the actual workpiece 200 and displaying it on the display unit 45 based on the coordinates of the plurality of irradiation lights sequentially acquired. Also, as shown in FIG. 8, the processing unit 51 of the fixed display device 50 performs a process of superimposing a virtual image A1 of a strip-shaped irradiation light on the image of the 3D model M of the workpiece 200 and displaying it on the display unit 53 based on the coordinates of the plurality of irradiation lights sequentially acquired. Specifically, the processing unit 41 of the wearable display device 40 and the processing unit 51 of the fixed display device 50 respectively store the coordinates of the plurality of irradiation lights sequentially acquired in the storage unit 42 of the wearable display device 40 and the storage unit 52 of the fixed display device 50. The storage of the coordinates of the plurality of irradiation lights is during the start to end of the movement of the imaging unit 20 by the robot arm 12. Then, after the end of the movement of the imaging unit 20 by the robot arm 12, the processing unit 41 of the wearable display device 40 reads out the coordinates of the plurality of irradiation lights from the storage unit 42, and superimposes a virtual image A1 of a strip-shaped irradiation light on the image of the actual workpiece 200 based on the read coordinates of the plurality of irradiation lights and displays it on the display unit 45. The processing unit 51 of the fixed display device 50 also performs a similar process, and performs a process of superimposing a virtual image A1 of a strip-shaped irradiation light on the image of the 3D model M of the workpiece 200 and displaying it on the display unit 53. Note that when the movement of the imaging unit 20 by the robot arm 12 is performed along a plurality of paths, a plurality of virtual images A1 of the strip-shaped irradiation light are displayed.

[0038] (Effect of the Second Embodiment) The processing unit 31 moves the imaging unit 20 or the workpiece 200 by the robot. The processing units 41 and 51 sequentially acquire the coordinates of the irradiation light in the image captured by the imaging unit 20 that is relatively moved with respect to the workpiece 200, and based on the coordinates of the plurality of irradiation lights sequentially acquired, perform processing of superimposing the virtual image A1 of the strip-shaped irradiation light on the image of the workpiece 200 and displaying it on the display unit 45 and the display unit 53. Thereby, the user U can recognize the inspectable range within the range where the robot arm 12 moves by visually recognizing the virtual image A1 of the strip-shaped irradiation light.

[0039] (Modification example) It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present disclosure is shown not by the description of the above-described embodiments but by the claims, and further includes all changes (modification examples) within the meaning and scope equivalent to the claims.

[0040] For example, in the above-described first and second embodiments, an example where the robot is a vertically articulated robot is shown, but the present disclosure is not limited to this. In the present disclosure, the robot may be a robot other than a vertically articulated robot.

[0041] Also, in the above-described first and second embodiments, an example where the virtual image of the irradiation light corresponding to the portion where the pixel value of the image of the irradiation light captured by the imaging unit is equal to or greater than the threshold value is displayed on the display unit is shown, but the present disclosure is not limited to this. In the present disclosure, all of the images of the irradiation light captured by the imaging unit may be displayed on the display unit.

[0042] Also, in the above-described first and second embodiments, an example where the virtual image of the irradiation light corresponding to the portion where the pixel value of the image of the irradiation light captured by the imaging unit is equal to or greater than the threshold value is displayed on the display unit is shown, but the present disclosure is not limited to this. In the present disclosure, all of the images of the irradiation light captured by the imaging unit may be displayed on the display unit.

[0043] In addition, in the above-described first and second embodiments, an example in which the virtual image of the irradiation light is displayed on both the wearable display device and the fixed display device has been shown. However, the present disclosure is not limited to this. In the present disclosure, the virtual image of the irradiation light may be displayed on only one of the wearable display device and the fixed display device.

[0044] In addition, an example in which the first embodiment in which the linear virtual image of the irradiation light is displayed on the display unit and the second embodiment in which the strip-shaped virtual image of the irradiation light is displayed on the display unit are separate embodiments has been shown. However, the present disclosure is not limited to this. In the present disclosure, the user may switch between the mode in which the linear virtual image of the irradiation light is displayed on the display unit and the mode in which the strip-shaped virtual image of the irradiation light is displayed on the display unit.

[0045] In addition, in the above-described first and second embodiments, an example in which the wearable display device is a display unit that displays mixed reality has been shown. However, the present disclosure is not limited to this. In the present disclosure, the wearable display device may be a display unit that displays virtual reality. In this case, in the wearable display device, the virtual image of the irradiation light is displayed so as to overlap the image of the three-dimensional model of the workpiece.

[0046] In addition, in the above-described first and second embodiments, an example in which the image of the three-dimensional model of the workpiece is displayed on the fixed display device has been shown. However, the present disclosure is not limited to this. In the present disclosure, an image of the actual workpiece that the user visually recognizes may be displayed on the fixed display device, and the virtual image of the irradiation light may be displayed so as to overlap the image of the actual workpiece.

[0047] In addition, in the above-described first and second embodiments, an example in which the present disclosure is applied to a workpiece having a curved surface has been shown. However, the present disclosure is not limited to this. The present disclosure may be applied to a flat workpiece having no curved surface.

[0048] In addition, in the above-described first and second embodiments, an example in which the imaging unit is a line sensor has been shown. However, the present disclosure is not limited to this. In the present disclosure, the imaging unit may be an area camera that captures a two-dimensional image.

[0049] In addition, in the above-described first and second embodiments, an example in which the robot system of the present disclosure is applied to an appearance inspection system has been shown. However, the present disclosure is not limited thereto. The robot system of the present disclosure may be applied to systems other than the appearance inspection system.

[0050] The functions of the elements disclosed in this specification can be executed using a circuit or a processing circuit including a general-purpose processor, a dedicated processor, an integrated circuit, an ASIC (Application Specific Integrated Circuits), a conventional circuit, and / or a combination thereof that is configured or programmed to execute the disclosed functions. Since a processor includes transistors and other circuits, it is regarded as a processing circuit or a circuit. In the present disclosure, a circuit, a unit, or a means is hardware that executes the listed functions or hardware that is programmed to execute the listed functions. The hardware may be the hardware disclosed in this specification or other known hardware that is programmed or configured to execute the listed functions. When the hardware is a processor considered to be a type of circuit, the circuit, means, or unit is a combination of hardware and software, and the software is used for the configuration of the hardware and / or the processor.

[0051] [Aspect] Those skilled in the art will understand that the above-described exemplary embodiments are specific examples of the following aspects.

[0052] (Aspect 1) A robot arm, An imaging unit that includes an irradiation unit that irradiates light on a workpiece and images the workpiece irradiated with the irradiation light, A display unit on which the workpiece is displayed, The robot arm moves the imaging unit or the workpiece, and acquires the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece. A processing unit that performs a process of superimposing a virtual image of the irradiation light generated by computer graphics on an image of a workpiece based on the coordinates of the obtained irradiation light and the control points of the robot arm and displaying the result on the display unit, and a robot system including the processing unit.

[0053] (Aspect 2) The processing unit moves the imaging unit or the workpiece by the robot arm, sequentially obtains the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece, Based on the sequentially obtained coordinates of the irradiation light, the robot system according to Aspect 1, which performs a process of sequentially superimposing a virtual image of the linear irradiation light on the image of the workpiece and displaying the result on the display unit.

[0054] (Aspect 3) The processing unit moves the imaging unit or the workpiece by the robot arm, sequentially obtains the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece, Based on the sequentially obtained coordinates of a plurality of irradiation lights, the robot system according to Aspect 1, which performs a process of superimposing a virtual image of the strip-shaped irradiation light on the image of the workpiece and displaying the result on the display unit.

[0055] (Aspect 4) The processing unit performs a process of displaying the virtual image of the irradiation light on the display unit so as to overlap the image of the actual workpiece. The robot system according to any one of Aspects 1 to 3.

[0056] (Aspect 5) The display unit includes a display unit of a wearable display device worn by the user, and displays a virtual image generated by computer graphics superimposed on the actual image viewed by the user. The robot system according to aspect 4, wherein the processing unit performs a process of causing the display unit of the wearable display device to display a virtual image of the irradiation light so as to overlap an image of an actual workpiece visually recognized by the user.

[0057] (Aspect 6) The display unit includes a display unit of a fixed display device on which an image of a three-dimensional model of the workpiece is displayed. The robot system according to any one of aspects 1 to 5, wherein the processing unit performs a process of causing the display unit of the fixed display device to display a virtual image of the irradiation light so as to overlap an image of the three-dimensional model of the workpiece.

[0058] (Aspect 7) A robot system control method, comprising: an irradiation unit that irradiates a workpiece with irradiation light by a robot arm; an imaging unit that images the workpiece irradiated with the irradiation light; or moving the workpiece, acquiring coordinates of the irradiation light in an image captured by the imaging unit that is relatively moved with respect to the workpiece; and based on the acquired coordinates of the irradiation light and a control point of the robot arm, superimposing a virtual image of the irradiation light generated by computer graphics on an image of the workpiece and displaying the virtual image on a display unit.

Explanation of Signs

[0059] 12 Robot arm 20 Imaging unit 21 Irradiation unit 31 Processing unit 40 Wearable display device 41 Processing unit 45 Display unit 50 Fixed display device 51 Processing unit 53 Display unit 100 Appearance inspection system (robot system) 200 Workpiece A Virtual image A1 Virtual image M Three-dimensional model TCP Control point

Claims

1. A robot arm, an imaging unit that includes an irradiation unit that irradiates light on a workpiece and images the workpiece irradiated with the irradiation light, a display unit on which the workpiece is displayed, the robot arm moves the imaging unit or the workpiece, and obtains the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece, a processing unit that performs a process of superimposing a virtual image of the irradiation light generated by computer graphics on the image of the workpiece based on the obtained coordinates of the irradiation light and the control point of the robot arm and displaying the superimposed image on the display unit. A robot system comprising:

2. The processing unit, moves the imaging unit or the workpiece by the robot arm, sequentially obtains the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece, Based on the sequentially obtained coordinates of the irradiation light, performs a process of sequentially superimposing a virtual image of the linear irradiation light on the image of the workpiece and displaying the superimposed image on the display unit. The robot system according to claim 1.

3. The processing unit, moves the imaging unit or the workpiece by the robot arm, sequentially obtains the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece, Based on the sequentially obtained coordinates of a plurality of irradiation lights, performs a process of superimposing a virtual image of the strip-shaped irradiation light on the image of the workpiece and displaying the superimposed image on the display unit. The robot system according to claim 1.

4. The processing unit performs a process of displaying a virtual image of the irradiation light on the display unit so as to overlap the image of the actual workpiece. The robot system according to claim 1.

5. The display unit includes a display unit of a wearable display device that superimposes and displays a virtual image generated by computer graphics on a real image visually recognized by a user, The processing unit performs a process of displaying a virtual image of the irradiation light on the display unit of the wearable display device so as to overlap the image of the actual workpiece visually recognized by the user. The robot system according to claim 4.

6. The display unit includes a display unit of a fixed display device on which an image of a three-dimensional model of the workpiece is displayed, The processing unit performs a process of displaying a virtual image of the irradiation light on the display unit of the fixed display device so as to overlap the image of the three-dimensional model of the workpiece. The robot system according to claim 1.

7. An imaging unit that images the workpiece irradiated with irradiation light, including an irradiation unit that irradiates the workpiece with irradiation light by a robot arm, or moves the workpiece, and acquiring the coordinates of the irradiation light in the image captured by the imaging unit that is relatively moved with respect to the workpiece, and displaying, on a display unit, by superimposing a virtual image of the irradiation light generated by computer graphics on an image of the workpiece based on the acquired coordinates of the irradiation light and a control point of the robot arm. A control method for a robot system comprising the above steps.

Images