Inspection systems, inspection methods, and robotics.

Abstract

Invention details;

Description

Inspection system, inspection method, and robot system

[0001] The present disclosure relates to an inspection system, an inspection method, and a robotic system.

[0002] Conventionally, a coating quality inspection method has been known, and such a coating quality inspection method is disclosed, for example, in Japanese Patent Laid-Open Publication No. 05-126758.

[0003] The aforementioned Japanese Patent Laid-Open Publication No. 05-126758 discloses a paint quality inspection method for inspecting the paint quality of a vehicle body as a painted object after painting. In this paint quality inspection method, the vehicle body is first imaged by a television camera. Image information based on the image of the vehicle body is then analyzed by a computer. A determination is then made as to whether or not a paint defect exists, and if a paint defect is determined, the location of the paint defect on the vehicle body is determined. The location of the paint defect is then printed on a picture of the vehicle body drawn on printout paper.

[0004] Japanese Patent Application Publication No. 05-126758

[0005] However, the paint quality inspection method described in JP 05-126758 A is configured to print the location of paint defects on a two-dimensional image of a vehicle body. Therefore, while it is possible to accurately indicate the location of paint defects in the vehicle body image on flat or curved surfaces with small curvatures on the actual vehicle body, it is sometimes difficult to accurately indicate the location of paint defects in the vehicle body image on curved surfaces with large curvatures or complex curves on the actual vehicle body. This disadvantage is also true when detecting objects other than paint defects and when inspecting workpieces other than vehicle bodies. Because the image of the workpiece may not accurately indicate the location of objects on curved surfaces with large curvatures or complex curves, it is desirable to accurately indicate the location of objects even on curved surfaces with large curvatures or complex curves on the workpiece.

[0006] This disclosure has been made to solve the above-mentioned problems, and one object of this disclosure is to provide an inspection system and an inspection method that can accurately indicate the position of an object even if the workpiece has a curved surface with a large curvature or a complex curved surface. Another object of this disclosure is to provide a robot system that can accurately perform work on a workpiece using a working unit.

[0007] An inspection system according to a first aspect of this disclosure comprises a robot, an inspection unit that inspects a workpiece, and a processing unit that performs the following processes: generating a movement path for the robot when the inspection unit inspects the workpiece by moving the inspection unit relative to the workpiece; generating coordinate transformation information based on the generated movement path for converting the coordinate values ​​of the inspection coordinate system of an inspection image obtained by inspecting the workpiece with the inspection unit into coordinate values ​​of a three-dimensional coordinate system that can represent the coordinate values ​​of the workpiece in three dimensions; operating the robot based on the generated movement path to inspect the workpiece with the inspection unit to obtain an inspection image; detecting an object of the workpiece in the obtained inspection image; converting the coordinate values ​​of the inspection coordinate system of the object into coordinate values ​​of the three-dimensional coordinate system based on the generated coordinate transformation information; and displaying the position of the object on the actual workpiece or a three-dimensional image of the workpiece based on the converted coordinate values ​​of the object in the three-dimensional coordinate system.

[0008] In the inspection system according to the first aspect of this disclosure, as described above, the following processes are performed: converting the coordinate values ​​of the inspection coordinate system of the object into coordinate values ​​of a three-dimensional coordinate system based on the generated coordinate transformation information; and displaying the position of the object on the actual workpiece or a three-dimensional image of the workpiece based on the converted coordinate values ​​of the object in the three-dimensional coordinate system. This makes it possible to display the position of the object on the actual workpiece or a three-dimensional image of the workpiece, so that, unlike when the position of the object is displayed on a two-dimensional image of the workpiece, the position of the object can be displayed with high accuracy even on a curved surface with a large curvature or a complex curved surface.

[0009] An inspection method according to a second aspect of this disclosure comprises: generating a movement path for the robot when the inspection unit inspects the workpiece by moving the inspection unit relative to the workpiece using the robot; generating coordinate transformation information, based on the generated movement path, for converting the coordinate values ​​of an inspection coordinate system of an inspection image obtained by inspecting the workpiece using the inspection unit into coordinate values ​​of a three-dimensional coordinate system that can represent the coordinate values ​​of the workpiece in three dimensions; operating the robot based on the generated movement path to inspect the workpiece using the inspection unit, thereby obtaining an inspection image; detecting an object of the workpiece in the obtained inspection image; converting the coordinate values ​​of the inspection coordinate system of the object to coordinate values ​​of a three-dimensional coordinate system based on the generated coordinate transformation information; and indicating the position of the object on the actual workpiece or a three-dimensional image of the workpiece based on the converted coordinate values ​​of the three-dimensional coordinate system of the object.

[0010] In the inspection method according to the second aspect of this disclosure, as described above, the coordinate values ​​of the inspection coordinate system of the object are converted into coordinate values ​​of a three-dimensional coordinate system based on the coordinate transformation information generated based on the movement path of the robot, and the position of the object is displayed on the actual workpiece or a three-dimensional image of the workpiece based on the converted coordinate values ​​of the object in the three-dimensional coordinate system. This makes it possible to display the position of the object on the actual workpiece or a three-dimensional image of the workpiece, and therefore, unlike when the position of the object is displayed on a two-dimensional image of the workpiece, it is possible to provide an inspection method that can accurately display the position of the object even on a curved surface with a large curvature or a complex curved surface.

[0011] A robot system according to a third aspect of this disclosure includes a robot, a working unit that performs work on a workpiece, and a processing unit that performs the following processes: a process of generating a movement path for the robot when the robot moves the working unit relative to the workpiece and the working unit performs work on the workpiece; a process of generating coordinate information based on the generated movement path, the coordinate information including the amount of movement of the robot in a direction along the movement path and coordinate values ​​of a three-dimensional coordinate system that can express the coordinate values ​​of the workpiece in three dimensions; and a process of operating the robot based on the generated movement path and coordinate information, and the working unit performing work on the workpiece.

[0012] A robot system according to a third aspect of this disclosure includes a processing unit that performs the following processes: based on the generated movement path, generates coordinate information including the amount of movement of the robot in a direction along the movement path and coordinate values ​​in a three-dimensional coordinate system that can represent the coordinate values ​​of the workpiece in three dimensions; and operates the robot based on the generated movement path and the coordinate information to perform work on the workpiece using a working unit. This allows the robot's movement and trajectory to be understood using not only the movement path but also the amount of movement of the robot and the coordinate values ​​of the workpiece included in the coordinate information, making it possible to accurately understand the timing of controlling work on the workpiece. As a result, work on the workpiece can be performed accurately by the working unit.

[0013] According to the present disclosure, as described above, the position of the target can be indicated with high accuracy even on a curved surface of a workpiece with a large curvature or a complex curved surface.

[0014] FIG. 1 is a diagram showing an inspection system according to a first embodiment. FIG. 2 is a block diagram showing an inspection system according to the first embodiment. FIG. 3 is a flowchart for explaining control processing of the inspection system according to the first embodiment. FIG. 4 is a diagram for explaining generation of a movement path of a robot according to the first embodiment. FIG. 5 is a diagram for explaining generation of coordinate transformation information according to the first embodiment. FIG. 6 is a diagram (1) for explaining coordinate transformation information according to the first embodiment. FIG. 7 is a diagram (2) for explaining coordinate transformation information according to the first embodiment. FIG. 8 is a diagram for explaining inspection of a workpiece according to the first embodiment. FIG. 9 is a diagram for explaining inspection of a workpiece according to the first embodiment. FIG. 10 is a diagram for explaining inspection of a workpiece according to the first embodiment. FIG. 11 is a diagram for explaining inspection of a workpiece according to the first embodiment. FIG. 12 is a diagram for explaining inspection of a workpiece according to the first embodiment. FIG. 13 is a diagram for explaining coordinate transformation according to the first embodiment. FIG. 14 is a diagram for explaining displaying the position of an object on an actual workpiece by a robot according to the first embodiment. FIG. 15 is a diagram for explaining displaying the position of an object on a three-dimensional image of the workpiece according to the first embodiment. FIG. 16 is a diagram for explaining displaying multiple objects as a single object according to the first embodiment. FIG. 17 is a diagram showing an inspection system according to a second embodiment. FIG. 18 is a block diagram showing an inspection system according to the second embodiment. FIG. 19 is a flowchart for explaining control processing of the inspection system according to the second embodiment. FIG. 10 is a diagram for explaining the inspection of a workpiece according to the second embodiment. FIG. 11 is a diagram for explaining an inspection image according to the second embodiment. FIG. 12 is a diagram for explaining the display of inspection results according to the second embodiment. FIG. 13 is a diagram for explaining a case where an operation image for operating a robot according to the second embodiment is operated. FIG. 14 is a diagram for explaining the display of an object position on an actual workpiece by a robot according to the second embodiment. FIG. 15 is a diagram for explaining a case where an operation image for displaying an inspection image according to the second embodiment is operated. FIG. 16 is a diagram for explaining the display of an inspection image according to the second embodiment. FIG. 17 is a diagram for explaining the display of whether or not treatment has been completed according to the second embodiment. FIG. 18 is a diagram showing a robot system according to a third embodiment. FIG. 19 is a block diagram showing a robot system according to the third embodiment. FIG. 19 is a flowchart for explaining the control processing of the robot system according to the third embodiment.FIG. 10 is a diagram for explaining generation of a movement path of a robot according to a third embodiment. FIG. 11 is a diagram for explaining generation of coordinate information according to the third embodiment. FIG. 12 is a diagram for explaining coordinate information according to the third embodiment. FIG. 13 is a diagram for explaining control of the operation of a robot using coordinate information according to the third embodiment. FIG. 14 is a block diagram (1) showing an inspection system according to a modified example of the first embodiment. FIG. 15 is a block diagram (2) showing an inspection system according to a modified example of the first embodiment. FIG. 16 is a flowchart for explaining control processing of an inspection system according to a modified example of the first embodiment. FIG. 17 is a diagram showing a robot according to a modified example of the first embodiment.

[0015] Hereinafter, embodiments embodying the present disclosure will be described with reference to the drawings.

[0016] First Embodiment (Configuration of Inspection System) The overall configuration of an inspection system 100 according to a first embodiment will be described with reference to FIGS. 1 and 2. FIG.

[0017] 1, the inspection system 100 is an appearance inspection system that inspects the appearance of a workpiece 200. The workpiece 200 is, for example, a product or part related to an automobile, agricultural machinery, ceramics, or household electrical appliances. The workpiece 200 is not particularly limited.

[0018] The inspection system 100 includes a robot 10 , an inspection unit 20 , an instruction unit 30 , a robot controller 40 , an image processing device 50 , and a result display device 60 .

[0019] The robot 10 moves the inspection unit 20 relative to the workpiece 200. The robot 10 is a vertical articulated robot. The robot 10 includes a base unit 11 and an arm unit 12 connected to the base unit 11. The base unit 11 is fixed to an installation surface such as a floor, wall, or ceiling. The base unit 11 may also be attached to a movable carriage. The arm unit 12 has multiple joints. Each of the multiple joints has a servo motor as a drive source. The tip of the arm unit 12 holds the inspection unit 20 and the instruction unit 30. The robot 10 moves the inspection unit 20 and the instruction unit 30 held at the tip of the arm unit 12 relative to the fixed workpiece 200 by driving the multiple joints of the arm unit 12.

[0020] The inspection unit 20 is disposed on the robot 10 and inspects the workpiece 200. The inspection unit 20 is an imaging unit and captures an image of the workpiece 200. Specifically, the inspection unit 20 is a line-type camera that is moved along the surface of the workpiece 200 by the robot 10 and scans and captures an image of the surface of the workpiece 200.

[0021] The instruction unit 30 is disposed on the robot 10 and indicates the position of an object 201 (described later) acquired by inspection to the workpiece 200. The instruction unit 30 is a laser irradiation unit that irradiates laser light to indicate the position of the object 201 to the workpiece 200.

[0022] 1 and 2, the robot controller 40 controls the operation of the robot 10. The robot controller 40 includes a processing unit 41 and a storage unit 42. The processing unit 41 includes a processor and performs various processes related to the operation of the robot 10. The storage unit 42 includes a non-volatile memory and stores coordinate transformation information 71 or 72, which will be described later, and the like.

[0023] The image processing device 50 performs image processing on the images captured by the inspection unit 20. The image processing device 50 also controls the timing of imaging by the inspection unit 20. The image processing device 50 includes a processing unit 51 and a storage unit 52. The processing unit 51 includes a processor, and performs various processes related to the images captured by the inspection unit 20 and the timing of imaging by the inspection unit 20. The storage unit 52 includes a non-volatile memory, and stores the inspection image 21, which will be described later, and the like.

[0024] The result display device 60 displays the inspection results of the workpiece 200. The result display device 60 includes a processing unit 61, a storage unit 62, a display unit 63, and an operation unit 64. The processing unit 61 includes a processor and performs various processes related to the display of the inspection results of the workpiece 200. The storage unit 62 includes a non-volatile memory and stores coordinate transformation information 72, a three-dimensional image of the workpiece 200, and the like. The display unit 63 includes a monitor such as an LCD monitor and displays a screen showing the inspection results of the workpiece 200, and the like. The operation unit 64 includes input devices such as a mouse and a keyboard and accepts input operations from the user. Note that the display unit 63 and the operation unit 64 may be integrated. In other words, the display unit 63 and the operation unit 64 may be configured as an operation unit and display unit such as a touch panel.

[0025] (Control Processing of Inspection System) The control processing of the inspection system 100 will be described with reference to FIGS.

[0026] 3 and 4 , in step S1, the processing unit 41 of the robot controller 40 performs processing to generate a movement path 13 for the robot 10 when the robot 10 moves the inspection unit 20 relative to the workpiece 200 and inspects the workpiece 200 with the inspection unit 20. The movement path 13 is a path for operating the arm unit 12 of the robot 10, and multiple movement paths 13 are generated to inspect the workpiece 200.

[0027] For example, the processing unit 41 receives instructions from the user regarding the operation of the arm unit 12, and performs processing to generate the movement path 13 of the robot 10 based on the received instructions. Alternatively, for example, the processing unit 41 performs processing to automatically generate the movement path 13 of the robot 10 without relying on instructions from the user regarding the operation of the arm unit 12. Alternatively, the processing unit 41 performs processing to generate the movement path 13 that follows the surface of the workpiece 200, such as a curved surface. Alternatively, the processing unit 41 may perform processing to generate the movement path 13 of the robot 10 in real time.

[0028] In the first embodiment, as shown in FIGS. 3 and 5 to 7 , in step S2, the processing unit 41 of the robot controller 40 performs processing to generate coordinate transformation information 71 or 72 based on the generated movement path 13. The coordinate transformation information 71 or 72 is information that converts the coordinate values ​​of an inspection coordinate system of an inspection image 21 (described later) acquired by inspecting the workpiece 200 using the inspection unit 20 into coordinate values ​​of a three-dimensional coordinate system that can represent the coordinate values ​​of the workpiece 200 three-dimensionally. The inspection coordinate system is a Cartesian coordinate system with two mutually orthogonal axes, and the three-dimensional coordinate system is a Cartesian coordinate system with three mutually orthogonal axes. Details of the coordinate transformation using the coordinate transformation information 71 or 72 will be described later.

[0029] As shown in FIG. 5 , the processing unit 41 acquires coordinate values ​​in a three-dimensional coordinate system at first distance intervals D1 along the movement path 13 and generates coordinate transformation information 71 or 72. In this process, the processing unit 41 actually moves the inspection unit 20 along the movement path 13 relative to the workpiece 200 using the robot 10, and acquires coordinate values ​​in the three-dimensional coordinate system at first distance intervals D1. The first distance intervals D1 are the distance intervals between control points 14a. The processing unit 41 acquires coordinate values ​​in the three-dimensional coordinate system of the control points 14a at first distance intervals D1. When the inspection unit 20 is an imaging unit, the control points 14a are set at the focal position of the imaging unit 20. The focal position of the imaging unit 20 is set near the surface of the workpiece 200. The control points 14a are provided for acquiring coordinate values ​​in the three-dimensional coordinate system.

[0030] In the first embodiment, the first distance D1 is larger than the second distance D2 (described later) when inspecting the workpiece 200. For example, the first distance D1 is approximately 5 mm, and the second distance D2 is approximately 0.1 mm. The first distance D1 and the second distance D2 differ depending on the workpiece 200, and are therefore not particularly limited. The first distance D1 is set to a value that can ensure the accuracy required to indicate the position of the target 201 (described later). The second distance D2 is set to a value that can ensure the accuracy required to inspect the workpiece 200.

[0031] For convenience, only one movement path 13 is shown in FIG. 5, but the processing unit 41 acquires coordinate values ​​in a three-dimensional coordinate system for all movement paths 13 and performs processing to generate coordinate conversion information 71 or 72.

[0032] 6 and 7, the coordinate conversion information 71 or 72 is a coordinate conversion table that associates the movement amount of the robot 10 in a direction along the movement path 13 with coordinate values ​​in a three-dimensional coordinate system. In Figures 6 and 7, the path number represents the number of the movement path 13, the position number represents the number of the control point 14a, the movement amount represents the movement amount of the control point 14a of the robot 10 along the movement path 13, and the coordinate value represents the coordinate value of the control point 14a in the three-dimensional coordinate system. That is, in the coordinate conversion information 71 or 72, for each movement path 13, the movement amount of the robot 10 for each control point 14a is associated with the coordinate value of the control point 14a in the three-dimensional coordinate system.

[0033] The coordinate values ​​in the three-dimensional coordinate system of the coordinate transformation information 71 or 72 may consist solely of acquired coordinate values ​​in the three-dimensional coordinate system acquired at the first distance interval D1, or may consist of acquired coordinate values ​​in the three-dimensional coordinate system acquired at the first distance interval D1 and interpolated coordinate values ​​in the three-dimensional coordinate system obtained by interpolating the acquired values. That is, coordinate values ​​in the three-dimensional coordinate system may be acquired in 5 mm increments to generate the coordinate transformation information 71 or 72 in 5 mm increments, or coordinate values ​​in the three-dimensional coordinate system may be acquired in 10 mm increments to generate the coordinate transformation information 71 or 72 in 5 mm increments by interpolating the acquired coordinate values ​​in the three-dimensional coordinate system. Examples of interpolation include polynomial interpolation and linear interpolation. Examples of linear interpolation include linear interpolation and circular interpolation. The movement amount in the coordinate transformation information 71 or 72 may be calculated from the actual movement of the robot 10 or may be calculated from a movement command for the robot 10.

[0034] 6, in the coordinate transformation information 71, the three-dimensional coordinate system is a robot coordinate system for the robot 10. The robot coordinate system is a coordinate system based on the base unit 11. The coordinate transformation information 71 is a coordinate transformation table that associates the movement amount of the robot 10 with the coordinate values ​​in the robot coordinate system. In the coordinate transformation information 71, coordinate values ​​that indicate the position and orientation of the control point 14a in the robot coordinate system are used as the coordinate values.

[0035] 7, in the coordinate conversion information 72, the three-dimensional coordinate system is a workpiece coordinate system related to the workpiece 200. The workpiece coordinate system is a coordinate system based on the workpiece 200. The coordinate conversion information 72 is a coordinate conversion table that associates the movement amount of the robot 10 with the coordinate values ​​of the workpiece coordinate system. In the coordinate conversion information 72, coordinate values ​​that indicate the position of the control point 14a in the workpiece coordinate system are used as the coordinate values.

[0036] For example, the processing unit 41 acquires coordinate values ​​in the robot coordinate system to generate coordinate transformation information 71, and generates coordinate transformation information 72 based on the generated coordinate transformation information 71. Furthermore, for example, the processing unit 41 generates the coordinate transformation information 72 from the coordinate transformation information 71 by converting the coordinate values ​​in the robot coordinate system of the coordinate transformation information 71 into coordinate values ​​in the workpiece coordinate system using transformation information such as a transformation matrix.

[0037] Furthermore, the processing unit 41 performs a process of storing the coordinate transformation information 71 or 72 in the storage unit 42, and also performs a process of outputting the coordinate transformation information 72 to the processing unit 61 of the result display device 60. The processing unit 61 performs a process of storing the coordinate transformation information 72 in the storage unit 62.

[0038] 3 and 8 to 11, in step S3, the processing unit 41 of the robot controller 40 operates the robot 10 based on the movement path 13, and performs processing to inspect the workpiece 200 using the inspection unit 20. Then, the processing unit 51 of the image processing device 50 performs processing to acquire an inspection image 21 based on the output result of the inspection unit 20. The inspection image 21 is an image of the surface of the workpiece 200 captured by the inspection unit 20.

[0039] As shown in FIG. 9 , the processing unit 51 operates the inspection unit 20 to inspect the workpiece 200 at second intervals D2 along the movement path 13, thereby acquiring the inspection image 21. Specifically, the processing unit 51 operates the inspection unit 20 to capture an image of the workpiece 200 at second intervals D2, thereby scanning and capturing the image of the workpiece 200. More specifically, the processing unit 41 outputs a pulse signal to the processing unit 51 at second intervals D2. Based on the pulse signal from the processing unit 41, the processing unit 51 outputs a trigger signal to the inspection unit 20 at second intervals D2. Based on the trigger signal, the inspection unit 20 captures an image of the workpiece 200 at second intervals D2. Note that the second intervals D2 are the intervals between control points 14b. When the inspection unit 20 is an imaging unit, the control point 14b is set to the focal position of the imaging of the inspection unit 20. Furthermore, the focal position of the imaging of the inspection unit 20 is set near the surface of the workpiece 200. The control point 14b is provided for the inspection unit 20 to perform a process of capturing an image of the workpiece 200.

[0040] 9 shows only one movement path 13 for convenience, the processing unit 41 performs processing to inspect the workpiece 200 by the inspection unit 20 for all movement paths 13. The processing unit 51 also performs processing to acquire inspection images 21 for all movement paths 13.

[0041] 10 , the inspection ranges 22 of the inspection unit 20 are set so as to partially overlap with each other on adjacent movement paths 13. That is, the inspection range 22 of the inspection unit 20 for a certain movement path 13 partially overlaps with the inspection range 22 of the inspection unit 20 for a movement path 13 adjacent to the certain movement path 13. This makes it possible to prevent inspection omissions. Note that the inspection range 22 is the imaging range when scanning and imaging along the movement path 13.

[0042] 3 and 11 , in step S4, the processing unit 51 of the image processing device 50 performs processing to detect the object 201 of the workpiece 200 in the inspection image 21. The processing unit 51 performs predetermined image processing on the inspection image 21 to detect the object 201 in the inspection image 21. The object 201 is, for example, a defect such as a scratch, a foreign object, or a dent. The processing unit 51 performs processing to detect the object 201 in the inspection image 21 for all inspection images 21.

[0043] As shown in FIG. 11 , the inspection coordinate system of the inspection image 21 is a two-dimensional coordinate system in which the direction along the movement path 13 is the Y-axis direction and the direction perpendicular to the movement path 13 is the X-axis direction. The processing unit 51 performs processing to acquire the coordinate values ​​of the object 201 in the inspection coordinate system. That is, the processing unit 51 performs processing to acquire the coordinate values ​​of the X-axis and Y-axis of the inspection coordinate system of the object 201. The processing unit 51 also performs processing to acquire the coordinate values ​​of the object 201 in the inspection coordinate system for all inspection images 21 in which the object 201 is detected. The processing unit 51 also performs processing to store the inspection image 21, the number of the movement path 13 corresponding to the inspection image 21, and the coordinate values ​​of the object 201 in the inspection coordinate system in the memory unit 52. The processing unit 51 also performs processing to output the number of the movement path 13 corresponding to the inspection image 21 and the coordinate values ​​of the object 201 in the inspection coordinate system to the processing unit 41 of the robot controller 40 and the processing unit 61 of the result display device 60.

[0044] 3 and 12, in step S5, the processing unit 41 of the robot controller 40 performs processing to convert the coordinate values ​​of the inspection coordinate system of the object 201 into coordinate values ​​of the robot coordinate system based on the coordinate transformation information 71. Also in step S5, the processing unit 61 of the result display device 60 performs processing to convert the coordinate values ​​of the inspection coordinate system of the object 201 into coordinate values ​​of a three-dimensional coordinate system based on the coordinate transformation information 72. First, the processing of the processing unit 41 will be described.

[0045] 12 , the processing unit 41 performs processing to identify the number of the movement path 13 in the coordinate transformation information 71 based on the number of the movement path 13 corresponding to the inspection image 21. Then, the processing unit 41 performs processing to acquire the movement amount of the robot 10 in the coordinate transformation information 71 corresponding to the coordinate value of the inspection coordinate system in the Y-axis direction along the movement path 13 of the object 201 for the identified number of the movement path 13. At this time, the processing unit 41 performs processing to acquire the movement amount of the robot 10 that is closest to the coordinate value of the inspection coordinate system in the Y-axis direction of the object 201 as the corresponding movement amount of the robot 10. Then, the processing unit 41 performs processing to acquire the coordinate value of the robot coordinate system in the coordinate transformation information 71 that corresponds to the movement amount of the robot 10 in the acquired coordinate transformation information 71. As a result, the coordinate value of the robot coordinate system corresponding to the coordinate value of the inspection coordinate system in the Y-axis direction of the object 201 is acquired.

[0046] On the other hand, the coordinate values ​​in the robot coordinate system of the acquired coordinate transformation information 71 do not reflect the coordinate values ​​of the inspection coordinate system in the X-axis direction of the object 201, and therefore contain a deviation corresponding to that. For this reason, the processing unit 41 performs a process of correcting the coordinate values ​​in the robot coordinate system of the acquired coordinate transformation information 71 based on the coordinate values ​​of the inspection coordinate system in the X-axis direction perpendicular to the movement path 13 of the object 201. At this time, the processing unit 41 performs a process of correcting the coordinate values ​​in the robot coordinate system of the coordinate transformation information 71 by adding the coordinate values ​​of the inspection coordinate system in the X-axis direction of the object 201. As a result, the processing unit 41 performs a process of acquiring the coordinate values ​​of the robot coordinate system of the object 201. Furthermore, the processing unit 41 performs a process of converting the coordinate values ​​for all of the objects 201 to acquire the coordinate values ​​of the robot coordinate system.

[0047] For example, in the example shown in FIG. 12 , the number of the movement path 13 corresponding to the inspection image 21 is 2, the coordinate value of the object 201 in the X-axis direction is 5.5, and the coordinate value of the object 201 in the Y-axis direction is 15.2. In this case, the processing unit 41 performs processing to identify 2 as the number of the movement path 13. Then, the processing unit 41 performs processing to acquire 15 as the movement amount of the robot 10 that is closest to the coordinate value 15.2 of the object 201 in the Y-axis direction for the identified number 2. Then, the processing unit 41 performs processing to acquire (xr, yr, zr, or, ar, tr) as coordinate values ​​in the robot coordinate system corresponding to the movement amount 15 of the robot 10. Then, the processing unit 41 performs processing to acquire the coordinate value of the object 201 in the robot coordinate system by adding the coordinate value 5.5 of the object 201 in the X-axis direction to the coordinate values ​​(xr, yr, zr, or, ar, tr) in the robot coordinate system.

[0048] The processing of the processing unit 41 of the robot controller 40 has been described above. However, the processing of the processing unit 61 of the result display device 60 is similar except for the use of the coordinate transformation information 72. That is, the processing unit 61 performs processing to identify the number of the movement path 13 in the coordinate transformation information 72 based on the number of the movement path 13 corresponding to the inspection image 21. Then, the processing unit 61 performs processing to acquire the movement amount of the robot 10 in the coordinate transformation information 72 corresponding to the coordinate value of the inspection coordinate system in the Y-axis direction along the movement path 13 of the object 201 for the identified number of the movement path 13. At this time, the processing unit 61 performs processing to acquire the movement amount of the robot 10 closest to the coordinate value of the inspection coordinate system in the Y-axis direction of the object 201 as the corresponding movement amount of the robot 10. Then, the processing unit 61 performs processing to acquire the coordinate value of the workpiece coordinate system in the coordinate transformation information 72 corresponding to the movement amount of the robot 10 in the acquired coordinate transformation information 72. As a result, the coordinate value of the workpiece coordinate system corresponding to the coordinate value of the inspection coordinate system in the Y-axis direction of the object 201 is acquired.

[0049] On the other hand, the coordinate values ​​of the workpiece coordinate system in the acquired coordinate transformation information 72 do not reflect the coordinate values ​​of the inspection coordinate system in the X-axis direction of the object 201, and contain a corresponding deviation. For this reason, the processing unit 61 performs a process of correcting the coordinate values ​​of the workpiece coordinate system in the acquired coordinate transformation information 72 based on the coordinate values ​​of the inspection coordinate system in the X-axis direction perpendicular to the movement path 13 of the object 201. At this time, the processing unit 61 performs a process of correcting the coordinate values ​​of the workpiece coordinate system in the coordinate transformation information 72 by adding the coordinate values ​​of the inspection coordinate system in the X-axis direction of the object 201. As a result, the processing unit 61 performs a process of acquiring the coordinate values ​​of the workpiece coordinate system of the object 201. Furthermore, the processing unit 61 performs a process of converting the coordinate values ​​for all of the objects 201 to acquire the coordinate values ​​of the workpiece coordinate system.

[0050] 3 and 13 , in step S6, the processing unit 41 of the robot controller 40 performs processing to indicate the position of the target 201 on the actual workpiece 200 based on the coordinate values ​​of the converted target 201 in the three-dimensional coordinate system. Specifically, the processing unit 41 operates the robot 10 based on the coordinate values ​​of the target 201 converted into coordinate values ​​in the robot coordinate system, and performs processing to indicate the position of the target 201 on the actual workpiece 200 using the instruction unit 30. That is, the processing unit 41 operates the robot 10 to move the instruction unit 30 to a predetermined position where the position of the target 201 can be indicated. Then, with the instruction unit 30 positioned at the predetermined position, the processing unit 41 irradiates laser light from the instruction unit 30 to indicate the position of the target 201 on the actual workpiece 200.

[0051] 3 and 14 , in step S6, the processing unit 61 of the result display device 60 performs processing to display the position of the object 201 on the three-dimensional image of the workpiece 200 based on the coordinate values ​​of the converted object 201 in the three-dimensional coordinate system. Specifically, the processing unit 61 performs processing to display the position of the object 201 on the three-dimensional image of the workpiece 200 based on the coordinate values ​​of the object 201 converted into coordinate values ​​in the workpiece coordinate system. That is, the processing unit 61 performs processing to superimpose an image indicating the position of the object 201 on the three-dimensional image of the workpiece 200. Then, the processing unit 61 performs processing to display the three-dimensional image of the workpiece 200 on which the image indicating the position of the object 201 is superimposed, on the display unit 63. Note that the three-dimensional image of the workpiece 200 on which the image indicating the position of the object 201 is superimposed can be enlarged, reduced, or rotated based on a user operation using the operation unit 64.

[0052] 15 , in the first embodiment, when the coordinate values ​​of the multiple targets 201 converted into coordinate values ​​in the robot coordinate system are within a threshold range, the processing unit 41 treats the multiple targets 201 as a single target 201 and displays the positions of the targets 201 on the actual workpiece 200. The threshold value is a value for determining whether the coordinate values ​​can be considered to indicate the same position. That is, when the coordinate values ​​of the multiple targets 201 converted into coordinate values ​​in the robot coordinate system can be considered to indicate the same position, the processing unit 41 combines the multiple targets 201 that should actually be one into a single target 201 and displays the positions of the targets 201 on the actual workpiece 200. For example, when the coordinate values ​​of the target 201 detected from the inspection image 21 for a certain movement path 13 and the target 201 detected from the inspection image 21 for a movement path 13 adjacent to the certain movement path 13 are each converted into coordinate values ​​in the robot coordinate system within a threshold range, the processing unit 41 treats the multiple targets 201 as a single target 201 and displays the positions of the target 201 on the actual workpiece 200.

[0053] Effect of First Embodiment In the first embodiment, as described above, the following processes are performed: converting the coordinate values ​​of the inspection coordinate system of the object 201 into coordinate values ​​of a three-dimensional coordinate system based on the generated coordinate transformation information 71 or 72; and displaying the position of the object 201 on the actual workpiece 200 or a three-dimensional image of the workpiece 200 based on the converted coordinate values ​​of the three-dimensional coordinate system of the object 201. This makes it possible to display the position of the object 201 on the actual workpiece 200 or a three-dimensional image of the workpiece 200. Therefore, unlike when the position of the object 201 is displayed on a two-dimensional image of the workpiece 200, the position of the object 201 can be displayed with high accuracy even on a portion of the workpiece 200 that has a curved surface with a large curvature or a complex curved surface.

[0054] Furthermore, when the position of the target 201 is displayed on the actual workpiece 200 or a three-dimensional image of the workpiece 200, the coordinate values ​​of the inspection coordinate system of the target 201 must be converted into coordinate values ​​in a three-dimensional coordinate system. In the first embodiment, however, a process of converting the coordinate values ​​of the inspection coordinate system of the target 201 into coordinate values ​​in the three-dimensional coordinate system is performed based on the coordinate transformation information 71 or 72 generated based on the movement path 13 of the robot 10. This allows the coordinate values ​​of the inspection coordinate system of the target 201 to be converted into coordinate values ​​in the three-dimensional coordinate system with high accuracy, since the coordinate transformation information 71 or 72 generated based on the movement path 13 of the robot 10, which has a high correlation with the shape of the workpiece 200, is used. As a result, the position of the target 201 can be accurately displayed on the actual workpiece 200 or a three-dimensional image of the workpiece 200 using the coordinate values ​​of the three-dimensional coordinate system of the target 201 that have been accurately converted. This effect is particularly effective when inspecting a workpiece 200 with a complex shape.

[0055] Furthermore, by using the coordinate conversion information 71 or 72 created in advance, the process of converting the coordinate values ​​of the inspection coordinate system of the object 201 into coordinate values ​​of the three-dimensional coordinate system can be efficiently performed.

[0056] Furthermore, in the first embodiment, as described above, the processing unit 41 performs processing to acquire coordinate values ​​in a three-dimensional coordinate system at first distance intervals D1 along the movement path 13 and generate the coordinate conversion information 71 or 72. This makes it possible to acquire coordinates in the three-dimensional coordinate system uniformly along the movement path 13 and generate the coordinate conversion information 71 or 72, thereby suppressing unevenness in the accuracy of the coordinate conversion by the coordinate conversion information 71 or 72.

[0057] Furthermore, in the first embodiment, as described above, the processing unit 51 operates the inspection unit 20 to inspect the workpiece 200 at every second distance interval D2 along the movement path 13, and performs processing to acquire the inspection image 21, and the first distance interval D1 is greater than the second distance interval D2. This makes it possible to reduce the amount of information in the coordinate transformation information 71 or 72 while ensuring the accuracy required for coordinate transformation by the coordinate transformation information 71 or 72, by taking advantage of the fact that the inspection of the workpiece 200 needs to be performed precisely but the position of the target 201 does not need to be indicated in such detail.

[0058] Furthermore, in the first embodiment, as described above, the coordinate transformation information 71 or 72 is a coordinate transformation table that associates the amount of movement of the robot 10 in the direction along the movement path 13 with coordinate values ​​in a three-dimensional coordinate system, and in the process of converting the coordinate values ​​of the inspection coordinate system of the object 201 into coordinate values ​​in the three-dimensional coordinate system, the processing units 41 and 61 perform the following processes: a process of acquiring the amount of movement of the robot 10 in the coordinate transformation table that corresponds to the coordinate values ​​of the inspection coordinate system in the direction along the movement path 13 of the object 201; a process of acquiring the coordinate values ​​of the three-dimensional coordinate system in the coordinate transformation table that correspond to the amount of movement of the robot 10 in the acquired coordinate transformation table; and a process of correcting the coordinate values ​​of the three-dimensional coordinate system in the acquired coordinate transformation table based on the coordinate values ​​of the inspection coordinate system in the direction perpendicular to the movement path 13 of the object 201. In this way, by utilizing the correspondence between the amount of movement of the robot 10 in the direction along the movement path 13 and the coordinate values ​​of the inspection coordinate system in the direction along the movement path 13 of the object 201, the coordinate values ​​of the inspection coordinate system of the object 201 can be easily and accurately converted into coordinate values ​​in the three-dimensional coordinate system.

[0059] Furthermore, in the first embodiment, as described above, the inspection system 100 includes an indicator 30 disposed on the robot 10 and configured to indicate the position of the target 201 relative to the actual workpiece 200. The three-dimensional coordinate system is a robot coordinate system related to the robot 10. The processing unit 41 operates the robot 10 based on the coordinate values ​​of the target 201 converted into coordinate values ​​in the robot coordinate system, and performs processing to indicate the position of the target 201 on the actual workpiece 200 using the indicator 30. This allows the coordinate values ​​of the target 201 to be converted into coordinate values ​​in the robot coordinate system. Therefore, when indicating the position of the target 201 on the actual workpiece 200 using the robot 10, the position of the target 201 on the actual workpiece 200 can be easily grasped. As a result, the position of the target 201 can be easily indicated on the actual workpiece 200. Furthermore, because the indicator 30 is disposed on the robot 10 that performs the inspection, there is no need to provide a robot with an indicator 30 separately from the robot 10 that performs the inspection. As a result, the complexity of the structure can be reduced.

[0060] Furthermore, in the first embodiment, as described above, the three-dimensional coordinate system is a workpiece coordinate system related to the workpiece 200, and the processing unit 61 performs processing to indicate the position of the object 201 in the three-dimensional image of the workpiece 200 based on the coordinate values ​​of the object 201 converted into coordinate values ​​in the workpiece coordinate system. This allows the coordinate values ​​of the object 201 to be converted into coordinate values ​​in the workpiece coordinate system, making it possible to easily grasp the position of the object 201 in the three-dimensional image of the workpiece 200. As a result, the position of the object 201 can be easily indicated in the three-dimensional image of the workpiece 200.

[0061] Furthermore, in the first embodiment, as described above, when the coordinate values ​​of the multiple targets 201 converted into coordinate values ​​in the robot coordinate system are within a threshold range, the processing unit 41 treats the multiple targets 201 as a single target 201 and performs processing to display the position of the target 201 on the actual workpiece 200. This makes it possible to prevent a single target 201 from being erroneously displayed as multiple targets 201, even when the single target 201 is erroneously detected as multiple targets 201. This effect is particularly effective in a configuration in which the inspection ranges 22 of the inspection unit 20 partially overlap, as in the first embodiment, and therefore a single target 201 can be detected from different inspection images 21.

[0062] Second Embodiment A second embodiment of the present disclosure will be described with reference to Figures 16 to 26. In the second embodiment, in addition to the configuration of the first embodiment, an example will be described in which, when an input is made to a displayed object, processing related to an actual workpiece is performed for the input object.

[0063] (Configuration of Inspection System) The overall configuration of an inspection system 300 according to the second embodiment will be described with reference to FIGS. 16 and 17. FIG.

[0064] 16, the inspection system 300 is an appearance inspection system that inspects the appearance of a workpiece 400. The workpiece 400 is, for example, a product or part related to an automobile, agricultural machinery, ceramics, or household electrical appliances. The workpiece 400 is not particularly limited.

[0065] The inspection system 300 includes a robot 310, an inspection unit 320, an instruction unit 330, a robot controller 340, an image processing device 350, and a result display device 360. The inspection system 300 is configured to be able to perform the same processing as in the first embodiment.

[0066] The robot 310 moves the inspection unit 320 relative to the workpiece 400. The robot 310 is a vertical articulated robot. The robot 310 includes a base unit 311 and an arm unit 312 connected to the base unit 311. The base unit 311 is fixed to an installation surface such as a floor, wall, or ceiling. The base unit 311 may also be attached to a movable cart. The arm unit 312 has multiple joints. Each of the multiple joints has a servo motor as a drive source. The tip of the arm unit 312 holds the inspection unit 320 and the instruction unit 330. The robot 310 moves the inspection unit 320 and the instruction unit 330 held at the tip of the arm unit 312 relative to the fixed workpiece 400 by driving the multiple joints of the arm unit 312.

[0067] The inspection unit 320 is disposed on the robot 310 and inspects the workpiece 400. The inspection unit 320 is an imaging unit and captures an image of the workpiece 400. Specifically, the inspection unit 320 is a line-type camera that is moved along the surface of the workpiece 400 by the robot 310 and scans and captures an image of the surface of the workpiece 400.

[0068] The instruction unit 330 is disposed on the robot 310, and indicates the position of an object 401 (described later) acquired by inspection to the workpiece 400. The instruction unit 330 is a laser irradiation unit, and indicates the position of the object 401 to the workpiece 400 by irradiating it with laser light.

[0069] 16 and 17 , the robot controller 340 controls the operation of the robot 310. The robot controller 340 includes a processing unit 341 and a storage unit 342. The processing unit 341 includes a processor and performs various processes related to the operation of the robot 310. The storage unit 342 includes a non-volatile memory and stores various information such as programs for operating the robot 310.

[0070] The image processing device 350 performs image processing on the image captured by the inspection unit 320. The image processing device 350 also controls the timing of image capture by the inspection unit 320. The image processing device 350 includes a processing unit 351 and a storage unit 352. The processing unit 351 includes a processor, and performs various processes related to the image captured by the inspection unit 320 and the timing of image capture by the inspection unit 320. The storage unit 352 includes a non-volatile memory, and stores various information such as programs for performing image processing.

[0071] The result display device 360 ​​displays the inspection results of the workpiece 400. The result display device 360 ​​includes a processing unit 361, a storage unit 362, a display unit 363, and an operation unit 364. The processing unit 361 includes a processor and performs various processes related to the display of the inspection results of the workpiece 400. The storage unit 362 includes a non-volatile memory and stores various information such as a program for displaying the inspection results of the workpiece 400 and a 3D image of the workpiece 400. The display unit 363 includes a monitor such as an LCD monitor and displays a screen showing the inspection results of the workpiece 400. The operation unit 364 includes input devices such as a mouse and a keyboard and accepts user input operations. The display unit 363 and the operation unit 364 may be integrated. In other words, the display unit 363 and the operation unit 364 may be configured as an operation unit and display unit such as a touch panel.

[0072] (Control Processing of Inspection System) The control processing of the inspection system 300 will be described with reference to FIGS.

[0073] 18 and 19 , in step S301, the processing unit 341 of the robot controller 340 operates the robot 310 based on the movement path 313, and performs processing to inspect the workpiece 400 using the inspection unit 320. The movement path 313 is a path for operating the arm unit 312 of the robot 310, and multiple movement paths 313 are generated in order to inspect the workpiece 400. Furthermore, the movement paths 313 are generated in advance before the workpiece 400 is inspected.

[0074] For example, the processing unit 341 receives instructions from the user regarding the operation of the arm unit 312, and performs processing to generate a movement path 313 for the robot 310 based on the received instructions. Alternatively, for example, the processing unit 341 performs processing to automatically generate the movement path 313 for the robot 310, regardless of instructions from the user regarding the operation of the arm unit 312. Alternatively, the processing unit 341 performs processing to generate the movement path 313 that follows the surface of the workpiece 400, such as a curved surface. Alternatively, the processing unit 341 may perform processing to generate the movement path 13 for the robot 10 in real time.

[0075] 18 to 20 , the processing unit 351 of the image processing device 350 performs processing to acquire an inspection image 321 based on the output result of the inspection unit 320. The inspection image 321 is an image of the surface of the workpiece 400 captured by the inspection unit 320.

[0076] The processing unit 351 operates the inspection unit 320 to inspect the workpiece 400 at regular intervals along the movement path 313, thereby acquiring the inspection image 321. Specifically, the processing unit 351 operates the inspection unit 320 to capture an image of the workpiece 400 at regular intervals, thereby scanning and capturing the image of the workpiece 400. More specifically, the processing unit 341 outputs a pulse signal to the processing unit 351 at regular intervals. Based on the pulse signal from the processing unit 341, the processing unit 351 outputs a trigger signal to the inspection unit 320 at regular intervals. Based on the trigger signal, the inspection unit 320 captures an image of the workpiece 400 at regular intervals. Note that the regular intervals are the intervals between the control points of the robot 310. If the inspection unit 320 is an imaging unit, the control point of the robot 310 is set to the focal position of the imaging of the inspection unit 320. Furthermore, the focal position of the imaging of the inspection unit 320 is set near the surface of the workpiece 400. The control point of the robot 310 is provided so that the inspection unit 320 can perform the process of capturing an image of the workpiece 400 .

[0077] The processing unit 341 performs a process of inspecting the workpiece 400 by the inspection unit 320 for all of the movement paths 313. The processing unit 351 also performs a process of acquiring the inspection image 321 for all of the movement paths 313.

[0078] 18 and 20 , in step S302, the processing unit 351 of the image processing device 350 performs processing to detect the object 401 of the workpiece 400 in the inspection image 321. The processing unit 351 performs predetermined image processing on the inspection image 321 to detect the object 401 in the inspection image 321. The object 401 is, for example, a defect such as a scratch, a foreign object, or a dent. The processing unit 351 performs processing to detect the object 401 in the inspection image 321 for all of the inspection images 321. In addition, the processing unit 351 performs processing to output the inspection results of the workpiece 400 to the processing unit 341 of the robot controller 340 and the processing unit 361 of the result display device 360.

[0079] As shown in FIGS. 18 and 21 , in step S303, the processing unit 361 of the result display device 360 ​​performs a process of displaying the position of the target 401 on the three-dimensional image of the workpiece 400. Specifically, the processing unit 361 performs a predetermined coordinate conversion process of converting the coordinate values ​​of the target 401 in the inspection image 321 into the coordinate values ​​of the target 401 in the three-dimensional image of the workpiece 400, thereby displaying the position of the target 401 on the three-dimensional image of the workpiece 400. At this time, the processing unit 361 performs a process of superimposing an image indicating the position of the target 401 on the three-dimensional image of the workpiece 400. Then, the processing unit 361 performs a process of displaying the three-dimensional image of the workpiece 400 on which the image indicating the position of the target 401 is superimposed, on the display unit 363. Note that the three-dimensional image of the workpiece 400 on which the image indicating the position of the target 401 is superimposed can be enlarged, reduced, or rotated based on a user operation using the operation unit 364. Furthermore, the image showing the position of the object 401 may be displayed in a different color or shape depending on the type of object 401, such as a scratch, foreign object, or dent.

[0080] The processing unit 361 also performs processing to display the inspection results of the workpiece 400 in a list format. The inspection results of the workpiece 400 represent the results of processing to detect the objects 401 in the inspection images 321 for all of the inspection images 321. The inspection results of the workpiece 400 include the numbers of the detected objects 401 and the types of the detected objects 401.

[0081] In the second embodiment, the processing unit 361 performs processing to display the operation image 371 or 372 in list format along with the inspection results of the workpiece 400. The operation image 371 is an image for performing processing to operate the robot 310 so that the position of the target 401 (described later) is displayed on the actual workpiece 400. The operation image 372 is an image for performing processing to display the inspection image 321 of the target 401 (described later). The operation image 371 or 372 is displayed in list format along with the inspection results of the workpiece 400 so that the correspondence between the number and type of the detected target 401 can be identified. The processing unit 361 performs processing to display the inspection results of the workpiece 400 in list format and the operation image 371 or 372 on the display unit 363. The processing unit 361 also performs processing to display the 3D image of the workpiece 400, the inspection results of the workpiece 400 in list format, and the operation image 371 or 372 in the same frame. The operation image 371 is an example of a first operation image. The operation image 372 is an example of a second operation image.

[0082] 18 and 22 to 25, in the second embodiment, when an input is made to the target 401, the processing unit 361 performs processing on the actual workpiece 400 in relation to the input target 401 in steps S304 and S305. Specifically, when the operation image 371 or 372 corresponding to the target 401 is operated, the processing unit 361 performs processing on the actual workpiece 400 in relation to the target 401 corresponding to the operated operation image 371 or 372. The user operates the desired operation image 371 or 372 using the operation unit 364.

[0083] When the operation image 371 is operated, the processing unit 361 performs processing of step S304. As shown in FIGS. 22 and 23 , in step S304, the processing unit 361 performs processing to operate the robot 310 so that the position of the target 401 corresponding to the operated operation image 371 is shown on the actual workpiece 400. Specifically, the processing unit 361 performs processing to output identification information for identifying the target 401 corresponding to the operated operation image 371 to the processing unit 341 of the robot controller 340. The identification information is, for example, the number of the target 401. The processing unit 341 identifies the target 401 based on the identification information from the processing unit 361, and performs processing to operate the robot 310 so that the position of the identified target 401 is shown on the actual workpiece 400 as the position of the target 401 corresponding to the operated operation image 371.

[0084] More specifically, the processing unit 341 operates the robot 310 and performs processing to indicate the position of the target 401 on the actual workpiece 400 using the instruction unit 330. That is, the processing unit 341 operates the robot 310 and performs processing to move the instruction unit 330 to a predetermined position where the position of the target 401 can be indicated. Then, with the instruction unit 330 placed at the predetermined position, the processing unit 341 irradiates laser light from the instruction unit 330 and performs processing to indicate the position of the target 401 on the actual workpiece 400.

[0085] Note that the position of the object 401 corresponding to the operation image 371 may be displayed on the actual workpiece 400 only for one operation image 371, or may be displayed collectively for multiple operation images 371. For example, the processing unit 361 may perform processing to operate the robot 310 each time an operation image 371 is operated so that the position of the object 401 corresponding to the operated operation image 371 is displayed on the actual workpiece 400. Furthermore, for example, when multiple operation images 371 are operated collectively, the processing unit 361 may perform processing to operate the robot 310 so that the positions of multiple objects 401 corresponding to the operated operation images 371 are successively displayed on the actual workpiece 400. In this case, it is possible to easily confirm the positions of multiple objects 401 on the actual workpiece 400. Furthermore, when multiple operation images 371 of the same type are operated collectively, it is possible to easily confirm the positions of multiple objects 401 of the same type on the actual workpiece 400.

[0086] Furthermore, in the second embodiment, when the operation image 371 is operated, the processing unit 361 performs processing to highlight the position of the target 401 corresponding to the operated operation image 371 in the three-dimensional image of the workpiece 400, and to operate the robot 310 so that the position of the target 401 corresponding to the operated operation image 371 is shown on the actual workpiece 400. For example, the processing unit 361 performs processing to highlight the position of the target 401 in the three-dimensional image of the workpiece 400 by a highlighting method such as blinking, enlarging, or changing the color.

[0087] When the operation image 372 is operated, the processing unit 361 performs processing of step S305. As shown in FIGS. 24 and 25 , in step S305, the processing unit 361 performs processing to display the inspection image 321 of the object 401 corresponding to the operated operation image 372. Specifically, the processing unit 361 performs processing to display the inspection image 321 in a frame different from the frame in which the 3D image of the workpiece 400, the list-format inspection results of the workpiece 400, and the operation image 371 or 372 are displayed. Note that the processing unit 361 may also perform processing to display the inspection image 321 in the same frame as the frame in which the 3D image of the workpiece 400, the list-format inspection results of the workpiece 400, and the operation image 371 or 372 are displayed.

[0088] Furthermore, in the second embodiment, when an operation image 372 is operated, the processing unit 361 performs processing to highlight the position of the target 401 corresponding to the operated operation image 372 in the three-dimensional image of the workpiece 400, and to display the inspection image 321 of the target 401 corresponding to the operated operation image 372. For example, the processing unit 361 performs processing to highlight the position of the target 401 in the three-dimensional image of the workpiece 400 by a highlighting method such as blinking, enlarging, or changing the color.

[0089] Furthermore, when the operation image 372 is operated, the processing unit 361 moves the three-dimensional image of the workpiece 400 so that the position of the object 401 corresponding to the operated operation image 372 is easily visible, and performs processing to display the inspection image 321 of the object 401 corresponding to the operated operation image 372. For example, the processing unit 361 performs processing to make the position of the object 401 easily visible by rotating the three-dimensional image of the workpiece 400 so that the position of the object 401 corresponding to the operated operation image 372 faces forward. At this time, the position of the three-dimensional image of the workpiece 400 is not fixed to a position facing forward, and the user can freely rotate the three-dimensional image of the workpiece 400 using the operation unit 364.

[0090] 26 , in the second embodiment, the processing unit 361 performs processing to display an identification indicator 373 indicating whether or not treatment of the target 401 has been completed. Treatment of the target 401 means, for example, performing repair work on the target 401 as a defect. The identification indicator 373 is a check mark placed in a check box 373 a. When treatment of the target 401 has been completed, the user uses the operation unit 364 to check the check box 373 a, thereby displaying the identification indicator 373.

[0091] The processing unit 361 also performs processing to display the identification indicator 373 and check box 373a in list format together with the inspection results of the workpiece 400 and the operation image 371 or 372. The identification indicator 373 and check box 373a are displayed in list format together with the inspection results of the workpiece 400 so that the correspondence between the identification indicator 373 and the type of the detected object 401 can be identified. The processing unit 361 also performs processing to display the inspection results of the workpiece 400 in list format, the operation image 371 or 372, the identification indicator 373, and the check box 373a on the display unit 363. The processing unit 361 also performs processing to display the three-dimensional image of the workpiece 400, the inspection results of the workpiece 400 in list format, the operation image 371 or 372, the identification indicator 373, and the check box 373a in the same frame.

[0092] Furthermore, in the second embodiment, in addition to the process of displaying the identification indicator 373, the processing unit 361 performs a process of displaying the position of the target 401 for which treatment has been completed in a color different from the position of the target 401 for which treatment has not been completed in a three-dimensional image. For example, if the color of the target 401 for which treatment has not been completed is red, the position of the target 401 for which treatment has been completed is displayed in green. In other words, when the identification indicator 373 is displayed in the check box 373a, the processing unit 361 performs a process of displaying the position of the target 401 corresponding to the check box 373a in which the identification indicator 373 is displayed in a color different from that before the identification indicator 373 was displayed.

[0093] (Effects of Second Embodiment) As described above, the second embodiment performs the following processes: displaying the position of the detected target 401 on a three-dimensional image of the workpiece 400; and, when input is made to the displayed target 401, performing processing on the actual workpiece 400 for the input target 401. This not only enables the user to check the position of the target 401 on the workpiece 400 using the three-dimensional image of the workpiece 400, but also enables the user to perform processing on the actual workpiece 400 for the input target 401 by making input to the target 401. As a result, the user's convenience regarding the image of the workpiece 400 can be improved compared to when the position of the target 401 is simply displayed on the image of the workpiece 400.

[0094] Furthermore, by displaying the position of the object 401 on a three-dimensional image of the workpiece 400, the position of the object 401 can be displayed more clearly than when the position of the object 401 is displayed on a two-dimensional image of the workpiece 400, since the image is three-dimensional.

[0095] Furthermore, in the second embodiment, as described above, the processing unit 361 displays the operation image 371 or 372 corresponding to the target 401, and when the displayed operation image 371 or 372 is operated, performs processing related to the actual workpiece 400 for the target 401 corresponding to the operated operation image 371 or 372. This allows the user to easily input to the target 401 using the operation image 371 or 372, thereby easily improving the user's convenience regarding the image of the workpiece 400.

[0096] Furthermore, in the second embodiment, as described above, when the operation image 371 or 372 is operated, the processing unit 361 performs either a process of operating the robot 310 so that the position of the target 401 corresponding to the operated operation image 371 or 372 is shown on the actual workpiece 400, or a process of displaying the inspection image 321 of the target 401 corresponding to the operated operation image 371 or 372. As a result, when operating the robot 310 so that the position of the target 401 corresponding to the operated operation image 371 or 372 is shown on the actual workpiece 400, the user can operate the robot 310 and show the position of the target 401 on the actual workpiece 400 simply by operating the operation image 371 or 372 while checking the 3D image of the workpiece 400. As a result, convenience for a user who wishes to check the target 401 on the actual workpiece 400 can be effectively improved. Furthermore, when displaying the inspection image 321 of the object 401 corresponding to the operated operation image 371 or 372, the user can display the inspection image 321 acquired from the actual workpiece 400 simply by operating the operation image 371 or 372 while checking the three-dimensional image of the workpiece 400. As a result, the convenience of the user who wishes to check the inspection image 321 can be effectively improved.

[0097] Furthermore, in the second embodiment, as described above, when the operation image 371 or 372 is operated, the processing unit 361 performs processing to highlight the position of the target 401 corresponding to the operated operation image 371 or 372 in the three-dimensional image of the workpiece 400 and to operate the robot 310 so that the position of the target 401 corresponding to the operated operation image 371 or 372 is shown on the actual workpiece 400. As a result, when operating the robot 310 so that the position of the target 401 is shown on the actual workpiece 400, the user can easily confirm the position of the target 401 corresponding to the operated operation image 371 or 372 in the three-dimensional image of the workpiece 400. As a result, if the user mistakenly operates the operation image 371 or 372 corresponding to the target 401 that the user does not want to confirm, the user can easily recognize that he or she has made the mistaken operation.

[0098] Furthermore, in the second embodiment, as described above, when the operation image 371 or 372 is operated, the processing unit 361 performs processing to highlight the position of the target 401 corresponding to the operated operation image 371 or 372 in the 3D image of the workpiece 400 and display the inspection image 321 of the target 401 corresponding to the operated operation image 371 or 372. As a result, when the inspection image 321 of the target 401 corresponding to the operated operation image 371 or 372 is displayed, the user can easily confirm the position of the target 401 corresponding to the operated operation image 371 or 372 in the 3D image of the workpiece 400. As a result, when the user mistakenly operates the operation image 371 or 372 corresponding to the target 401 that the user does not want to confirm, the user can easily recognize that he or she made the mistaken operation.

[0099] Furthermore, in the second embodiment, as described above, operation image 371 or 372 includes operation image 371 for performing a process of operating robot 310 so that the position of target 401 matches the actual workpiece 400, and operation image 372 for performing a process of displaying inspection image 321 of target 401. When operation image 371 is operated, processing unit 361 performs a process of operating robot 310 so that the position of target 401 corresponding to the operated operation image 371 matches the actual workpiece 400, and when operation image 372 is operated, processing unit 361 performs a process of displaying inspection image 321 of target 401 corresponding to the operated operation image 372. As a result, since operation image 371 and operation image 372 are provided separately, the user can reliably perform both the process of operating robot 310 by operating operation image 371 and the process of displaying inspection image 321 by operating operation image 372.

[0100] Furthermore, in the second embodiment, as described above, the processing unit 361 performs processing to display the operation image 371 or 372 in a list format together with the inspection result of the workpiece 400. This allows the operation image 371 or 372 to be displayed in a list format together with the inspection result of the workpiece 400, so that the user can easily identify and operate the operation image 371 or 372 of the target 401 that the user wishes to check.

[0101] Furthermore, in the second embodiment, as described above, the processing unit 361 performs processing to display the identification display 373 that indicates whether or not the treatment for the target 401 has been completed. This allows the user to reliably confirm whether or not the treatment for the target 401 has been completed, using the identification display 373. Furthermore, by displaying whether or not the treatment for the target 401 has been completed, traceability can be ensured.

[0102] Furthermore, in the second embodiment, as described above, the processing unit 361 performs processing to display the positions of the targets 401 for which treatment has been completed in a color different from that of the positions of the targets 401 for which treatment has not been completed in a three-dimensional image of the workpiece 400, in addition to processing to display the identification indicator 373. This allows the user to more reliably confirm whether or not the treatment of the target 401 has been completed, based on both the identification indicator 373 and the color of the target 401.

[0103] Third Embodiment A third embodiment of the present disclosure will be described with reference to FIGS.

[0104] (Configuration of Inspection System) The overall configuration of a robot system 500 according to the third embodiment will be described with reference to FIGS. 27 and 28. FIG.

[0105] 27, the robot system 500 is a system that performs an operation on a workpiece 600. The operation is, for example, painting the workpiece 600.

[0106] The robot system 500 includes a robot 510 , a working unit 520 , and a robot controller 540 .

[0107] The robot 510 moves the working unit 520 relative to the workpiece 600. The robot 510 is a vertical articulated robot. The robot 510 includes a base unit 511 and an arm unit 512 connected to the base unit 511. The base unit 511 is fixed to an installation surface such as a floor, wall, or ceiling. The base unit 511 may be attached to a movable cart. The arm unit 512 has multiple joints. Each of the multiple joints has a servo motor as a drive source. The tip of the arm unit 512 holds the working unit 520. The robot 510 moves the working unit 520 held at the tip of the arm unit 512 relative to the fixed workpiece 600 by driving the multiple joints of the arm unit 512.

[0108] The working unit 520 is disposed on the robot 510 and performs work on the workpiece 600. For example, the working unit 520 is a painting unit that performs painting work on the workpiece 600. In this case, for example, the working unit 520 is a painting unit that sprays paint by inkjet, and is moved along the surface of the workpiece 600 by the robot 510 to apply paint to the surface of the workpiece 600, thereby performing painting work on the workpiece 600.

[0109] 1 and 2, the robot controller 540 controls the operation of the robot 510. The robot controller 540 includes a processing unit 541 and a storage unit 542. The processing unit 541 includes a processor and performs various processes related to the operation of the robot 510. The storage unit 542 includes a non-volatile memory and stores coordinate information 571 (described later) and the like.

[0110] The working unit control device 550 controls the timing of work by the working unit 520. The working unit control device 550 includes a processing unit 551 and a storage unit 552. The processing unit 551 includes a processor, and performs various processes related to the timing of work by the working unit 520. The storage unit 552 includes non-volatile memory, and stores various information such as programs for controlling the timing of work by the working unit 520.

[0111] (Control Processing of Robot System) The control processing of the robot system 500 will be described with reference to FIGS. 29 to 33. FIG.

[0112] 29 and 30 , in step S501, processing is performed to generate a movement path 513 for the robot 510, similar to the processing to generate the movement path 13 in the first embodiment described above. That is, the processing unit 541 of the robot controller 540 performs processing to generate a movement path 513 for the robot 510 when the robot 510 moves the working unit 520 relative to the workpiece 600 and the working unit 520 performs work on the workpiece 600. The movement path 513 is a path for operating the arm unit 512 of the robot 510, and multiple movement paths 513 are generated to perform work on the workpiece 600.

[0113] For example, the processing unit 541 receives instructions from the user regarding the operation of the arm unit 512, and performs processing to generate a movement path 513 for the robot 510 based on the received instructions. Furthermore, for example, the processing unit 541 performs processing to automatically generate the movement path 513 for the robot 510, regardless of instructions from the user regarding the operation of the arm unit 512. Furthermore, the processing unit 541 performs processing to generate the movement path 513 that follows the surface of the workpiece 600, such as a curved surface. Furthermore, the processing unit 541 may perform processing to generate the movement path 513 for the robot 510 in real time.

[0114] 29 and 32 , in the third embodiment, in step S502, a process for generating coordinate information 571 is performed similarly to the process for generating the coordinate transformation information 71 in the first embodiment. That is, the processing unit 541 of the robot controller 540 performs a process for generating coordinate information 571 based on the generated movement path 513. The coordinate information 571 is information including the amount of movement of the robot 510 in a direction along the movement path 513 and coordinate values ​​of a three-dimensional coordinate system that can represent the coordinate values ​​of the workpiece 600 three-dimensionally. The three-dimensional coordinate system is an orthogonal coordinate system with three axes that are orthogonal to each other.

[0115] 31 , the processing unit 541 performs processing to acquire coordinate values ​​in a three-dimensional coordinate system at first distance intervals D501 along the movement path 513 and generate coordinate information 571. At this time, the processing unit 541 causes the robot 510 to actually move the working unit 520 along the movement path 513 relative to the workpiece 600, and performs processing to acquire coordinate values ​​in the three-dimensional coordinate system at first distance intervals D501. Furthermore, the first distance intervals D501 are distance intervals between control points 514a set for the working unit 520. The processing unit 541 performs processing to acquire coordinate values ​​in the three-dimensional coordinate system of the control points 514a at first distance intervals D501. The control points 514a are provided for performing processing to acquire coordinate values ​​in the three-dimensional coordinate system.

[0116] For convenience, only one movement path 513 is shown in FIG. 31, but the processing unit 541 acquires coordinate values ​​in a three-dimensional coordinate system for all movement paths 513 and performs processing to generate coordinate information 571.

[0117] 32, the coordinate information 571 is a coordinate table that associates the movement amount of the robot 510 in a direction along the movement path 513 with coordinate values ​​in a three-dimensional coordinate system. In Fig. 32, the path number represents the number of the movement path 513, the position number represents the number of the control point 514a, the movement amount represents the movement amount of the control point 514a of the robot 510 along the movement path 513, and the coordinate value represents the coordinate value of the control point 514a in the three-dimensional coordinate system. In other words, in the coordinate information 571, the movement amount of the robot 510 for each control point 514a is associated with the coordinate value of the control point 514a in the three-dimensional coordinate system for each movement path 513.

[0118] The coordinate values ​​in the three-dimensional coordinate system of the coordinate information 571 may consist solely of acquired coordinate values ​​in the three-dimensional coordinate system acquired at each first distance interval D501, or may consist of acquired coordinate values ​​in the three-dimensional coordinate system acquired at each first distance interval D501 and interpolated coordinate values ​​in the three-dimensional coordinate system obtained by interpolating the acquired values. That is, coordinate values ​​in the three-dimensional coordinate system may be acquired in 5 mm increments to generate the coordinate information 571 in 5 mm increments, or coordinate values ​​in the three-dimensional coordinate system may be acquired in 10 mm increments to generate the coordinate information 571 in 5 mm increments by interpolating the acquired coordinate values ​​in the three-dimensional coordinate system. Examples of interpolation include polynomial interpolation and linear interpolation. Examples of linear interpolation include linear interpolation and circular interpolation. The movement amount in the coordinate information 571 may be calculated from the actual movement of the robot 510 or may be calculated from a movement command for the robot 510.

[0119] In the coordinate information 571, the three-dimensional coordinate system is a robot coordinate system related to the robot 510. The robot coordinate system is a coordinate system based on the base unit 511. The coordinate information 571 is a coordinate conversion table that associates the movement amount of the robot 510 with the coordinate values ​​of the robot coordinate system. In the coordinate information 571, coordinate values ​​that indicate the position and orientation of the control point 514a in the robot coordinate system are used as the coordinate values. In addition, the processing unit 541 performs processing to store the coordinate information 571 in the storage unit 542.

[0120] 29 and 33 , in step S503, processing units 541 and 551 operate robot 510 based on the generated movement path 513 and coordinate information 571, thereby performing processing to have working unit 520 perform work on workpiece 600. At this time, processing units 541 and 551 perform processing to control, based on coordinate information 571, the work start position at which working unit 520 starts work on workpiece 600. If working unit 520 is a painting unit, the work start position is the application start position at which application of paint begins. Furthermore, processing unit 541 controls the timing at which robot 510 dispenses paint based on movement path 513.

[0121] 33 , the processing unit 541 performs processing to identify the number of the movement path 513 in the coordinate information 571 based on the number of the movement path 513 along which the work is to be performed. The processing unit 541 also performs processing to acquire coordinate values ​​in the robot coordinate system of the robot 10 that correspond to the work start position on the movement path 513 along which the work is to be performed. The processing unit 541 then performs processing to acquire, as the work start position, from the coordinate information 571, the movement amount of the robot 10 that corresponds to the number of the identified movement path 513 and the acquired coordinate values ​​in the robot coordinate system of the robot 10. In this way, the processing unit 541 performs processing to acquire the movement amount of the robot 10 that will be the work start position from the coordinate information 571. In this way, the processing unit 541 performs processing to confirm an appropriate work start position.

[0122] The processing units 541 and 551 then perform processing to control the work start position based on the movement amount of the robot 10, which is the work start position obtained from the coordinate information 571. Specifically, the processing unit 541 is configured to be able to output a number of pulse signals corresponding to the movement amount of the robot 10. Therefore, the processing unit 541 performs processing to set the number of pulse signals corresponding to the movement amount of the robot 10 obtained from the coordinate information 571 as the work start position. The processing unit 541 then performs processing to output the set number of pulse signals to the processing unit 551. The processing unit 551 then performs processing to output a trigger signal to the working unit 520 at the position of the movement amount of the robot 10 obtained as the work start position, based on the pulse signals from the processing unit 541. The working unit 520 performs work on the workpiece 600 based on the trigger signal. If the working unit 520 is a painting unit, the working unit 520 applies paint to the workpiece 600.

[0123] Effect of Third Embodiment As described above, the third embodiment includes processing units 541 and 551 that perform the following operations: generate, based on the generated movement path 513, coordinate information 571 including the movement amount of the robot 510 in a direction along the movement path 513 and coordinate values ​​in a three-dimensional coordinate system that can represent the coordinate values ​​of the workpiece 600 in three dimensions; and operate the robot 510 and have the working unit 520 perform work on the workpiece 600 based on the generated movement path 513 and the coordinate information 571. This allows the movement and trajectory of the robot 510 to be understood using not only the movement path 513 but also the movement amount of the robot 510 and the coordinate values ​​of the workpiece 600 included in the coordinate information 571, thereby enabling accurate understanding of the timing to control the work on the workpiece. As a result, the working unit 520 can accurately perform work on the workpiece 600. Furthermore, if the working unit 520 is a painting unit, the occurrence of a painting edge position can be suppressed.

[0124] Furthermore, in the third embodiment, as described above, the processing units 541 and 551 perform processing to control the work start position based on the movement amount of the robot 510, which is the work start position where the working unit 520 starts work on the workpiece 600, acquired from the coordinate information 571. Here, if the position where the working unit 520 starts work on the workpiece 600 is important, the robot 510 may be stopped temporarily so that the working unit 520 can start work on the workpiece 600 at an accurate position where positional deviation is suppressed. However, when the robot 510 is stopped temporarily, it is difficult for the working unit 520 to efficiently perform work on the workpiece 600. Therefore, by configuring as described above, the work start position can be controlled based on the movement amount of the robot 510. Therefore, even if the robot 10 is operating continuously, the working unit 520 can start work on the workpiece 600 at an accurate position where positional deviation is suppressed. As a result, the working unit 520 can perform work on the workpiece 600 more efficiently than when the robot 510 is stopped temporarily. Furthermore, when painting using an inkjet, the position at which paint application starts is important because painting failures can occur due to deviations in the position at which paint application starts. Therefore, when the working unit 520 is a painting unit, it is particularly effective to be able to start work on the workpiece 600 by the working unit 520 at an accurate position where deviations are suppressed, even when the robot 10 is operating continuously.

[0125] (Modifications) The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the description of the above-mentioned embodiments, and further includes all modifications (modifications) within the meaning and scope equivalent to the claims.

[0126] For example, in the above first and second embodiments, an example was shown in which the inspection system is an appearance inspection system that inspects the appearance of a workpiece, but the present disclosure is not limited to this. In the present disclosure, the inspection system may be an inspection system that inspects the interior of a workpiece. In this case, the inspection unit may be an ultrasonic flaw detector that inspects the workpiece by transmitting ultrasonic waves into the interior of the workpiece and receiving the ultrasonic waves reflected from the interior of the workpiece. Using the ultrasonic flaw detector, it is possible to detect objects such as defects inside the workpiece. Furthermore, the inspection unit may be an eddy current inspection unit that inspects the workpiece by bringing a coil through which AC power is passed close to the workpiece and detecting eddy currents generated in the workpiece. Furthermore, the inspection unit may be an X-ray inspection unit that inspects the workpiece by irradiating X-rays on the workpiece and detecting the X-rays that have passed through the workpiece.

[0127] In addition, although the robot is a vertical articulated robot in each of the first to third embodiments, the present disclosure is not limited to this. In the present disclosure, the robot may be an industrial robot other than a vertical articulated robot.

[0128] In the first and second embodiments, an example has been described in which the robot moves the inspection unit relative to the workpiece, but the present disclosure is not limited to this. In the present disclosure, the robot may move the workpiece, thereby moving the inspection unit relative to the workpiece.

[0129] Furthermore, in the first and second embodiments, examples have been described in which the processing unit of the robot controller, the processing unit of the image processing device, and the processing unit of the result display device share the responsibilities of performing various processes. However, the present disclosure is not limited to this. In the present disclosure, the number and configuration of the processing units are not particularly limited. One processing unit may perform the various processes of the above embodiments, or multiple processing units may perform the various processes of the above first and second embodiments. The configuration of the storage unit is also not limited. The configurations of the robot controller, the image processing device, and the result display device are also not limited. The robot controller, the image processing device, and the result display device may be integrated, or may be separate, as in the first and second embodiments. Furthermore, the robot controller, the image processing device, and the result display device may be further separated. For example, an operation device for operating the result display device may be provided separately from the result display device.

[0130] Furthermore, in the first embodiment, an example was described in which the processing unit of the robot controller generates a movement path and generates coordinate transformation information. However, the present disclosure is not limited to this. In the present disclosure, as in a modified example shown in FIG. 34 , the inspection system 100 may include a simulation device 180 that generates a movement path 13 of the robot 10 on a display screen through simulation without using an actual robot. In this modified example, the simulation device 180 includes a processing unit 181, a storage unit 182, and a display unit 183. The processing unit 181 includes a processor and performs various processes related to the simulation. The storage unit 182 includes a non-volatile memory and stores the coordinate transformation information 71 or 72, etc. The display unit 183 includes a monitor such as an LCD monitor and displays a simulation screen, etc.

[0131] The processing unit 181 displays three-dimensional images of the robot 10, the inspection unit 20, and the workpiece 200 on the display unit 183 and performs a process of generating a movement path 13 of the robot 10 through simulation. The processing unit 181 also performs a process of generating coordinate transformation information 71 or 72 based on the generated movement path 13. The processing unit 181 also performs a process of storing the coordinate transformation information 71 or 72 in the storage unit 182. The processing unit 181 also performs a process of outputting the coordinate transformation information 71 to the processing unit 41 of the robot controller 40 and a process of outputting the coordinate transformation information 72 to the processing unit 61 of the result display device 60. Note that in the second embodiment, the process of generating a movement path of the robot may be performed by a simulation device. Note that in the third embodiment, the process of generating a movement path of the robot and the process of generating coordinate information may be performed by a simulation device.

[0132] In addition, in the first embodiment, an example has been shown in which both a process for displaying the position of the target on the actual workpiece and a process for displaying the position of the target on a three-dimensional image of the workpiece are performed, but the present disclosure is not limited to this. In the present disclosure, only one of a process for displaying the position of the target on the actual workpiece and a process for displaying the position of the target on a three-dimensional image of the workpiece may be performed.

[0133] In the first embodiment, the coordinate conversion information is a coordinate conversion table, but the present disclosure is not limited to this. In the present disclosure, the coordinate conversion information may be information other than a table, such as a coordinate conversion formula.

[0134] Although the first and second embodiments illustrate examples in which the indicator is a laser irradiation unit, the present disclosure is not limited thereto. In the present disclosure, the indicator is not particularly limited as long as it can indicate the position of the target on the workpiece. For example, in the modified example shown in FIG. 35 , the indicator 730 is an ink application unit that applies ink to indicate the position of the target 201 on the workpiece 200, a stamping unit that stamps the position of the target 201 on the workpiece 200, a paint gun that sprays paint to indicate the position of the target 201 on the workpiece 200, or an attachment unit that affixes a sticky note or other such label to indicate the position of the target 201 on the workpiece 200. In this case, as shown in FIG. 36 , in step S706, the processing unit 41 of the robot controller 40 performs processing to indicate the position of the target 201 on the actual workpiece 200 by applying ink, stamping, painting, or attaching. Unlike the first and second embodiments, the position of the target may be indicated on the actual workpiece by an augmented reality display device that displays augmented reality. The augmented reality display device displays an augmented reality image showing the position of the object, thereby showing the position of the object on the workpiece.

[0135] Furthermore, in the above first and second embodiments, an example was shown in which the instruction unit was arranged on the inspection robot, but the present disclosure is not limited to this. In the present disclosure, an instruction robot having an instruction unit arranged thereon may be provided separately from the inspection robot. The instruction robot operates to indicate the target position on the actual workpiece. Furthermore, the location where the inspection robot inspects the workpiece and the location where the instruction robot gives instructions to the workpiece may be separate, remote locations.

[0136] In the first embodiment, the first distance is greater than the second distance, but the present disclosure is not limited to this. In the present disclosure, the first distance may be equal to or less than the second distance.

[0137] In the second embodiment, an example was shown in which both the process of operating the robot so that the position of the target corresponding to the operation image is shown on the actual workpiece and the process of displaying the inspection image corresponding to the operation image are performed, but the present disclosure is not limited to this. In the present disclosure, only one of the process of operating the robot so that the position of the target corresponding to the operation image is shown on the actual workpiece and the process of displaying the inspection image corresponding to the operation image may be performed.

[0138] In the second embodiment, an example is shown in which the operation images are displayed in a list format together with the workpiece inspection results, but the present disclosure is not limited to this. In the present disclosure, the operation images may be displayed separately from the workpiece inspection results.

[0139] In the second embodiment, when an operation image is operated, the position of the target corresponding to the operated operation image is highlighted in the three-dimensional image of the workpiece. However, the present disclosure is not limited to this. In the present disclosure, when an operation image is operated, the position of the target corresponding to the operated operation image does not have to be highlighted in the three-dimensional image of the workpiece.

[0140] In the second embodiment, an example is shown in which an identification indicator indicating whether or not treatment on a target has been completed is displayed, but the present disclosure is not limited to this. In the present disclosure, an identification indicator indicating whether or not treatment on a target has been completed does not necessarily have to be displayed.

[0141] In the second embodiment, the positions of treated targets are displayed in a color different from that of targets for which treatment has not been completed in the three-dimensional image, but the present disclosure is not limited to this. In the present disclosure, the positions of treated targets do not have to be displayed in a color different from that of targets for which treatment has not been completed in the three-dimensional image.

[0142] Furthermore, in the first and second embodiments, examples have been described in which the tip of the arm unit holds the inspection unit and the support unit, but the present disclosure is not limited thereto. For example, in a modified example shown in FIG. 37 , the tip of the arm unit 12 may hold the inspection unit 20, the instruction unit 30, and the working unit 820a. The working unit 820a performs an operation on the workpiece 200 that requires inspection by the inspection unit 20. For example, the working unit 820a may be a dispenser that applies a sealant to the workpiece 200. The working unit 820a may also be an application unit that applies a sealant or tape to the workpiece 200. The working unit 820a may also be a painting unit that paints the workpiece 200. The working unit 820a may also be a polishing unit that polishes, deburrs, grinds, buffs, etc. the workpiece 200. The working unit 820a may also be a sewing unit that sews the workpiece 200. Furthermore, the working unit 820a may be an application unit that applies a highly viscous liquid such as sauce, mayonnaise, or chocolate to the workpiece 200. Furthermore, the working unit 820a may be a car washing unit that washes the workpiece 200 such as a car.

[0143] Furthermore, in the third embodiment, an example was shown in which the working unit is a painting unit, but the present disclosure is not limited to this. For example, the working unit may be an inspection unit that inspects a workpiece. For example, the working unit may be a dispenser that applies a sealant to a workpiece. For example, the working unit may be an application unit that applies a sealant or tape to a workpiece. For example, the working unit may be a painting unit that paints a workpiece. For example, the working unit may be a polishing unit that polishes, deburrs, grinds, buffs, etc. the workpiece. For example, the working unit may be a sewing unit that sews the workpiece. For example, the working unit may be an application unit that applies a viscous liquid such as sauce, mayonnaise, or chocolate to the workpiece. For example, the working unit may be a car wash unit that washes a workpiece such as a car.

[0144] In the third embodiment, the processing unit controls the work start position based on the coordinate information, but the present disclosure is not limited to this. In the present disclosure, the processing unit may control a position other than the work start position based on the coordinate information.

[0145] In the third embodiment, the robot moves the working unit relative to the workpiece, but the present disclosure is not limited to this. In the present disclosure, the robot may move the workpiece, thereby moving the working unit relative to the workpiece.

[0146] Furthermore, in the third embodiment, an example was shown in which the processing unit of the robot controller and the processing unit of the working unit control device shared the responsibilities of performing various processes, but the present disclosure is not limited to this. In the present disclosure, the number and configuration of the processing units are not particularly limited. One processing unit may perform the various processes of the third embodiment, or multiple processing units may perform the various processes of the above embodiments. The configuration of the storage unit is also not limited. The configurations of the robot controller and the working unit control device are also not limited. The robot controller and the working unit control device may be configured as an integrated unit, or may be configured separately as in the third embodiment. The robot controller and the working unit control device may also be configured further separately.

[0147] In the third embodiment, the coordinate information is a coordinate table, but the present disclosure is not limited to this. In the present disclosure, the coordinate information may be information other than a table, such as an equation.

[0148] The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, special-purpose processors, integrated circuits, ASICs (Application Specific Integrated Circuits), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuitry. In this disclosure, a circuit, unit, or means is hardware that performs the recited functions or hardware that is programmed to perform the recited functions. The hardware may be hardware disclosed herein or other known hardware that is programmed or configured to perform the recited functions. Where the hardware is a processor, which is considered a type of circuit, the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0149] [Aspects] The above-described embodiments are specific examples of the following aspects.

[0150] (Aspect 1) An inspection system comprising: a robot; an inspection unit that inspects a workpiece; and a processing unit that performs the following operations: a process of generating a movement path for the robot when the inspection unit inspects the workpiece by moving the inspection unit relative to the workpiece using the robot; a process of generating coordinate transformation information, based on the generated movement path, for converting coordinate values ​​of an inspection coordinate system of an inspection image obtained by inspecting the workpiece using the inspection unit into coordinate values ​​of a three-dimensional coordinate system that can represent the coordinate values ​​of the workpiece in three dimensions; a process of operating the robot based on the generated movement path and inspecting the workpiece using the inspection unit to obtain the inspection image; a process of detecting an object of the workpiece in the obtained inspection image; a process of converting coordinate values ​​of the inspection coordinate system of the object into coordinate values ​​of the three-dimensional coordinate system based on the generated coordinate transformation information; and a process of indicating the position of the object on the actual workpiece or a three-dimensional image of the workpiece, based on the converted coordinate values ​​of the object in the three-dimensional coordinate system.

[0151] (Aspect 2) In the inspection system according to aspect 1, the processing unit acquires coordinate values ​​of the three-dimensional coordinate system at first distance intervals along the movement path and performs processing to generate the coordinate transformation information.

[0152] (Aspect 3) The inspection system according to Aspect 2, wherein the processing unit operates the inspection unit to inspect the workpiece at second distance intervals along the movement path, and performs processing to acquire the inspection image, and the first distance interval is greater than the second distance interval.

[0153] (Aspect 4) An inspection system according to any one of Aspects 1 to 3, wherein the coordinate transformation information is a coordinate transformation table that associates the movement amount of the robot in a direction along the movement path with coordinate values ​​in the three-dimensional coordinate system, and the processing unit, in the process of converting the coordinate values ​​of the inspection coordinate system of the object into coordinate values ​​in the three-dimensional coordinate system, performs the following processes: acquires the movement amount of the robot in the coordinate transformation table that corresponds to the coordinate value of the inspection coordinate system in the direction along the movement path of the object; acquires the coordinate value of the three-dimensional coordinate system in the coordinate transformation table that corresponds to the movement amount of the robot in the acquired coordinate transformation table; and corrects the coordinate value of the three-dimensional coordinate system in the acquired coordinate transformation table based on the coordinate value of the inspection coordinate system in a direction perpendicular to the movement path of the object.

[0154] (Aspect 5) An inspection system according to any one of Aspects 1 to 4, further comprising an indicator disposed on the robot for indicating the position of the target relative to the actual workpiece, the three-dimensional coordinate system being a robot coordinate system relating to the robot, and the processing unit operating the robot based on the coordinate values ​​of the target converted into coordinate values ​​in the robot coordinate system, and performing processing to indicate the position of the target on the actual workpiece using the indicator.

[0155] (Aspect 6) An inspection system according to any one of Aspects 1 to 5, wherein the three-dimensional coordinate system is a work coordinate system relating to the work, and the processing unit performs processing to indicate the position of the object in a three-dimensional image of the work based on the coordinate values ​​of the object converted into coordinate values ​​of the work coordinate system.

[0156] (Aspect 7) An inspection system according to any one of Aspects 1 to 6, wherein the three-dimensional coordinate system is a robot coordinate system relating to the robot, and the processing unit, when the coordinate values ​​of the plurality of objects converted into the coordinate values ​​of the robot coordinate system are within a threshold range, performs processing to show the positions of the objects on the actual workpiece as a single object for the plurality of objects.

[0157] (Aspect 8) An inspection system according to any one of Aspects 1 to 7, wherein the processing unit performs the following processes: displaying the position of the object on a three-dimensional image of the workpiece; and, when input is made to the displayed object, performing processing related to the actual workpiece in relation to the input object.

[0158] (Aspect 9) The processing unit displays an operation image corresponding to the target, and when the displayed operation image is operated, performs processing on the actual workpiece with respect to the target corresponding to the operated operation image. This is the inspection system described in Aspect 8.

[0159] (Aspect 10) In the inspection system according to Aspect 9, when the operation image is operated, the processing unit performs one of the following processes: operating the robot so that the position of the target corresponding to the operated operation image is shown on the actual workpiece; and displaying the inspection image of the target corresponding to the operated operation image.

[0160] (Aspect 11) The inspection system according to Aspect 10, wherein when the operation image is operated, the processing unit performs processing to highlight the position of the target corresponding to the operated operation image in the three-dimensional image and to operate the robot so that the position of the target corresponding to the operated operation image is shown on the actual workpiece.

[0161] (Aspect 12) The inspection system according to aspect 10 or 11, wherein when the operation image is operated, the processing unit performs processing to highlight the position of the object corresponding to the operated operation image in the three-dimensional image and to display the inspection image of the object corresponding to the operated operation image.

[0162] (Aspect 13) The operation image includes a first operation image for performing a process of operating the robot so that the position of the target is shown on the actual workpiece, and a second operation image for performing a process of displaying the inspection image of the target, and the processing unit, when the first operation image is operated, performs a process of operating the robot so that the position of the target corresponding to the operated first operation image is shown on the actual workpiece, and when the second operation image is operated, performs a process of displaying the inspection image of the target corresponding to the operated second operation image, an inspection system described in any one of Aspects 10 to 12.

[0163] (Aspect 14) An inspection method comprising: generating a movement path for the robot when an inspection unit is moved relative to a workpiece by the robot to inspect the workpiece with the inspection unit; generating coordinate transformation information, based on the generated movement path, for converting coordinate values ​​of an inspection coordinate system of an inspection image obtained by inspecting the workpiece with the inspection unit into coordinate values ​​of a three-dimensional coordinate system that can represent the coordinate values ​​of the workpiece in three dimensions; operating the robot based on the generated movement path to inspect the workpiece with the inspection unit, thereby obtaining the inspection image; detecting an object of the workpiece in the obtained inspection image; converting coordinate values ​​of the inspection coordinate system of the object into coordinate values ​​of the three-dimensional coordinate system based on the generated coordinate transformation information; and indicating the position of the object on the actual workpiece or a three-dimensional image of the workpiece, based on the converted coordinate values ​​of the object in the three-dimensional coordinate system.

[0164] (Aspect 15) A robot system comprising: a robot; a working unit that performs work on a workpiece; and a processing unit that performs the following processes: a process of generating a movement path for the robot when the robot moves the working unit relative to the workpiece and the working unit performs work on the workpiece; a process of generating, based on the generated movement path, coordinate information including an amount of movement of the robot in a direction along the movement path and coordinate values ​​of a three-dimensional coordinate system that can express coordinate values ​​of the workpiece in three dimensions; and a process of operating the robot based on the generated movement path and the coordinate information, so that the working unit performs work on the workpiece.

[0165] 10, 310, 510 Robot 13, 313, 513 Movement path 20, 320 Inspection unit 21, 321 Inspection image 30 Instruction unit 41, 341, 541 Processing unit 51, 351, 551 Processing unit 61, 361 Processing unit 71, 72 Coordinate conversion information 100, 300 Inspection system 181 Processing unit 200, 400, 600 Workpiece 201, 401 Object 371 Operation image (first operation image) 372 Operation image (second operation image) 500 Robot system 520 Working unit 571 Coordinate information D1 First distance interval D2 Second distance interval

Claims

DEPCT6929 / 05 / 25691. The inspection system comprises: an inspection robot that inspects the workpiece and a processor that performs the following processing: inspection movement relative to the workpiece using the robot to create a robotic trajectory for inspecting the workpiece; generation of coordinate transformation information to convert the coordinate values ​​of the inspection image into the inspection coordinate system obtained by inspecting the workpiece using the inspector, based on the generated trajectory, into three-dimensional coordinates representing the workpiece in three dimensions; deployment of the robot based on the generated trajectory to inspect the workpiece using the inspector and to provide images for inspection; detection of the target on the workpiece in the obtained inspection image; conversion of the target's coordinate values ​​in the inspection coordinate system to three-dimensional coordinates based on the generated coordinate transformation information; and identification of the target's position on the actual workpiece or in the three-dimensional image of the workpiece based on the converted coordinate values ​​of the target in three-dimensional coordinates.2.A verification system under claim 1 where the processor performs processing to provide coordinate values ​​in a three-dimensional coordinate system with a first interval along the trajectory to generate coordinate transformation information.

3. A verification system under claim 2 where the processor performs processing to use the inspector along the trajectory to inspect the workpiece with a second interval and to provide an image for inspection and the first interval is larger than the second interval. 4.The monitoring system, under Claim 1, provides coordinate transformation information in a transformation table that links the robot's displacement quantities along the trajectory to three-dimensional coordinate values. The processor performs the following processing to transform the target's coordinates in the monitoring system into three-dimensional coordinate values: providing the robot's displacement quantities in the transformation table that correspond to the target's coordinates in the monitoring system along the trajectory; providing three-dimensional coordinate values ​​in the transformation table that correspond to the received robot displacement quantities in the transformation table; and correcting the received three-dimensional coordinate values ​​in the transformation table based on the target's coordinates in the monitoring system in the direction perpendicular to the trajectory.The verification system under Claim 1, which incorporates additional indicators arranged on the robot and indicates the position of the target relative to the actual workpiece, where the three-dimensional coordinate system is the robot's coordinate system linked to the robot and the processor performs processing to operate the robot based on the target's coordinates, which are converted to coordinates in the robot's coordinate system to indicate the target's position on the actual workpiece using indicator 6. The verification system under Claim 1, where the three-dimensional coordinate system is the workpiece's coordinate system linked to the workpiece and the processor performs processing to indicate the target's position in the three-dimensional image of the workpiece based on the target's coordinates, which are converted to coordinates in the workpiece's coordinate system 7.A verification system under Claim 1 where the three-dimensional coordinate system is the robot's coordinate system linked to the robot and the processor performs processing to handle more than one target as a single target and indicates the position of the target on the actual workpiece when the coordinates of more than one target, which are converted to coordinates in the robot's coordinate system, are within the threshold range.

8. A verification system under Claim 1 where the processor performs the following processing: displaying the position of the target in three-dimensional image of the workpiece and performing processing on the actual workpiece for the target where the input is received, in response to the input that will be received for the displayed target.

9. A verification system under Claim 8 where the processor performs processing to display an operational image corresponding to the target and performing processing on the actual workpiece for the target corresponding to the operational image used when the displayed operational image is used. 10.

11. The inspection system under claim 10, where when an operational image is used, the processor performs one of the following actions: processing to use a robot to identify the position of the target corresponding to the operational image used on the actual workpiece, or processing to display an image for inspection of the target corresponding to the operational image used.

12. The inspection system under claim 10, where when an operational image is used, the processor performs processing to highlight the position of the target corresponding to the operational image used in three dimensions and processing to use a robot to identify the position of the target corresponding to the operational image used on the actual workpiece.

13. The inspection system under claim 10, where when an operational image is used, the processor performs processing to highlight the position of the target corresponding to the operational image used in three dimensions and processing to display an image for inspection of the target corresponding to the operational image used.The inspection system under claim 10 includes the following operational images: the first operational image corresponding to the processing used to operate the robot to identify the position of the target on the actual workpiece, and the second operational image corresponding to the processing used to display the image for inspection of the target. The processor performs the following processing: operating the robot to identify the position of the target corresponding to the first operational image used on the actual workpiece when the first operational image is used, and displaying the image for inspection of the target corresponding to the second operational image used when the second operational image is used.14.The inspection method comprises: relative movement inspection using a robot to create a robotic trajectory for inspecting the workpiece; generating coordinate transformation information to convert the coordinate values ​​of the inspection image into the inspection coordinate system obtained by inspecting the workpiece using the inspector based on the generated trajectory; converting the coordinate values ​​of the target in the inspection coordinate system into three-dimensional coordinates representing the workpiece in three dimensions; using the robot based on the generated trajectory to inspect the workpiece using the inspector and to provide images for inspection; detecting the target on the workpiece in the obtained inspection image; converting the coordinate values ​​of the target in the inspection coordinate system to three-dimensional coordinates based on the generated coordinate transformation information; and identifying the position of the target on the actual workpiece or in the three-dimensional image of the workpiece based on the converted coordinate values ​​of the target in the three-dimensional coordinate system.15.The robotic system consists of: a robot effector that operates on the workpiece and a processor that performs the following processing: movement of the effector relative to the workpiece, using the robot to generate the robot's motion trajectory during work on the workpiece using the effector; generation of coordinate information including the amount of robot movement along the motion trajectory and coordinate values ​​in a three-dimensional coordinate system that represents the workpiece in three dimensions based on the generated motion trajectory; and operation of the robot based on the generated motion trajectory and coordinate information to perform operations on the workpiece using the effector.

Images