Optical inspection method, optical inspection program, processing device, and optical inspection device

Abstract

To provide a method for an optical examination which can optically examine the desired range of the surface of a target object without missing a spot (image missing).SOLUTION: The method for an optical examination includes: calculating irradiation field information on an irradiation field in the surface of a target body when the surface of the target body is irradiated with a beam of light from a movable illumination device supported by a movable body; and performing route calculation processing of calculating a route where the illumination device moves on the basis of the irradiation field information.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] Embodiments of the present invention relate to an optical inspection method, an optical inspection program, a processing device, and an optical inspection apparatus.

Background Art

[0002] There is a method of optically inspecting the presence or absence of scratches or the like on the surface of a subject by moving a robot equipped with a camera to scan the surface of the subject with a complex shape using the camera.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem to be solved by the present invention is to provide an optical inspection method, an optical inspection program, a processing device, and an optical inspection device that can perform optical inspection without overlooking (image missing) a desired range in the surface inspection of a subject.

Means for Solving the Problem

[0006] According to an embodiment, the optical inspection method includes irradiation field information regarding an irradiation field on the surface of the subject when irradiating the surface of the subject with a light beam from an illumination device supported and moved by a movable body, and Adjust the position and orientation so that at least a part of the surface of the subject within the angular range is orthogonal or substantially orthogonal to the optical axis. calculating imaging irradiation field information regarding an imaging irradiation field in which the surface of the subject can be imaged by a moving imaging device, and based on the irradiation field information and the imaging irradiation field information, when imaging an imaging point within the imaging irradiation field included in the irradiation field where illumination light is applied to the surface of the subject with the imaging device, performing a path calculation process of calculating a path along which the illumination device and the imaging device move so that the position and orientation of the imaging device are such that the image contrast becomes clear.

Brief Description of the Drawings

[0007] [FIG. 1] Schematic diagram showing an optical inspection device according to the first embodiment. [FIG. 2] Schematic block diagram of an optical inspection device according to the first embodiment. [FIG. 3] Schematic diagram showing an example of shape data of the surface of a subject stored in a data storage unit in a processing device according to the first embodiment. [FIG. 4] Schematic diagram for explaining an example of a process of setting (calculating) a plurality of imaging points performed by a path calculation unit of a processing device according to the first embodiment. [FIG. 5] Schematic diagram for explaining an example of a process of setting (calculating) one or more of a plurality of imaging points based on one polygon formed by a plurality of triangles arranged in a minute portion of an angle difference, performed by a path calculation unit of a processing device according to the first embodiment. [FIG. 6]A flowchart schematically showing an example of a process executed by a process execution unit of a processing apparatus according to the first embodiment. [FIG. 7] A subroutine showing an example of some of the processes in FIG. 6. [FIG. 8] A subroutine showing an example of some of the processes in FIG. 6. [FIG. 9] A schematic diagram showing an optical inspection apparatus according to a modification of the first embodiment. [FIG. 10] A schematic diagram showing an optical inspection apparatus according to the second embodiment. [FIG. 11] A schematic diagram showing an optical inspection apparatus according to a first modification of the second embodiment. [FIG. 12] A schematic diagram showing an optical inspection apparatus according to a second modification of the second embodiment.

Embodiments for Carrying Out the Invention

[0008] Hereinafter, some embodiments will be described with reference to the drawings.

[0009] (First Embodiment) The optical inspection apparatus 1 according to the first embodiment will be described with reference to FIGS. 1 to 8. Hereinafter, imaging is treated as synonymous with photographing.

[0010] FIG. 1 is a diagram schematically showing the optical inspection apparatus 1 according to the first embodiment. FIG. 2 is a block diagram schematically showing the control system and the like of the optical inspection apparatus 1 of the present embodiment. As shown in FIGS. 1 and 2, the optical inspection apparatus 1 of the present embodiment is used for optical inspection of the surface of a specimen (object) S to be inspected. In one inspection, using the optical inspection apparatus 1, an optical inspection is performed on a certain range on the surface of the specimen S. In one example, the entire surface of the specimen S may be inspected in one inspection. In another example, only a predetermined range (a part) on the surface of the specimen S may be inspected in one inspection. The entire surface of the specimen S may be used as the predetermined range. That is, the entire surface of the specimen S may be used as the predetermined range, or a part of the surface of the specimen S may be used as the predetermined range.

[0011] The optical inspection device 1 includes an illumination unit (illumination device) 2, an imaging unit (imaging device) 3, a movable body 4, and a processing device 5.

[0012] The processing device 5 includes a processing execution unit 6, a data storage unit 7, and a user interface 8. The processing execution unit 6 includes a data acquisition unit 11, a path calculation unit 12, an image processing unit 13, and an operation control unit 15.

[0013] The illumination unit 2 irradiates the surface of the object S with a light beam B. The light beam B is an electromagnetic wave such as X-rays, ultraviolet rays, visible light, infrared rays, far-infrared rays, microwaves, etc. Here, the light beam B is, for example, visible light, and the wavelength range is from 400 nm to 800 nm. In this embodiment, the illumination unit 2 uses, for example, an LED or the like to irradiate the light beam B in a desired direction. Then, the light beam B from the illumination unit 2 appropriately illuminates a desired range on the surface of the object S. When the light beam B from the illumination unit 2 hits the surface of the object S, the area where the light beam B hits is called the irradiation field IF. The imaging unit 3 images the inside of the irradiation field where the light beam from the illumination unit 2 irradiates the surface of the object S and acquires an image. The imaging unit 3 is an imaging device such as a camera. The imaging unit 3 acquires a light beam (reflected light) B1 that is incident after the light beam irradiated from the illumination unit 2 is reflected from the surface of the object S. When the light beam (reflected light) B1 reflected in the irradiation field IF on the surface of the object S enters the imaging aperture 31 of the imaging unit 3 and is within the imaging angle that can be imaged as an image, the area on the surface of the object S that reflects such a light beam is called the imaging irradiation field PF1. Also, the entire area on the surface of the object S that can be imaged by the imaging unit 3 is simply called the imaging area. That is, it is called the imaging area including the area that becomes dark in the acquired image without illumination. The shape of the imaging aperture 31 is determined by the imaging unit 3. For example, if the imaging optical element included in the imaging unit 3 is a lens with an effective diameter of φ40 mm, the imaging aperture 31 has a disk shape of φ40 mm. The light beam incident on the imaging aperture 31 is imaged on the image sensor by the optical element of the imaging unit 3 and is acquired as an image by the image sensor. And the imaging unit 3 of the optical inspection device 1 according to this embodiment images the imaging irradiation field PF1 that is within the range of the irradiation field IF where the object S is illuminated.

[0014] Imaging by the imaging unit 3, that is, acquisition of an image, is performed by exposing the image sensor. The exposure is performed by the imaging shutter of the imaging unit 3. Note that the imaging shutter may be electrical or mechanical. However, not limited to this, instead of the imaging shutter of the imaging unit 3, it is also possible to turn on / off the illumination of the illumination unit 2 to expose the image sensor of the imaging unit 3 and acquire an image. In this case, when the illumination of the illumination unit 2 is turned on and a predetermined position of the subject S is illuminated, an image is acquired by the image sensor of the imaging unit 3.

[0015] The image sensor is one in which a plurality of pixels that convert light into an electrical signal are arranged, and there are, for example, an area sensor or a line sensor depending on the shape of the arrangement. Each pixel includes a plurality of channels that receive different wavelength spectra. However, not limited thereto, each pixel may include one channel that receives one wavelength spectrum.

[0016] The movable body 4 is controlled by the processing device 5. The movable body 4 is, for example, either a robot arm having one or more joints, or a linear motion mechanism, a parallel link mechanism, or the like. In the present embodiment, the movable body 4 includes a base 40, a first robot 41 that arranges the illumination unit 2 at a desired position and in a desired orientation within a predetermined range, and a second robot 42 that arranges the imaging unit 3 at a desired position and in a desired orientation within a predetermined range.

[0017] The first robot 41 of the movable body 4 is mechanically connected to the illumination unit 2. In an example such as FIG. 1, one end of the first robot 41 is supported by the base 40, and a connection portion to the illumination unit 2 is formed at the other end. When the first robot 41 operates, the position and orientation of the illumination unit 2 change within a predetermined range. Then, the first robot 41 maintains the illumination unit 2 in a state where it is at a desired position and in a desired orientation within a predetermined range. Therefore, the position and orientation of the illumination unit 2 change corresponding to the operation of the first robot 41.

[0018] The illumination unit 2 moves, for example, along the first scan path D1. The first scan path D1 may be, for example, one-dimensional moving in one axial direction, two-dimensional moving along one plane, or three-dimensional moving along one curved surface. In any case, the scan path is a line or a curve. Also, the scan direction at any point is the direction vector along the scan path at that point. In the present embodiment, the position and orientation of the first robot 41 are, for example, indicated by the position and orientation in three dimensions.

[0019] The irradiation field IF of the subject S by the illumination unit 2 varies depending on the surface shape of the subject S. For example, the first irradiation field IF1 at a certain position on the surface of the subject S changes to a second irradiation field IF2 different from the first irradiation field IF1 due to the movement of the illumination unit 2. Therefore, the illumination unit 2 of the optical inspection apparatus 1 according to the present embodiment is used as so-called active illumination.

[0020] The second robot 42 of the movable body 4 is mechanically connected to the imaging unit 3. In an example such as FIG. 1, one end of the second robot 42 is supported by the base 40, and a connection portion to the imaging unit 3 is formed at the other end. By operating the second robot 42, the position and orientation of the imaging unit 3 change within a predetermined range. And the second robot 42 maintains the imaging unit 3 in a state where it has a desired position and orientation within a predetermined range. Therefore, the position and orientation of the imaging unit 3 change corresponding to the operation of the second robot 42.

[0021] The imaging unit 3 moves, for example, along a second scan path D2 different from the first scan path D1. The second scan path D2 may be, for example, one-dimensional moving in one axial direction, two-dimensional moving along one plane, or three-dimensional moving on a curved surface. That is, it is a line or a curve. In the present embodiment, the position and orientation of the second robot 42 are, for example, indicated by the position and orientation in three dimensions.

[0022] The imaging irradiation field and imaging region of the subject S by the imaging unit 3 vary depending on the surface shape of the subject S. For example, the first imaging irradiation field PF1 at a certain position on the surface of the subject S changes to a second imaging irradiation field PF2 different from the first imaging irradiation field PF1 due to the movement of the illumination unit 2 and the imaging unit 3.

[0023] Hereinafter, a point on the object surface imaged by the imaging unit 3 is referred to as an imaging point.

[0024] Note that the arrangements of the first robot 41 and the second robot 42 with respect to the base 40 can be set as appropriate.

[0025] The processing device 5 is composed of, for example, a computer or the like, and includes a processor (processing circuit) and a storage medium. The processor includes any one of a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), a microcomputer, an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), and the like. The storage medium may include an auxiliary storage device in addition to a main storage device such as a memory. Examples of the storage medium include a magnetic disk, an optical disk (CD-ROM, CD-R, DVD, etc.), a magneto-optical disk (MO, etc.), and a semiconductor memory.

[0026] In the processing device 5, only one or a plurality of each of the processor and the storage medium may be provided. In the processing device 5, the processor performs processing by executing programs and the like stored in the storage medium and the like. Further, the program executed by the processor of the processing device 5 may be stored in a computer (server) connected to the processing device 5 via a network such as the Internet, or a server in a cloud environment. In this case, the processor downloads the program via the network. In the processing device 5, the processing described later by the processing execution unit 6 including the data acquisition unit 11, the path calculation unit 12, the image processing unit 13, and the operation control unit 15 is executed by the processor and the like, and the storage medium functions as the data storage unit 7.

[0027] Note that a plurality of processing devices (computers) separate from each other may be provided in the optical inspection apparatus 1. In this case, each of the plurality of processing devices includes a processor and a storage medium. Then, the processing described later by the processing execution unit 6 is executed by the processors of the plurality of processing devices. In one example, the processing described later by the data acquisition unit 11 and the path calculation unit 12 is executed by the processor of a certain one processing device. And the processing described later by the image processing unit 13 and the operation control unit 15 is executed by the processor of a processing device separate from the processing device that executes the processing by the data acquisition unit 11 and the path calculation unit 12.

[0028] Also, at least a part of the processing by the processing execution unit 6 may be executed by a cloud server configured in a cloud environment. The infrastructure of the cloud environment is composed of virtual processors such as virtual CPUs and cloud memories. In one example, the processing by the data acquisition unit 11 and the path calculation unit 12 is executed by a virtual processor, and the cloud memory functions as the data storage unit 7. And the processing by the image processing unit 13 and the operation control unit 15 is executed by the processor of a processing device such as a computer. Also, the data storage unit 7 may be provided in a computer separate from the processing device 5 in which the processing execution unit 6 is provided. In this case, the processing device 5 is connected to the computer in which the data storage unit 7 and the like are provided via a network.

[0029] Also, the user interface 8 includes an operation unit into which an operation command is input by an operator or the like in the inspection using the optical inspection apparatus 1, and a notification unit that notifies an operator or the like of information related to the inspection. In the notification unit, information is notified, for example, by any of screen display and voice. Note that the user interface 8 may be provided separately from the processing device 5.

[0030] The first robot 41 and the second robot 42 of the movable body 4 operate by the operation of one or a plurality of actuators. The operation control unit 15 of the processing execution unit 6 of the processing device 5 controls the operation of the actuators that operate the first robot 41 and the second robot 42 of the movable body 4. Therefore, the operation control unit 15 controls the operations of the first robot 41 and the second robot 42 of the movable body 4. By controlling the operations of the first robot 41 and the second robot 42 of the movable body 4 by the operation control unit 15, the position and orientation of the connection portion of the first robot 41 to the illumination unit 2 are adjusted, and the position and orientation of the connection portion of the second robot 42 to the imaging unit 3 are adjusted. That is, by adjusting the position and orientation of the connection portion of the first robot 41 to the illumination unit 2 by the operation control unit 15, the position and orientation of the illumination unit 2 are adjusted. Also, by adjusting the position and orientation of the connection portion of the second robot 42 to the imaging unit 3 by the operation control unit 15, the position and orientation of the imaging unit 3 are adjusted.

[0031] When the imaging unit 3 images a certain position of the subject S, that position is defined as the imaging point, and the operation control unit 15 controls the position and orientation of the illumination unit 2 such that the light beam from the illumination unit 2 illuminates the imaging point and its surroundings, and controls the position and orientation of the imaging unit 3 such that the angle of the optical axis O of the imaging unit 3 is within a predetermined angular range with respect to the surface of the subject S at that imaging point. That is, the operation control unit 15 controls the positions and orientations of the first robot 41 and the second robot 42 such that the imaging point and its vicinity where the subject S is illuminated by the illumination unit 2 are imaged by the imaging unit 3. The imaging unit 3 performs imaging using an imaging optical element such as a lens. Generally, imaging by an imaging optical element focuses on a plane orthogonal to the optical axis O of the imaging optical element. Therefore, in an example such as FIG. 1, it is preferable that the operation control unit 15 adjusts the position and orientation of the imaging unit 3 such that at least a part of the surface of the subject S within the angle-of-view range is orthogonal (the angle is a right angle) or substantially orthogonal to the optical axis O of the imaging unit 3 in the imaging of the subject S. On the other hand, when the region where the optical axis O is orthogonal to the surface (that is, the direction of the normal to the surface coincides with the direction of the optical axis O) is not included within the angle-of-view range, there is no in-focus region, so the sharpness of the image is impaired.

[0032] An electrical signal generated by the image sensor of the imaging unit 3 is input to the image processing unit 13 of the processing execution unit 6 of the processing device 5. Therefore, when imaging is performed by the imaging unit 3, the image processing unit 13 acquires information regarding the image formed on the image sensor. The image processing unit 13 generates a captured image by the imaging unit 3 as an image of the imaging range of the imaging unit 3 based on the electrical signal from the image sensor. Then, the image processing unit 13 executes operations and determinations based on the captured image by performing image processing on the image captured by the imaging unit 3.

[0033] In the actual optical inspection by the optical inspection device 1, the object S is placed at a predetermined position in a predetermined orientation. However, this is not limited thereto, and the object S may be moving.

[0034] The data acquisition unit 11 of the processing execution unit 6 of the processing device 5 acquires the surface information of the object S from the CAD data of the object S on the assumption that the object S is placed at a predetermined position in a predetermined orientation. However, when the object S is moving, the data acquisition unit 11 acquires the surface information from the CAD data corresponding to the object S at the position at a certain time. FIG. 3 shows an example of the shape data of the surface of the object S. In the example of FIG. 3, the shape data is shown in the STL (standard triangulated language) format. In the shape data of the example of FIG. 3, the shape of the surface of the object S is represented by a point cloud composed of a large number of points P. Also, in the shape data of the example of FIG. 3, based on the point cloud, the surface of the object S is divided into a large number of triangles T. In each of the large number of triangles T, three corresponding points P constituting the point cloud become the vertices. In the shape data of the example of FIG. 3, as information regarding the position on the surface of the object S, positions such as the three-dimensional positions of each of the points P constituting the point cloud are shown. And in the shape data, as information regarding the position on the surface of the object S, the normal vector of each of the large number of triangles T is shown.

[0035] Here, the normal vector is orthogonal to the surface of the object S and faces the outside of the object S. In FIG. 3, the illustration of the normal vector is omitted. Also, in the shape data, for example, the surface of the object S may be divided into a plurality (a large number) of polygons other than triangles, such as being divided into a large number of quadrilaterals. Also in this case, each vertex of the polygon is formed from four or more corresponding points P that constitute the point group, and in the shape data, as information regarding the position on the surface of the object S, the normal vectors of each of the plurality of polygons are shown.

[0036] The data acquisition unit 11 reads out the shape data of the object S from the data storage unit 7 or through the user interface 8. Then, based on the shape data, the data acquisition unit 11 acquires information regarding the shape of the surface of the object S in the inspection range. Also, the data acquisition unit 11 acquires information such as the area that can be illuminated by the illumination unit 2 (irradiation field), the area that can be imaged by the imaging unit 3 (imaging irradiation field), the distance between the imaging unit 3 and the object S, the resolution of the imaging unit 3, and the positions and postures where the illumination unit 2, the imaging unit 3, and the movable body 4 can be arranged, with respect to the information stored in the data storage unit 7 and / or the information set by the user interface 8. That is, the data acquisition unit 11 acquires various information regarding the illumination unit 2, the imaging unit 3, and the movable body 4.

[0037] Based on the shape data of the object S and the information regarding the imaging unit 3 and the movable body 4, the path calculation unit 12 of the processing execution unit 6 of the processing device 5 calculates the irradiation field and the imaging irradiation field based on the CAD data in the inspection range, and calculates the positional relationship between the plurality of imaging points for imaging, the illumination unit 2, and the imaging unit 3.

[0038] In FIG. 4, an example of a process of setting (determining) a plurality of imaging points using the path calculation unit 12 will be described. In an example of FIG. 4, the path calculation unit 12 may use, as the plurality of imaging points F, three imaging points Fa to Fc from the inspection range on the surface of the subject S set in advance. Then, the path calculation unit 12 defines an imaging range Ra at the imaging point Fa, an imaging range Rb at the imaging point Fb, and an imaging range Rc at the imaging point Fc based on the irradiation field where the light beam from the illumination unit 2 irradiates the surface of the subject S and the imaging irradiation field that can be imaged by the imaging unit 3. As will be described later, the irradiation field information can be theoretically calculated in advance by the path calculation unit 12 of the processing device 5 using the light beam information of the light beam emitted from the illumination unit 2 and the surface information of the subject S based on the CAD data. Also, the imaging irradiation field can be logically calculated in advance if the positional relationship between the imaging unit 3 and the subject S is known. Further, in an example of FIG. 4, a triangle Ta, which is one of the triangles T arranged in the inspection range, is shown. In the triangle Ta, any part is included in any one or more of the imaging ranges Ra to Rc. For this reason, the path calculation unit 12 calculates the plurality of imaging points F in a state where any part of the triangle Ta is imaged by any one or more of the imaging points Fa to Fc. Also, for each of the triangles T other than the triangle Ta arranged in the inspection range, the path calculation unit 12 calculates the plurality of imaging points F in a state where any part of the triangle T is imaged by any one or more of the plurality of imaging points F.

[0039] Also, in an example of FIG. 4, in calculating imaging points Fa to Fc, triangle Ta is divided into three polygons. Then, triangle Ta is divided such that each of the three polygons contains one corresponding imaging point among Fa to Fc. Also, triangle Ta is divided such that the centroid of each of the three polygons coincides with or substantially coincides with one corresponding imaging point among Fa to Fc. As a result, for each of imaging points Fa to Fc, the centroid of one corresponding polygon divided from triangle Ta is calculated. When calculating the centroid of each of the divided polygons as one corresponding imaging point, each imaging point is calculated using a centroidal Voronoi tessellation method or a well-known rectangle partition method or the like.

[0040] In addition, within the inspection range of the subject S, there may be a minute angular difference portion where the deviation of the angle formed by the normal direction of the surface of the subject S between a plurality of adjacent triangles T is minute or zero. Here, in the state where the subject S is being imaged by the imaging unit 3, the angle formed by the optical axis O of the imaging unit 3 with respect to the normal direction of the surface of the subject S may be adjusted within a predetermined angle range as described above. That is, the more the optical axis O and the normal direction of the object surface coincide, the sharper the focused image can be obtained. In the minute angular difference portion, even if the angular difference between a plurality of adjacent triangles T is not zero, for example, it is about several degrees. In such a case, in the minute angular difference portion, for example, compared with the size of the aforementioned predetermined angle range of the angle formed by the optical axis O of the imaging unit 3 with respect to the normal direction of the surface of the subject S, the angular difference of the surface of the subject S between adjacent triangles T is small. Further, when there is a minute angular difference portion in the inspection range, a plurality of calculated points are calculated on the assumption that a plurality of triangles T (polygons) arranged in the minute angular difference portion are located on the same plane with respect to each other. In one example, one or more of a plurality of imaging points are calculated and set based on one polygon formed by a plurality of triangles T arranged in the minute angular difference portion. And in one polygon formed by a plurality of triangles T arranged in the minute angular difference portion, the plurality of imaging points are calculated in such a state that any part of the polygon is included in the imaging range at any one or more of the plurality of imaging points.

[0041] In FIG. 5, an example of determining one or more of a plurality of imaging points based on one polygon formed by a plurality of triangles T arranged in the minute angular difference portion will be described. In one example of FIG. 5, four triangles Tb to Te are arranged in the minute angular difference portion, and the difference in the angle formed by the normal direction of the surface of the subject S with respect to each other between the triangles Tb to Te is minute or zero. And a polygon α is formed from the triangles Tb to Te arranged in the minute angular difference portion. Also, in one example of FIG. 5, at least two imaging points Fd and Fe are calculated as a plurality of imaging points F, and the path calculation unit 12 sets the imaging points Fd and Fe based on the polygon α. And the imaging range Rd at the imaging point Fd and the imaging range Re at the imaging point Fe are defined.

[0042] In one example of FIG. 5, any part of the polygon α formed from the triangles Tb to Te is included in one or more of the imaging ranges Rd and Re. Therefore, a plurality of imaging points F are calculated such that any part of the polygon α is imaged by one or more of the imaging points Fd and Fe. Also, in one example of FIG. 5, in calculating the imaging points Fd and Fe, the polygon α is divided into two polygons. Then, the polygon α is divided such that each of the two polygons includes one corresponding imaging point Fd and Fe. Then, in the same manner as the calculation of the imaging points Fa to Fc described above, for each of the imaging points Fd and Fe, the centroid of one corresponding polygon divided from the polygon α is calculated.

[0043] The path calculation unit 12 calculates the position and orientation of the illumination unit 2 and the position and orientation of the imaging unit 3 for each of the plurality of imaging points F (positions on the surface of the subject S) based on the irradiation field of the illumination unit 2 and the imaging irradiation field of the imaging unit 3. That is, the path calculation unit 12 acquires the control information of the actuator of the first robot 41 and the control information of the actuator of the second robot 42 for each of the plurality of imaging points F. Note that the control information of the actuator of the first robot 41 and the control information of the actuator of the second robot 42 include the moving speed, moving timing, etc. in addition to the moving direction of the illumination unit 2 and the imaging unit 3.

[0044] Based on the calculated position and orientation of the illumination unit 2 and the imaging unit 3, etc. at each of the plurality of imaging points F, the path calculation unit 12 calculates the position and orientation of the connection part of the movable body 4 to the illumination unit 2 and the imaging unit 3, and the control amount of the movable body 4. Here, when a robot arm having one or more joints serves as the movable body 4, examples of the control amount of the movable body 4 include the angles of the respective joints. Also, when a linear motion mechanism serves as the movable body 4, an example of the control amount of the movable body 4 is the moving amount due to the operation of the linear motion mechanism.

[0045] When the illumination unit 2 and the imaging unit 3 move, the path calculation unit 12 calculates the irradiation field on the surface of the subject S illuminated by the light beam from the illumination unit 2 and the imaging irradiation field information that can be captured by the imaging unit 3. The irradiation field information can be considered as information for determining the positions and postures of the illumination unit 2 and the imaging unit 3 and the control timings of the first robot 41 and the second robot 42 when performing the optical inspection of the surface of the subject S, which is the optical inspection apparatus 1.

[0046] As described above, in the irradiation field information, the imaging irradiation field of the imaging unit 3 is included in the range of the irradiation field of the subject S by the illumination unit 2. Therefore, a part of the portion of the subject S illuminated by the illumination unit 2 becomes the imaging irradiation field of the imaging unit 3.

[0047] The calculation of the irradiation field information based on the light beam information from the illumination unit 2 is performed, for example, using ray tracing simulation. Ray tracing simulation is a method of theoretically calculating the propagation of light beams geometrically while considering the refraction and reflection of light beams. To perform ray tracing simulation, various commercially available softwares are available. For example, ray tracing simulation can be performed using LightTools (registered trademark) of Synopsys, Inc. In this embodiment, the ray tracing simulation for the light beam of the illumination unit 2 is performed by the processing device 5. The ray tracing simulation for the light beam from the illumination unit 2 is performed by, for example, the path calculation unit 12 in the processing device 5.

[0048] By ray tracing simulation, it is theoretically calculated how the illumination light hits the surface of the subject S from the position and posture of the illumination unit 2. That is, the path calculation unit 12 can theoretically calculate the irradiation field on the surface of the subject S. Also, when the imaging unit 3 captures the imaging points within the irradiation field where the illumination light is applied to the surface of the subject S, the path calculation unit 12 theoretically calculates the position and posture of the imaging unit 3 such that the image contrast becomes the clearest. Therefore, the path calculation unit 12 can theoretically determine in advance, for example, the position and posture of the illumination unit 2 and the position and posture of the imaging unit 3 at a certain time (a certain timing) from the start of the optical inspection.

[0049] Note that, for the imaging points within the irradiation field on the surface of the subject S, the position and orientation of the imaging unit 3 become optimal, for example, when the optical axis O of the imaging unit 3 and the illumination direction of the illumination unit 2 are in a specular reflection relationship with respect to the normal direction of the surface of the subject S. That is, it is the case where the angles formed by the direction of the optical axis O of the imaging unit 3 and the illumination direction are equal with respect to the normal direction of the surface of the subject S. However, the positional relationship between the two is not limited to this.

[0050] The operation control unit 15 of the processing execution unit 6 of the processing device 5 acquires the path for moving the illumination unit 2 and the imaging unit 3 calculated by the path calculation unit 12. Then, the operation control unit 15 controls the operation of the movable body 4 based on the path, and moves the illumination unit 2 and the imaging unit 3 along the path. Further, the operation control unit 15 controls the operation of the movable body 4 to adjust the illumination unit 2 and the imaging unit 3 to the position and orientation calculated by the path calculation unit 12, enabling the acquisition of an imaging image including a desired imaging point. Then, for each imaging point, the illumination unit 2 illuminates the surface of the subject S at the calculated position and orientation, and the imaging unit 3 images the surface of the subject S at the calculated position and orientation.

[0051] The image processing unit 13 of the processing execution unit 6 of the processing device 5 performs image processing on the images captured at each of the plurality of imaging points, and based on the captured images, executes determination and the like regarding defects on the surface of the subject S. When the image processing unit 13 performs defect determination, for example, for a subject that guarantees no defects, the acquired images at each imaging point are stored in the data storage unit 7 as reference images. Then, the image processing unit 13 compares the acquired images at each imaging point on the surface of the subject S with the reference images to determine whether there are defects or the like within a desired range on the surface of the subject S.

[0052] Next, the operation of the optical inspection device 1 according to the present embodiment will be described.

[0053] FIG. 6 shows an example of the processing executed by the processing execution unit 6 of the processing device 5 in the present embodiment. FIGS. 7 and 8 each show a subroutine of a part of the processing in FIG. 6.

[0054] The process of FIG. 6 is executed once before performing an optical inspection on the surface of the subject S. When starting the process of FIG. 6, the data acquisition unit 11 assumes that the subject S is arranged at a predetermined position and in a predetermined orientation in the optical inspection apparatus 1. If the subject S is moving, it is assumed that the position of the subject S at a certain time is determined. The data acquisition unit 11 acquires the CAD data of the subject S through the data storage unit 7 or the user interface 8 in which the CAD data of the subject S is stored in advance (step S101). The data acquisition unit 11 acquires the surface information on the CAD data of the subject S arranged in a desired orientation at the position where the subject S is assumed to be arranged during the optical inspection. That is, the data acquisition unit 11 acquires the shape data of the surface of the subject S represented by a large number of point groups on the surface of the subject S.

[0055] The data acquisition unit 11 acquires the virtual movable range of the illumination unit 2 by the first robot 41 and the virtual movable range of the imaging unit 3 by the second robot 42. The virtual movable range of the illumination unit 2 by the first robot 41 includes the direction of the illumination light of the illumination unit 2. The virtual movable range of the imaging unit 3 by the second robot 42 includes the direction of the optical axis O of the imaging unit 3.

[0056] The path calculation unit 12 appropriately sets the first scan path D1 of the illumination unit 2 with respect to the surface of the subject S. The position and orientation of the illumination unit 2 with respect to the subject S are determined by the first robot 41. The position and orientation of the illumination unit 2 here include the initial position of the illumination unit 2 within a predetermined range of the subject S and the positions when the illumination unit 2 is moved in order. At this time, for each position and orientation of the illumination unit 2 with respect to the subject S, the light ray information to the subject S is determined. That is, the group of light rays incident on the subject S is determined. Taking the CAD data of the subject S and the light ray information from the illumination unit 2 as inputs, the path calculation unit 12 calculates the scan path of the illumination unit 2 so that all desired imaging regions on the surface of the subject S can be illuminated without omission by calculating the irradiation field information of the illumination unit 2 (step S102). The irradiation field information at this time is calculated by ray tracing simulation.

[0057] In step S102, the path calculation unit 12 of the processing device 5 executes processing as shown in, for example, FIG. 7. The path calculation unit 12 acquires the position, orientation, and ray information from the illumination unit 2 (step S1021). Based on the CAD data of the subject S, the position, orientation, and ray information from the illumination unit 2, the path calculation unit 12 theoretically calculates the irradiation field information of the illumination unit 2 on the surface of the inspection target of the subject S (step S1022). Then, the path calculation unit 12 calculates a scan path of the illumination unit 2 that illuminates all desired imaging regions on the surface of the subject S without omission from the irradiation field information of the illumination unit 2 (step S1023). At this time, the path calculation unit 12 can calculate a path for moving the illumination unit 2 or a path for moving the movable body 4 and the illumination unit 2.

[0058] As shown in FIG. 6, similar to the calculation of the irradiation field information, with the CAD data of the subject S and the information of the imaging unit 3 (aperture, field angle, position, orientation) as inputs, the path calculation unit 12 calculates the imaging irradiation field information (information regarding the imaging region) of the imaging unit 3 on the surface of the subject S. In this way, the optical inspection device 1 theoretically predicts the irradiation field information of the illumination unit 2 and the imaging irradiation field information of the imaging unit 3 by the processing device 5.

[0059] In relation to the illumination unit 2, the path calculation unit 12 appropriately sets the second scan path D2 of the imaging unit 3 with respect to the surface of the subject S. The path calculation unit 12 sets the second scan path D2 so as to image at least once in an illuminated state in all desired imaging regions. At this time, the imaging illumination field of the imaging unit 3 varies depending on the surface shape of the subject S. Then, the path calculation unit 12 directs the optical axis O of the imaging unit 3 that moves along the second scan path D2 within the irradiation field of the illumination unit 2 that moves along the first scan path D1, and calculates the paths of the illumination unit 2 and the imaging unit 3 so that the imaging illumination field is included in the irradiation field (so that the imaging illumination field exists) (step S103). That is, the path calculation unit 12 calculates the path of the imaging unit 3 so that the entire desired imaging range on the surface of the subject S is imaged at least once in an illuminated state. Therefore, the path calculation unit 12 calculates the path of the illumination unit 2 and the path of the imaging unit 3 so that the imaging illumination field covers all of the desired imaging regions based on the irradiation field information.

[0060] Also, in step S103, the path calculation unit 12 of the processing device 5 acquires, for example, as shown in FIG. 8, the position and orientation of the imaging unit 3 and the position and shape information of the imaging aperture of the imaging unit 3 (step S1031). The path calculation unit 12 theoretically calculates the imaging illumination field information of the imaging unit 3 on the surface of the inspection target of the subject S based on the irradiation field information of the illumination unit 2, the position and orientation of the imaging unit 3, and the position and shape information of the imaging aperture of the imaging unit 3 (step S1032). Then, the path calculation unit 12 calculates the movement paths of the illumination unit 2 and the imaging unit 3 (step S1033).

[0061] The path calculation unit 12 calculates the positions of the illumination unit 2 and the imaging unit 3 based on the theoretical calculation of the irradiation field information and the imaging illumination field information using the shape data of the subject S so that each imaging point is included in the imaging image. Then, the path calculation unit 12 calculates the irradiation field information using the CAD data, further calculates the imaging illumination field, and determines the imaging point F based on the imaging illumination field information. Once the imaging point F is determined, the exposure timing of the imaging unit 3 can be calculated (step S104).

[0062] For each of the plurality of imaging points F, the path calculation unit 12 optimizes the position and orientation of the imaging unit 3 that images the subject S. For each of the plurality of imaging points, the path calculation unit 12 optimizes the normal direction of the surface of the subject S and the angle between the optical axis O of the imaging unit 3 with respect to the normal direction of the surface of the subject S, and performs position correction so as to obtain a clear and in-focus image. Incidentally, the processing from step S103 to S104 will be further described. When the irradiation field of the illumination unit 2 obtained by ray tracing simulation becomes the first irradiation field IF1, the path calculation unit 12 determines the position and orientation of the first robot 41 that supports the illumination unit 2 and the position and orientation of the second robot 42 that supports the imaging unit 3 so that the first imaging irradiation field PF1 of the imaging unit 3 exists within the range of the first irradiation field IF1. At this time, the position of the imaging aperture of the imaging unit 3 is determined. The imaging aperture is, for example, in a disk shape with a diameter of φ40 mm when the imaging optical element of the imaging unit 3 is a lens having an effective diameter of φ40 mm. At this time, the first imaging irradiation field PF1 of the imaging unit 3 is also calculated by ray tracing simulation. The direction of imaging the subject S (the orientation of the optical axis O of the imaging unit 3) can be optimized by making the normal direction of the surface of the subject S and the angle formed by the optical axis O of the imaging unit 3 with respect to the normal direction of the surface of the subject S fall within a predetermined angular range.

[0063] When the irradiation field information of the illumination unit 2 is a certain second irradiation field IF2 by ray tracing simulation, the path calculation unit 12 determines the position and orientation of the first robot 41 that supports the illumination unit 2 and the position and orientation of the second robot 42 that supports the imaging unit 3 so that the second imaging irradiation field PF2 of the imaging unit 3 exists within the range of the second irradiation field IF2. That is, the imaging irradiation field information is calculated by ray tracing simulation, and the positions and orientations of the illumination unit 2 and the imaging unit 3 are determined. The path calculation unit 12 calculates the paths of the illumination unit 2 and the imaging unit 3 so that all desired regions can be imaged in the imaging at all imaging points F (step S103).

[0064] Incidentally, although the imaging unit 3 moves from the first imaging irradiation field PF1 to the second imaging irradiation field PF2, it is possible that the illumination unit 2 maintains the first irradiation field IF1.

[0065] For example, when a first image is acquired in a first imaging irradiation field PF1, a second image is acquired in a second imaging irradiation field PF2, and these are joined together to obtain a seamless image without gaps, they are called connectable images. To obtain connectable images, it is necessary that there be no gap between the first image of the first imaging irradiation field PF1 of the imaging unit 3 and the second image of the second imaging irradiation field PF2, and a part of the first image and the second image may overlap. That is, the adjacent imaging irradiation field information may have an overlapping part. This also applies to the irradiation field information of the illumination unit 2, and the adjacent irradiation field information may have an overlapping part.

[0066] Then, while illuminating all imaging points F with the light rays of the illumination unit 2, the path calculation unit 12 calculates the paths of the illumination unit 2 and the imaging unit 3 using ray tracing simulation so as to acquire connectable images by imaging with the imaging unit 3.

[0067] To obtain connectable images, in addition to the paths along the first scan path D1 of the illumination unit 2 and the second scan path D2 of the imaging unit 3, the path calculation unit 12 calculates the imaging shutter timing (exposure timing) of the imaging unit 3. That is, it calculates the timing for imaging the imaging point F. In this way, in addition to the paths along the first scan path D1 of the illumination unit 2 and the second scan path D2 of the imaging unit 3, the path calculation unit 12 calculates scan time series information in which the imaging shutter timings of the imaging unit 3 are arranged in time series.

[0068] Therefore, the path calculation unit 12 calculates, for example, the imaging timing (exposure timing) at which the imaging shutter of the imaging unit 3 is released and the image sensor is exposed to acquire a first image. An example of the imaging timing at this time is a state in which the first irradiation field IF1 is illuminated by the illumination unit 2 and the imaging irradiation field information of the imaging unit 3 is the first imaging irradiation field PF1. At this imaging timing, the path calculation unit 12 calculates so as to release the imaging shutter of the imaging unit 3 and expose the image sensor to acquire a first image.

[0069] Similarly, the path calculation unit 12 calculates, for example, the imaging timing at which the imaging shutter is released in the imaging unit 3 and the image sensor is exposed to acquire a second image. An example of the imaging timing at this time is a state in which the second irradiation field IF2 is illuminated by the illumination unit 2 and the imaging irradiation field information of the imaging unit 3 is the second imaging irradiation field PF2. At this imaging timing, the path calculation unit 12 calculates to release the imaging shutter in the imaging unit 3 and expose the image sensor to acquire a second image. At this time, the path calculation unit 12 calculates the imaging shutter timing of the imaging unit 3 in addition to the paths along the first scan path D1 of the illumination unit 2 and the second scan path D2 of the imaging unit 3.

[0070] Note that the size of the first imaging irradiation field PF1 of the first image and the size of the second imaging irradiation field PF2 of the second image may be the same as or different from each other.

[0071] Therefore, based on the irradiation field information of the illumination unit 2 and the imaging irradiation field information of the imaging unit 3, the path calculation unit 12 determines, before actually performing the optical inspection of the subject S, the scan path (the path along which the illumination unit 2 and the imaging unit 3 move) and the imaging timing for acquiring a connectable overall image of the surface of the subject S (step S104).

[0072] Next, the operation control unit 15 of the optical inspection apparatus 1 moves the illumination unit 2, the imaging unit 3, and the movable body 4 (the first robot 41 and the second robot 42) with respect to the surface of the actual subject S to perform an optical inspection of the subject S. The operation control unit 15 moves the illumination unit 2, the imaging unit 3, and the movable body 4 along the paths calculated by the path calculation unit 12, respectively. The operation control unit 15 controls and moves the first robot 41 and the second robot 42, respectively, sets a plurality of imaging points F by the path calculation unit 12, and images the images including each imaging point F at the imaging timing on the paths calculated by the path calculation unit 12 to acquire a connected image.

[0073] In this way, the operation control unit 15 controls the operation of the movable body 4 (step S105) based on the path for sequentially moving the illumination unit 2 and the imaging unit 3 so as to acquire an image including a plurality of imaging points F, and the calculation result of the path calculation unit 12 regarding the imaging shutter timing of the imaging unit 3. Thereby, the operation control unit 15 moves the illumination unit 2 and the imaging unit 3 along the path calculated by the path calculation unit 12, and adjusts the positions and postures of the illumination unit 2 and the imaging unit 3 when imaging an image including the imaging point F. Further, in a state where the illumination unit 2 and the imaging unit 3 are moving along the calculated path, the operation control unit 15 images each imaging point F on the surface of the subject S illuminated by the illumination unit 2 with the imaging unit 3.

[0074] Then, the image processing unit 13 images an image including a plurality of imaging points F and performs image processing on the image (step S106).

[0075] The image processing unit 13 determines defects and the like on the surface of the subject S based on an image or the like in which the imaging point F is imaged (step S107). For example, the image processing unit 13 previously acquires an image of a standard surface without scratches or the like on the surface of the subject S and stores it, for example, in the data storage unit 7. Then, the presence or absence of defects and the like is determined by comparing the image of the standard surface with the captured image.

[0076] In the present embodiment, the processing device 5 calculates irradiation field information of the illumination unit 2 by ray tracing simulation based on CAD data of the surface of the subject S, calculates imaging irradiation field information that can be imaged by the imaging unit 3 by ray tracing simulation, and can preset the scan path and imaging timing of the illumination unit 2 and the imaging unit 3 so as to acquire an image including each imaging point F. In this way, when optically inspecting the surface of the subject S with the optical inspection device 1 based on the path and imaging timing calculated by the processing device 5, the processing device 5 can acquire a connected image including a desired entire inspection area.

[0077] According to the optical inspection apparatus 1 according to the present embodiment, for example, when irradiating the surface of the subject S with a light beam by a lighting fixture on the ceiling, the light beam is blocked by the first robot 41 or the second robot 42, and it is possible to image a position (imaging point) on the surface of the subject S where a shadow is formed and imaging becomes difficult. That is, it is possible to prevent a region that cannot be acquired as an image unintentionally from occurring on the surface of the subject S.

[0078] By using the optical inspection of the optical inspection apparatus 1 according to the present embodiment, an image of the surface of the subject S can be acquired under the same imaging conditions for the subject S. Therefore, the optical inspection apparatus 1 according to the present embodiment can perform optical inspection with the same quality for a plurality of subjects S without considering the influence of, for example, the arrangement of robots that can create a shadow on the surface of the subject S.

[0079] By using the optical inspection of the optical inspection apparatus 1 according to the present embodiment, it is not necessary to determine the scan paths of the imaging unit 3 and the illumination unit 2 by actual machine shooting using the actual subject S, and there is an advantage that the scan paths can be set in advance from the CAD information of the subject S.

[0080] In addition, if the irradiation field information and the imaging irradiation field information cannot be theoretically predicted in advance as described above, it can be said that the scan time series information for acquiring a connectable overall image of the surface of the subject S cannot be determined in advance. That is, only by determining the irradiation field information and the imaging irradiation field information by the optical inspection apparatus 1 according to the present embodiment, it is possible to determine the scan time series information for acquiring a non-flickering, connectable overall image.

[0081] The optical inspection method (optical inspection program) according to this embodiment calculates irradiation field information regarding the irradiation field on the surface of the object S when irradiating the surface of the object S with a light beam from the illumination unit 2 that is supported by the movable body 4 and moves, and based on the irradiation field information, performs a path calculation process for calculating the path along which the illumination unit 2 moves, or the path along which the movable body 4 and the illumination unit 2 move. By this method (program), as the path along which the illumination unit 2 moves, or the path along which the movable body 4 and the illumination unit 2 move, it is possible to calculate so as to irradiate the desired range on the surface of the object S, for example, at least once. Then, by performing imaging at the imaging unit 3 of the positions where illumination is applied in a state where illumination is applied to the imaging points set in the desired range on the surface of the object S by this method, for example, an appearance inspection of the surface of the three-dimensional object S can be performed. When imaging the irradiation field on the surface of the object S with the imaging unit 3, the positional relationship between the illumination unit 2 and the imaging unit 3 can be optimized, and a good image can be obtained. In order to properly move and illuminate the illumination unit 2, each image can prevent, for example, a part of the image from being missing due to the influence of the shadow of the robot or the like.

[0082] In the optical inspection method, the irradiation field information includes information regarding the imaging irradiation field where the surface of the object S can be imaged by the imaging unit 3 that is supported by the movable body 4 and moves by the light beam reflected from the surface of the object S by the light beam from the moving illumination unit 2. And calculating the path by the path calculation unit includes calculating the paths of the movable body 4, the illumination unit 2, and the imaging unit 3 based on the irradiation field information. Therefore, by calculating the paths of the illumination unit 2 and the imaging unit 3 based on the irradiation field information, the imaging unit 3 can image the desired imaging points on the surface of the object S. Therefore, using the optical inspection method, for example, an appearance inspection of the surface of the three-dimensional object S can be performed. At this time, since the imaging unit 3 images the range within which illumination is applied by the illumination unit 2, a good image can be obtained in a state where good light is applied to the surface of the object S for imaging, and the inspection accuracy can be improved.

[0083] Calculating the path by the path calculation unit includes generating the timing of exposure for imaging an image by the imaging unit 3. Further, generating the timing of exposure includes operating at least one of the shutter of the imaging unit 3 and the ON / OFF of the illumination unit 2. Therefore, various methods can be used for the imaging unit 3 to acquire an image.

[0084] The optical inspection method includes acquiring the shape data of the surface of the object S represented by a large number of point groups on the surface of the object S, and determining the imaging points of the imaging unit 3 based on the shape data of the surface of the object S. By setting a path for moving the movable body 4, the illumination unit 2, and the imaging unit 3 so as to connect images including the imaging points, it is possible to acquire images in a predetermined range, for example, the entire range. Therefore, by comparing the image obtained by the optical inspection at each imaging point with, for example, the image of a standard surface, the result of the optical inspection of the surface of the object S can be obtained.

[0085] In the optical inspection method, calculating the path includes maintaining a state in which the imaging region of the imaging unit 3 is included in the irradiation field of the illumination unit 2 regardless of the position and orientation of the movable body 4. By maintaining such a relationship between the irradiation field and the imaging region, an image of the imaging point on the surface of the object S can be obtained in the illuminated state.

[0086] Incidentally, the distance between the imaging aperture of the imaging unit 3 and an arbitrary imaging point F on the surface of the subject S is calculated by the path calculation unit 12 of the processing device 5. In some cases, it may be preferable that the distance between the imaging aperture of the imaging unit 3 and the imaging point F on the surface of the subject S is an equal distance. By maintaining the distance between the imaging aperture of the imaging unit 3 and the surface of the subject S at a predetermined equal distance, the imaging distance of the imaging unit 3 can be maintained in a predetermined state. That is, it is possible to always keep the focus. In this way, by calculating the path so that the distances between the imaging aperture and each imaging point F are equal distances by the path calculation unit 12, the processing device 5 can move the second robot 42 while maintaining the distance between the imaging aperture of the imaging unit 3 and a certain imaging point F on the surface of the subject S by the operation control unit 15. In this way, by always ensuring the imaging distance at which the imaging unit 3 can acquire a clear image, the optical inspection device 1 can obtain a clear image.

[0087] The path of the imaging unit 3 by the path calculation unit 12 is preferably set, for example, to pass through all of a plurality of imaging points and to minimize the total movement cost from each of the plurality of imaging points to the next imaging point to be the movement destination. Since there are various methods for calculating the path that minimizes the total movement cost by the path calculation unit 12 using the processing device (processor) 5, the description here is omitted.

[0088] The path calculated by the path calculation unit 12 preferably does not pass through the same imaging point a plurality of times, for example, so as to keep the movement costs of the first robot 41 and the second robot 42 low. The path calculation unit 12 may pass the optical axis O of the imaging unit 3 through the same imaging point a plurality of times if necessary.

[0089] As described above, according to the present embodiment, it is possible to provide an optical inspection method, an optical inspection program, a processing device 5, and an optical inspection device 1 that can optically inspect the surface of the subject S without overlooking (image missing) a desired range.

[0090] (Modification example) The optical inspection apparatus 1 according to a modification of the first embodiment will be described with reference to FIG. 9.

[0091] In this modification, the movable body 4 has a base 40 and one robot (for example, the first robot 41) supported by the base 40. The illumination unit 2 and the imaging unit 3 are connected by a connecting portion 9. Therefore, the illumination unit 2 and the imaging unit 3 move integrally by the movable body 4. For example, when the imaging unit 3 is moved toward the second scan path D2, the illumination unit 2 moves in conjunction.

[0092] Note that, as described in the first embodiment, the processing device 5 acquires irradiation field information by ray tracing simulation based on the movable range of the movable body 4, and calculates the paths of the illumination unit 2 and the imaging unit 3. When acquiring the irradiation field information by ray tracing simulation, the positional relationship between the illumination unit 2 and the imaging unit 3 is reflected.

[0093] With the optical inspection apparatus 1 according to this modification, compared with the optical inspection apparatus 1 according to the first embodiment, the number of robots for moving the illumination unit 2 and the imaging unit 3 can be reduced.

[0094] According to this modification, an optical inspection method (optical inspection program) can be executed along the flowcharts shown in FIGS. 6 to 8. Therefore, according to this modification, an optical inspection method, an optical inspection program, a processing device 5, and an optical inspection apparatus 1 capable of optically inspecting a surface of a specimen S without overlooking (image missing) a desired range can be provided.

[0095] (Second Embodiment) The optical inspection apparatus 1 according to the second embodiment will be described with reference to FIG. 10. This embodiment is a further modification of the first embodiment including the modification, and the same members as those described in the first embodiment and members having the same functions are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0096] The optical inspection device 1 shown in FIG. 10 omits the illustration of the movable body 4 and the processing device 5. The illumination unit 2 and the imaging unit 3 of the optical inspection device 1 according to the present embodiment maintain a predetermined positional relationship. The movable body 4 moves the illumination unit 2 and the imaging unit 3 integrally.

[0097] The imaging unit 3 includes a lens 31 and an image sensor 32. The image sensor 32 is disposed on the optical axis O of the lens 31. Then, a light beam B1 is imaged on the image sensor 32 by the lens 31. The imaging aperture is the effective area of the lens 31. That is, the shape of the imaging aperture is a disk, and the diameter of the imaging aperture is the effective diameter of the lens 31.

[0098] The illumination unit 2 has, for example, a beam splitter 21 disposed between the lens 31 and the image sensor 32. Note that the position of the beam splitter 21 may be between the lens 31 and the surface of the subject S.

[0099] The light beam B from the illumination unit 2 is irradiated onto the surface of the subject S along the optical axis O through a beam splitter 21 disposed, for example, between the lens 31 and the image sensor 32. That is, the beam splitter 21 changes the direction of the light beam B from the illumination unit 2 so as to irradiate the surface of the subject S along the optical axis O. Such an illumination method is called a coaxial epi-illumination method.

[0100] The direction of the light beam B from the illumination unit 2 is changed by the beam splitter 21 and is irradiated onto the surface of the subject S through the lens 31. A part of the light beam B irradiated onto the surface of the subject S is reflected on the surface of the subject S. The intensity direction distribution of the light beam B1 reflected on the surface of the subject S can be represented by BRDF (Bidirectional Reflectance Distribution Function). BRDF is a quantity that depends on the direction of the normal N of the surface of the subject S and the illumination direction (incident direction on the object) L of the illumination. The imaging illumination field PF of the imaging unit 3 can be theoretically calculated based on geometric optics if BRDF is obtained. Thus, calculating the imaging illumination field PF using BRDF is also a kind of ray tracing simulation.

[0101] By using coaxial epi-illumination, the illumination unit 2 and the imaging unit 3 can be integrated and made compact.

[0102] In this embodiment, regardless of the positions of the illumination unit 2 and the imaging unit 3, the state where the imaging region of the imaging unit 3 is within the range of the irradiation field IF of the illumination unit 2 can be maintained. The path calculation unit 12 calculates a path, for example, such that the optical axis O of the imaging unit 3 passes through each imaging point F. In this case, while facilitating the control of the movable body 4, an image illuminated with illumination light can be acquired at each imaging point F.

[0103] According to this embodiment, the light beam B from the illumination unit 2 and the optical axis O of the imaging unit 3 are coaxial. And the light beam B from the illumination unit 2 and the normal line N at a certain imaging point F on the surface of the subject S can be made coaxial. For this reason, by fixing the positional relationship between the illumination unit 2 and the imaging unit 3 with respect to the movable body 4 (one robot 41), the processing device 5 can calculate the irradiation field IF and the imaging irradiation field PF based on the light beam information from the illumination unit 2 and the direction of the normal line N of each imaging point F of the surface information (CAD data) of the subject S. Then, the processing device 5 determines the scan time-series information for acquiring the entire connectable image of the surface of the subject S based on the calculated information of the irradiation field IF and the imaging irradiation field PF.

[0104] Here, when moving the first robot 41 and the second robot 42 described in the first embodiment, control is required to place the entire imaging region of the imaging unit 3 within the irradiation field of the illumination unit 2 while preferably maintaining a predetermined distance with respect to the surface of the subject S. Further, when the imaging unit 3 is fixed to the first robot 41 described in the modified example of the first embodiment and the illumination unit 2 is connected to the imaging unit 3, control is required to place the entire imaging region of the imaging unit 3 within the irradiation field of the illumination unit 2 while preferably maintaining a predetermined distance with respect to the surface of the subject S and to separate the illumination unit 2 from the subject S.

[0105] On the other hand, in the case of this embodiment, the entire imaging area of the imaging unit 3 can be included in the irradiation field IF of the light beam B of the illumination unit 2. Therefore, according to this embodiment, the imaging unit 3 can easily image the imaging irradiation field PF including the desired imaging point F in a state where the illumination unit 2 irradiates the desired imaging point F and its surroundings as the irradiation field IF. Further, by aligning the optical axis of the imaging unit 3 with the normal direction of the imaging point F, the focus around the imaging point F can be adjusted. Therefore, by using the optical inspection apparatus 1 according to this embodiment, a good image for determining the presence or absence of unevenness such as scratches on the surface of the object S can be obtained.

[0106] According to this embodiment, an optical inspection method (optical inspection program) can be executed along the flowcharts shown in FIGS. 6 to 8. Therefore, according to this embodiment, it is possible to provide an optical inspection method, an optical inspection program, a processing device 5, and an optical inspection apparatus 1 that can optically inspect the surface of the object S without overlooking a desired range (image missing).

[0107] (First Modification Example) The first modification example of the second embodiment will be described with reference to FIG. 11. FIG. 11 shows an optical inspection apparatus 1 according to the first modification example of the second embodiment.

[0108] The imaging unit 3 of the optical inspection apparatus 1 according to this modification further includes a multi-wavelength aperture 33. The multi-wavelength aperture 33 has a plurality of wavelength selection regions 33a, 33b, 33c arranged in parallel. When the light beam B1 reflected from the object S passes through the wavelength selection regions 33a, 33b, 33c, it becomes a light beam having a specific wavelength region according to the passed wavelength selection regions 33a, 33b, 33c. For example, assume that the multi-wavelength aperture 33 includes three wavelength selection regions 33a, 33b, 33c. When the white light beam B1 passes through the three wavelength selection regions 33a, 33b, 33c, assume that the respective wavelength regions are 450 nm to 500 nm, 500 nm to 600 nm, and 600 nm to 750 nm. That is, the white light becomes blue light, green light, and red light respectively by the three wavelength selection regions 33a, 33b, 33c of the multi-wavelength aperture 33. However, these wavelengths are not limited to this and can be appropriately set by the selection of the wavelength selection region.

[0109] The light beam B from the illumination unit 2 is irradiated onto the surface of the specimen S along the optical axis O via the beam splitter 21. As described above, the intensity direction distribution of the light beam reflected by the specimen S can be represented by BRDF (Bidirectional Reflectance Distribution Function). BRDF is a quantity that depends on the direction of the normal N of the surface of the specimen S and the illumination direction (the incident direction to the object) L of the illumination. The imaging irradiation field PF can be theoretically calculated based on geometric optics if the BRDF is obtained. Thus, calculating the imaging irradiation field PF using BRDF is also a kind of ray tracing simulation.

[0110] Also, if the BRDF is obtained, it is possible to know through which wavelength selection regions 33a, 33b, 33c of the multi-wavelength aperture 33 the light beam B1 passes. Therefore, the processing device 5 can calculate the color distribution of the captured image captured by the image sensor 32.

[0111] The BRDF varies significantly depending on the presence or absence of minute irregularities on the surface of the subject S. As a result, the color distribution of the captured image varies significantly depending on the presence or absence of minute irregularities on the surface of the subject S. Assume that the color distribution of the captured image with respect to the surface of the subject S when there are no minute defects is calculated theoretically in advance as a reference (standard surface). By using this reference, minute defects can be identified by taking the difference from the reference of the color distribution of the captured image. Note that, as the reference, an image actually acquired by the image sensor 32 may be used.

[0112] In the case of this modification, the illumination light of the illumination unit 2 is incident on the multi-wavelength aperture 33 through the beam splitter 21 and the lens 31. For this reason, light having a predetermined wavelength that has passed through the multi-wavelength aperture 33 is incident on a predetermined range of the subject S.

[0113] According to this modification, an optical inspection method (optical inspection program) can be executed along the flowcharts shown in FIGS. 6 to 8. Here, the optical inspection method includes acquiring an image through the imaging unit 3 provided with the multi-wavelength aperture 33 that becomes the light beam B1 having a specific wavelength spectrum when light rays pass through the respective regions 33a, 33b, and 33c. For this reason, for example, it is possible to easily inspect whether the surface of the subject S is normal by image processing.

[0114] And according to this modification, it is possible to provide an optical inspection method, an optical inspection program, a processing device 5, and an optical inspection device 1 that can perform optical inspection of the surface of the subject S without overlooking (image missing) a desired range.

[0115] (Second Modification) The second modification of the second embodiment will be described with reference to FIG. 12. FIG. 12 shows an optical inspection device 1 according to the second modification of the second embodiment. This modification is a further modification of the first modification of the second embodiment, and the same members as those described in the first modification or members having the same functions are denoted by the same reference numerals, and detailed description thereof is omitted.

[0116] In the optical inspection device 1 according to the first modification of the second embodiment shown in FIG. 11, an example in which the multi-wavelength aperture 33 is arranged immediately before the lens 31 has been described. That is, the multi-wavelength aperture 33 is arranged between the lens 31 and the surface of the specimen S. The arrangement of the multi-wavelength aperture 33 is not limited to this, and the multi-wavelength aperture 33 may be arranged at any position with respect to the lens 31. For example, as in the optical inspection device 1 according to this modification shown in FIG. 12, the multi-wavelength aperture 33 may be arranged on the focal plane of the lens 31. In this case, when the imaging unit 3 forms an image from the object point on the surface of the specimen S to the image point on the image sensor, the relationship between the light ray direction and the color of the light ray can be made constant regardless of the object point (see, for example, Non-Patent Document 2).

[0117] Note that, for example, the position of the illumination unit 2 may be shifted to a position closer to the surface of the specimen S, and the beam splitter 21 may be arranged between the multi-wavelength aperture 33 and the object S. Then, the illumination direction L of the illumination of the optical inspection device 1 is set to be the same as the state shown in FIG. 11. Even if the optical inspection device 1 according to this modification shown in FIG. 12 is formed in this way, optical inspection can be performed in the same manner as the optical inspection device 1 shown in FIG. 11.

[0118] According to at least one of the embodiments described above, it is possible to provide an optical inspection method, an optical inspection program, a processing device 5, and an optical inspection device 1 that can perform optical inspection of the surface of the specimen S without overlooking (image missing) a desired range.

[0119] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and its equivalent scope. The claims at the time of patent application of this application are appended below. [Appendix 1] Calculating irradiation field information regarding an irradiation field on the surface of the subject when irradiating the surface of the subject with a light beam from an illumination device supported and moved by a movable body; Performing a path calculation process for calculating a path along which the illumination device moves based on the irradiation field information; An optical inspection method comprising the above. [Appendix 2] The irradiation field information includes information regarding an imaging irradiation field where the surface of the subject can be imaged by a moving imaging device, The path calculation process includes calculating the moving path of the illumination device supported by the movable body and the moving path of the imaging device based on the irradiation field information. The optical inspection method according to Appendix 1. [Appendix 3] The imaging device is supported by the movable body. The optical inspection method according to Appendix 2. [Appendix 4] The path calculation process includes calculating the timing of exposure when imaging an image with the imaging device when calculating the paths of the illumination device and the imaging device. The optical inspection method according to Appendix 2 or Appendix 3. [Appendix 5] Calculating the timing of the exposure includes calculating the timing of operating at least one of the shutter of the imaging device and the ON / OFF of the illumination device. The optical inspection method according to Appendix 4. [Appendix 6] Obtaining shape data of the surface of the subject represented by a large number of point groups on the surface of the subject; Calculating imaging points of the imaging device based on the shape data The optical inspection method according to any one of Appendices 2 to 5. [Appendix 7] The path calculation process includes calculating the position and orientation of the movable body so as to maintain a state where the imaging irradiation field of the imaging device exists within the irradiation field of the illumination device when calculating the paths of the illumination device and the imaging device. The optical inspection method according to any one of Appendices 2 to 6. [Appendix 8] Obtaining an imaging image by the imaging device provided with a multi-wavelength aperture having a plurality of wavelength regions and the light beams passing through each wavelength region having different wavelength spectra. The optical inspection method according to any one of Appendices 2 to 7. [Appendix 9] A program to be executed by a computer, the program causing the computer to calculate irradiation field information regarding an irradiation field on the surface of a subject when irradiating the surface of the subject with a light beam from an illumination device supported by a movable body and moving, An optical inspection program that causes the computer to calculate a path for moving the illumination device based on the irradiation field information. [Appendix 10] A processing device related to optical inspection of the surface of a subject using a movable body and an illumination device supported by the movable body, Calculating irradiation field information regarding an irradiation field on the surface of the subject when irradiating the surface of the subject with a light beam from the illumination device supported by the movable body and moving, Calculating a path for moving the illumination device based on the irradiation field information, A processing device including a processor. [Appendix 11] The processing device according to Appendix 10, The movable body controlled by the processing device, The illumination device supported by the movable body, irradiating illumination light under the control of the processing device, and the irradiation field information on the surface of the subject being calculated by the processing device, An optical inspection device comprising the above. [Appendix 12] An imaging device that images with light rays reflected from the surface of the subject after irradiating the surface of the subject with the illumination device, The irradiation field information includes information regarding an imaging irradiation field such that light rays reflected from the surface of the subject after irradiating the surface of the subject with the illumination device enter the imaging device. The optical inspection device according to Appendix 11. [Appendix 13] The imaging device according to Appendix 12, comprising a multi-wavelength aperture having a plurality of wavelength regions, and light rays passing through each wavelength region having different wavelength spectra. [Appendix 14] The optical inspection device according to Appendix 12 or Appendix 13, wherein the movable body moves the illumination device and the imaging device integrally.

Explanation of Symbols

[0120] 1…Optical inspection device, 2…Illumination unit, 3…Imaging unit, 4…Movable body, 5…Processing device, 6…Processing execution unit, 7…Data storage unit, 8…User interface, 9…Connection unit, 11…Data acquisition unit, 12…Path calculation unit, 13…Image processing unit, 15…Operation control unit, 21…Beam splitter, 31…Imaging aperture (lens), 32…Image sensor, 33…Multi-wavelength aperture, 33a…Wavelength selection region, 33b…Wavelength selection region, 33c…Wavelength selection region, 40…Base, 41…First robot, 42…Second robot, S…Specimen, B, B1…Light rays, D1, D2…Scanning paths, Fa~Fc…Imaging points, IF1, IF2…Irradiation fields, PF1, PF2…Imaging irradiation fields, Ra~Rc…Imaging ranges, Tb~Te…Triangles.

Claims

1. When irradiating the surface of a subject with a light beam from an illuminating device supported by a movable body and moving, irradiation field information regarding the irradiation field on the surface of the subject, and imaging irradiation field information regarding an imaging irradiation field capable of imaging the surface of the subject by an imaging device that moves to adjust the position and orientation so that at least a part of the surface of the subject within the angular range of view is orthogonal or substantially orthogonal to the optical axis, Performing a path calculation process for calculating a path along which the illuminating device and the imaging device move so that, when imaging an imaging point within the imaging irradiation field included in the irradiation field where illumination light is applied to the surface of the subject based on the irradiation field information and the imaging irradiation field information, the position and orientation of the imaging device are such that the image contrast becomes clear, An optical inspection method comprising the above.

2. Performing the path calculation process includes calculating a path along which the illuminating device and the imaging device move so that, when light rays from a lighting fixture on the ceiling to the surface of the subject are blocked by at least one of the illuminating device and the imaging device, a position where a shadow is formed on the surface of the subject and imaging becomes difficult can be imaged, The optical inspection method according to Claim 1.

3. Including calculating the irradiation field information and the imaging irradiation field information in order, The optical inspection method according to Claim 1 or Claim 2.

4. The path calculation process includes calculating the timing of exposure when imaging an image with the imaging device when calculating the paths of the illuminating device and the imaging device, The optical inspection method according to any one of Claims 1 to 3.

5. Calculating the timing of exposure includes calculating the timing of operating at least one of the shutter of the imaging device and the ON / OFF of the illuminating device, The optical inspection method according to Claim 4.

6. Obtaining shape data of the surface of the subject represented by a large number of point groups on the surface of the subject, Calculating the imaging points based on the shape data Including The optical inspection method according to any one of Claims 1 to 5.

7. The path calculation process includes calculating the position and orientation of the movable body so as to maintain a state where the imaging irradiation field of the imaging device exists within the irradiation field of the illuminating device when calculating the paths of the illuminating device and the imaging device, The optical inspection method according to any one of claims 1 to 6.

8. Obtaining an imaging image by the imaging device provided with a multi-wavelength aperture having a plurality of wavelength regions and the light rays passing through each wavelength region being light rays having different wavelength spectra from each other. The optical inspection method according to any one of claims 1 to 7.

9. A program to be executed by a computer, the program being for irradiating the surface of a subject with a light ray from an illumination device supported by a movable body and moving, irradiation field information regarding the irradiation field on the surface of the subject, and the imaging device moving to adjust the position and posture so that at least a part of the surface of the subject within the angle-of-view range is in a posture orthogonal or substantially orthogonal to the optical axis, causing the computer to calculate imaging irradiation field information regarding the imaging irradiation field where the surface of the subject can be imaged. Based on the irradiation field information and the imaging irradiation field information, when imaging an imaging point within the imaging irradiation field included in the irradiation field where illumination light is applied to the surface of the subject with the imaging device, causing the computer to calculate a path for moving the illumination device and the imaging device so that the position and posture of the imaging device are such that the image contrast becomes clear. Optical inspection program.

10. Causing the computer to calculate the path includes causing the computer to calculate a path for the illumination device and the imaging device to move so that when the light ray from the lighting fixture on the ceiling to the surface of the subject is blocked by at least one of the illumination device and the imaging device, the position where a shadow is formed on the surface of the subject and imaging becomes difficult can be imaged. The optical inspection program according to claim 9.

11. Calculating irradiation field information regarding the irradiation field on the surface of the subject when irradiating the surface of the subject with a light ray from an illumination device supported by a movable body and moving, and imaging irradiation field information regarding the imaging irradiation field where the surface of the subject can be imaged by an imaging device moving to adjust the position and posture so that at least a part of the surface of the subject within the angle-of-view range is in a posture orthogonal or substantially orthogonal to the optical axis. When imaging an imaging point within the imaging field of view that is included in the illumination field of view in which illumination light is applied to the surface of the subject based on the illumination field of view information and the imaging illumination field of view information, a path for moving the illumination device and the imaging device is calculated so that the position and orientation of the imaging device are such that the image contrast becomes clear. A processing device including a processor.

12. Based on the path, the processor moves the movable body, the illumination device, and the imaging device. The processing device according to claim 11.

13. When light rays from a lighting fixture on the ceiling to the surface of the subject are blocked by at least one of the illumination device and the imaging device, the processor calculates a path for the illumination device and the imaging device to move so that positions where shadows are formed on the surface of the subject and imaging becomes difficult can be imaged. The processing device according to claim 11 or claim 12.

14. The processing device according to any one of claims 11 to 13, the movable body controlled by the processing device, the illumination device supported by the movable body, irradiating illumination light under the control of the processing device, and the illumination field of view information on the surface of the subject being calculated by the processing device, and the imaging device that moves to adjust its position and orientation so that at least a part of the surface of the subject within the angle-of-view range is orthogonal or substantially orthogonal to the optical axis. An optical inspection device comprising:

15. After irradiating the surface of the subject with the illumination device, the imaging device images with light rays reflected from the surface of the subject. The optical inspection device according to claim 14.

16. The imaging device includes a multi-wavelength aperture having a plurality of wavelength regions, and light rays passing through each wavelength region become light rays having different wavelength spectra. The optical inspection device according to claim 14 or claim 15.

17. The movable body moves the illumination device and the imaging device integrally. The optical inspection device according to any one of claims 14 to 16.

Images