Processing apparatus, optical inspection system, subject surface image composition method, defect identification image composition method, subject surface image composition program, subject surface angle estimation method, and subject surface inclination angle estimation program

Abstract

To provide a processing apparatus that performs processing to grasp a state of a subject surface having various shapes.SOLUTION: A processing apparatus used for a subject surface image composition includes one or more processors. The one or more processors, with respect to a plurality of captured images that an imaging portion has obtained while a surface of a subject moving relative to illumination light is irradiated with beams of the illumination light traveling in different directions in at least one plane and that includes a reference captured image in a plurality of times, shifts one or more captured images selected from the plurality of captured images and superimposes the shifted captured images on the reference captured image to compose at least one subject surface image.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The embodiments of the present invention relate to a processing device, an optical inspection system, a method for constructing an object surface image, a method for constructing a defect identification image, a program for constructing an object surface image, a method for estimating the angle of an object surface, and a program for estimating the inclination angle of an object surface. [Background technology]

[0002] In various industries, it is important to inspect the surface of an object in a non-contact manner while it is being transported. Conventional methods include a method of inspecting the surface of an object by irradiating the object with illumination light while it is being transported, capturing an image of the light reflected from the object surface using an imaging element, and analyzing the captured image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6391281 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem that the present invention aims to solve is to provide a processing device that performs processing to grasp the condition of a subject's surface having various shapes or is used to construct a defect identification image of the subject's surface, an optical inspection system having a processing device, a method for constructing a subject's surface image, a method for constructing a defect identification image, a program for constructing a subject's surface image, a method for estimating the angle of the subject's surface, and a program for estimating the inclination angle of the subject's surface. [Means for solving the problem]

[0005] According to an embodiment, a processing device used to construct a surface image of a subject includes one or more processors, which shift one or more captured images selected from a plurality of captured images at a plurality of times including a reference captured image obtained by an imaging unit when illuminating a surface of a subject moving relatively to the illuminating light with illumination light directed in different directions in at least one plane, and superimpose the captured image on the reference captured image to construct at least one surface image of the subject. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a schematic diagram showing an optical inspection system according to a first embodiment. [Diagram 2] FIG. 2 is a schematic diagram of the illumination section shown in FIG. 1 along the ZX plane. [Diagram 3] 11 is a schematic diagram showing an example in which the surface of a subject transported on a transport section is imaged by an imaging section using an optical inspection device of a comparative example. [Figure 4] Images of the surface of a subject taken at an appropriate frame rate using the optical inspection device shown in Figure 3 are arranged. [Diagram 5] 1 is a schematic diagram showing a flat-plate-shaped subject transported by a transport section of the optical inspection system according to the first embodiment, an optical inspection device that optically acquires an image of the subject, and illumination light onto the surface of the subject in the ZX plane. [Figure 6] 6 is a schematic diagram showing an object, an optical inspection device, and illumination light onto the surface of the object in the YZ plane, as viewed from the direction indicated by the symbol VI in FIG. 5. [Figure 7] 5. FIG. 7 is a schematic diagram different from FIG. 6, showing the subject, the optical inspection device, and illumination light onto the surface of the subject in the YZ plane, as viewed from the direction indicated by the symbol VI in FIG. [Figure 8] 7 is a schematic diagram of images I1, I2, . . . , In that can be acquired at appropriate time intervals by the optical inspection device shown in FIGS. 5 and 6. [Figure 9] A surface constructed image of a single subject, constructed based on the image shown in Figure 8. [Figure 10]5 is a flowchart for constructing a surface configuration image of a single subject using the optical inspection system according to the first embodiment. [Figure 11] 11 is a schematic diagram showing a state in which images I1, I2, . . . , In that can be acquired at appropriate time intervals are appropriately shifted and arranged in the flowchart shown in FIG. 10. [Figure 12] Images of the surface of an actual plastic flat plate, which had been scratched using the optical inspection system according to the first embodiment, are arranged in chronological order. [Figure 13] A surface constructed image of a single subject constructed based on the image shown in FIG. [Figure 14] The flowchart shown in Fig. 10 is supplemented with a step for constructing a defect identification image. [Figure 15] A defect identification image of a single subject constructed based on the image shown in FIG. [Figure 16] 6 is a schematic diagram showing an example in which the surface of the subject is an inclined plane that is inclined with respect to the XY plane in the optical inspection device shown in FIG. 5. [Figure 17] 17 is a schematic diagram of images I1, I2, . . . , In that can be acquired at appropriate time intervals by the optical inspection device shown in FIG. 16 and FIG. 6. [Figure 18] A surface constructed image of a single subject constructed based on the image shown in FIG. [Figure 19] 6 is a schematic diagram showing an example in which the surface of the subject is a continuous plane parallel to the XY plane and an inclined plane inclined with respect to the XY plane in the optical inspection device shown in FIG. 5. [Figure 20] 19 and 6. FIG. 20 is a schematic diagram of images I1, I2, . . . , In that can be acquired at appropriate time intervals by the optical inspection device shown in FIG. [Figure 21] A surface constructed image of a single subject constructed based on the image shown in FIG. [Figure 22] 11 is a flowchart for constructing a surface configuration image of one subject using an optical inspection system according to a modified example of the first embodiment. [Figure 23] FIG. 11 is a schematic diagram showing an optical inspection system according to a second embodiment. [Figure 24] 24 is a schematic diagram showing the optical inspection device shown in FIG. 23 and the relationship between the surface of an object and reflected light in the optical inspection device. [Diagram 25] The surface of a subject, a plastic flat plate with scratches on its surface, was photographed at an appropriate frame rate using the optical inspection system according to the second embodiment, and images were then arranged. [Figure 26] A surface constructed image of a single subject constructed based on the image shown in FIG. [Figure 27] A defect identification image of a single subject constructed based on the image shown in FIG. [Figure 28] FIG. 13 is a schematic diagram showing an example of constructing a defect identification image of one subject without going through the construction of a surface structure image of the subject in an optical inspection system according to a modified example of the second embodiment. [Figure 29] 29 is a flow chart of a processing device for constructing a defect identification image of one object as shown in FIG. 28. [Diagram 30] FIG. 11 is a schematic diagram showing an optical inspection system according to a third embodiment. [Diagram 31] 31 is a schematic diagram of an optical inspection device of the optical inspection system shown in FIG. 30. [Diagram 32] 31 is a flow chart of a processing device for estimating the inclination angle of a surface of an object based on an image acquired by the optical inspection system shown in FIG. 30 . DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Each embodiment will be described below with reference to the drawings. The drawings are schematic or conceptual, and the relationship between the thickness and width of each part, the size ratio between parts, etc. are not necessarily the same as the actual ones. Even when the same part is shown, the dimensions and ratios may be different depending on the drawing. In this specification and each drawing, elements similar to those described above with respect to the previous drawings are given the same reference numerals, and detailed explanations are omitted as appropriate.

[0008] In this specification, light is a type of electromagnetic wave, and includes X-rays, ultraviolet light, visible light, infrared light, microwaves, etc. In the following embodiments, light is considered to be visible light, for example, in the wavelength range of 400 nm to 750 nm.

[0009] (First embodiment) Fig. 1 shows an optical inspection system 10 according to this embodiment. As shown in Fig. 1, the optical inspection system 10 includes a transport section 12 that transports a subject (sample) S, an optical inspection device 14 that optically acquires an image of the transported subject S, and a processing device 16 that is connected to the optical inspection device 14 by wire or wirelessly.

[0010] Here, an XYZ Cartesian coordinate system shown in FIG. 1 is defined for the transport unit 12 and the optical inspection device 14. The X-axis is the direction in which the subject S is moved by the transport unit 12. Here, the right side of the paper in FIG. 1 is defined as the +X direction. The Y-axis is perpendicular to the X-axis, and together with the X-axis, defines a plane horizontal to the floor surface. Here, the direction toward the back of the paper in FIG. 1 is defined as the +Y direction. The Z-axis is perpendicular to the X-axis and Y-axis, and the direction, for example, upward, is defined as the +Z direction.

[0011] The transport unit 12 has an appropriate width along the Y-axis direction, and transports the subject S in one direction, for example, the +X direction. It is preferable that the transport unit 12 moves the subject S in the +X direction at a predetermined transport speed. It is preferable that the transport unit 12 keeps the Z-axis coordinate constant and the height of the subject S unchanged at the position inspected by the optical inspection device 14. The transport unit 12 is controlled by the processing device 16, as will be described later.

[0012] The transportation here means that the subject S moves relative to the optical inspection device 14, which irradiates the subject S with illumination light and acquires multiple images of the subject S. For example, the subject S may be moved relative to the stationary optical inspection device 14, the optical inspection device 14 may be moved relative to the stationary subject S, or both may be moved. In other words, anything is acceptable as long as the subject S moves relative to the optical inspection device 14. The transport unit 12 may be used to transport the subject S, or may be used to transport the optical inspection device 14. Furthermore, the movement of the subject S relative to the optical inspection device 14 may be continuous or intermittent. For example, the optical inspection device 14 and the subject S may be stationary at the moment when the optical inspection device 14 acquires an image, or the optical inspection device 14 and the subject S may be moving relatively at the moment when the image is acquired. In other words, any manner of movement is acceptable as long as the subject S moving relatively can be photographed. In this embodiment, for example, the subject S is moved at a constant speed by the conveying unit 12 relative to the stationary optical inspection device 14, and images of the surface of the subject S that comes within the field of view of the optical inspection device 14 are acquired multiple times by the imaging unit 24 of the optical inspection device 14, which will be described later.

[0013] The optical inspection device 14 has an illumination unit 22 that illuminates the subject S with illumination light, and an imaging unit (camera) 24. The illumination unit 22 and imaging unit 24 of the optical inspection device 14 are controlled by the processing device 16, which will be described later.

[0014] The illumination unit 22 has a light source 32. The light source 32 illuminates the surface of the subject S with light diverging with respect to the optical axis of the imaging unit 24. A lens or a reflector may be used as a means for generating such illumination light. It is sufficient that the light hits the surface of the subject S at a certain time at an incident angle that differs depending on the position of the subject S within the field of view captured by the imaging unit 24.

[0015] FIG. 2 shows a schematic diagram of the illumination unit 22 along the ZX plane. As shown in FIG. 2, the light source 32 has light-emitting diodes (LEDs) 32a, 32b, and 32c arranged along the Z axis, and in this embodiment, for example, emits white light in the +X direction. The number of LEDs 32a, 32b, and 32c is not limited to three and can be set appropriately. The light source 32 is not limited to the LEDs 32a, 32b, and 32c, and may be an incandescent light bulb, a fluorescent tube, a mercury lamp, or the like. The light emitted by the light source 32 is not limited to white. The light source 32 may be disposed on the optical axis C of the imaging unit 24, or may not be disposed on the optical axis C. When the light source 32 is not disposed on the optical axis of the imaging unit 24, for example, a beam splitter 34 such as a half mirror may be used to illuminate the subject with the light emitted from the light source 32 via the beam splitter 34. The beam splitter 34 may be a polarizing beam splitter or a non-polarizing beam splitter. In this embodiment, the illumination unit 22 has a light source 32, a half mirror 34, and an illumination lens 36. Light from the light source 32 passes through the illumination lens 36, is reflected by the half mirror 34, and is directed toward the surface of the subject S.

[0016] An illumination lens 36 is disposed between the light source 32 and the half mirror 34. The illumination lens 36 may be a cylindrical lens, a free-form lens, a Fresnel lens, or a concave mirror. The illumination lens 36 converts the light from the light sources 32a, 32b, and 32c into parallel light in the ZX plane. In the ZX plane, the parallel illumination light from the light source 32a travels in the -Z direction along the illumination light La on the optical axis C of the imaging unit 24. The parallel illumination light Lb from the light source 32b travels in a different direction at a different position from the illumination light La in the ZX plane. The parallel illumination light La from the light source 32c travels in a different direction at a different position from the illumination lights La and Lb in the ZX plane.

[0017] In this way, the illumination unit 22 irradiates the surface of the subject S with illumination light that is directed in different directions in at least one plane (ZX plane). The LEDs 32a, 32b, and 32c of the light source 32 are turned on at the same time. The illumination unit 22 may be configured to place the light source 32 capable of emitting illumination light that can be considered to be substantially parallel light on the optical axis C and irradiate the surface of the subject S with the illumination light.

[0018] As shown in FIG. 1, the imaging unit 24 includes, for example, an imaging optical element 42 and an image sensor 44.

[0019] The imaging optical element 42 may be a single lens, or may be a plurality of lenses, or may be a combination of a lens and a mirror as necessary. The imaging optical element 42 may have any configuration as long as it can form an image of light.

[0020] The image sensor 44 captures a plurality of images I1, I2, ..., In (n is an integer of 2 or more) formed by the imaging optical element 42 at an appropriate frame rate. The image sensor 44 can capture, for example, RGB by dispersing the light at each pixel. The image sensor 44 captures, for example, n m +1(n m is a natural number), and in the Y-axis direction, for example, m +1(m m The image sensor 44 has a number of pixels (wherein m is a natural number). The image sensor 44 operates to capture an image within a suitable field of view at a suitable frame rate controlled by the processing device 16. The frame rate can be adjusted, for example, by the speed at which the subject S is transported by the transport unit 12.

[0021] It is assumed that each pixel can receive light rays of at least two different wavelengths, that is, light rays of a first wavelength and light rays of a second wavelength. A surface including an area where the image sensor 44 is arranged is the image surface of the imaging optical element 42. The image sensor 44 may be an area sensor or a line sensor. An area sensor is a sensor in which pixels are arranged in an area on the same surface. A line sensor is a sensor in which pixels are arranged in a line. Each pixel may have three color channels of R, G, and B. In this embodiment, it is assumed that the image sensor 44 is an area sensor, and each pixel has three color channels of red, blue, and green.

[0022] The processing device 16 has a processor 52 for holding images captured by the imaging section 24 and performing image processing, which will be described later, and a storage device 54 for saving images.

[0023] The processor 52 is, for example, a CPU or a GPU, but may be any element capable of performing image processing, which will be described later. The processor 52 corresponds to the central part of a computer that performs processing such as calculation and control required for processing of the processing device 16, and controls the entire processing device 16 in an integrated manner. The processor 52 executes control to realize various functions of the processing device 16 based on programs such as system software, application software, or firmware stored in a storage device 54 such as a ROM or an auxiliary storage device. The processor 52 includes, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a graphics processing unit (GPU). Alternatively, the processor 52 is a combination of a plurality of these. The processor 52 provided in the processing device 16 may be one or more.

[0024] The processing device 16 executes processes to perform various functions by having the processor 52 execute programs and the like stored in the storage device 54. It is preferable that the control program of the processing device 16 is not stored in the storage device 54 of the processing device 16, but is placed on an appropriate server or cloud. In this case, the control program is executed while communicating with, for example, the processor 52 of the optical inspection system 10 via a communication interface. That is, the processing device 16 according to the present embodiment may be included in the optical inspection system 10, or may be located on a server or cloud of various inspection site systems away from the optical inspection system 10. For this reason, it is also preferable that the optical inspection program is not stored in the storage device 54, but is on a server or cloud, and the program is executed while communicating with, for example, the processor 52 of the optical inspection system 10 via a communication interface. Therefore, the processor 52 (processing device 16) can execute an optical inspection program (optical inspection algorithm) described later.

[0025] The processor 52 (processing device 16) controls the timing of light emission of the light source 32 of the illumination unit 22, the timing of image data acquisition by the image sensor 44, acquisition of image data from the image sensor 44, etc., and can also superimpose multiple images using arithmetic operations, etc., or perform appropriate image processing on an image.

[0026] Furthermore, the storage device 54 is, for example, a HDD or SSD, but may be anything that can store (save) images.

[0027] Hereinafter, the operation of the optical inspection system 10 according to the first embodiment will be described.

[0028] The subject S is assumed to be moving at a constant speed in a direction perpendicular to the optical axis of the imaging unit 24, and the optical inspection system 10 acquires multiple images of the subject S during this movement. The subject S is also assumed to be a flat surface, a curved surface, or a combination of these.

[0029] 3 is a schematic diagram showing an example in which the surface of the subject S transported in the +X direction on the transport section 12 is imaged by the imaging unit 24 using a comparative optical inspection device 114 that uses illumination light different from that of the optical inspection device 14 of the optical inspection system 10 according to this embodiment. The comparative optical inspection device 114 illuminates the surface of the subject S using diffuse light, which is normal illumination light, regardless of, for example, the ZX plane or the YZ plane, and acquires an image by the imaging unit 24.

[0030] Fig. 4 shows an arrangement of images of the surface of the subject S captured at an appropriate frame rate using the optical inspection device 114 shown in Fig. 3. The images on the upper side were captured earlier than the images on the lower side. The dashed line in Fig. 4 indicates the virtual outline of the subject S relative to the field of view F of the imaging unit 24.

[0031] For ease of explanation, it is assumed here that the subject S shown in Fig. 3 is formed as a rectangular flat plate, and stars are drawn on the surface of the subject S. It is assumed that the surface of the subject S (the upper surface in Fig. 3) is, for example, a glossy surface.

[0032] 4, the optical inspection device 114 can acquire an image of the entire surface of the subject S within the field of view F of the imaging unit 24 according to the transport position of the transport unit 12. The images that can be acquired by the optical inspection device 114 are images of the entire field of view F.

[0033] Fig. 5 is a schematic diagram showing a flat-plate-shaped subject S transported by the transport unit 12 of the optical inspection system 10 shown in Fig. 1, an optical inspection device 14 that optically acquires an image of the subject S, and illumination light in the ZX plane. In Fig. 5, the illumination unit 22 is omitted. Note that the subject S here is the same as that shown in Fig. 4, photographed by the optical inspection device 114 shown in Fig. 3.

[0034] Fig. 6 is a schematic diagram showing the subject S, the optical inspection device 14, and the illumination light in the YZ plane, as viewed from the direction indicated by symbol VI in Fig. 5. Fig. 7 is a schematic diagram different from Fig. 6, showing the subject S, the optical inspection device 14, and the illumination light in the YZ plane, as viewed from the direction indicated by symbol VI in Fig. 5. The illumination unit 22 is omitted from Figs. 6 and 7.

[0035] As shown in Fig. 6, an example of the illumination light in the YZ plane is, for example, diffused light. As shown in Fig. 7, another example of the illumination light in the YZ plane is, for example, light parallel to the Z axis. In this embodiment, the illumination light in the YZ plane will be described as being diffused light as shown in Fig. 6.

[0036] Fig. 8 is a schematic diagram showing images I1, I2, ..., In that can be acquired at appropriate time intervals by the optical inspection device 14 according to this embodiment shown in Fig. 5 and Fig. 6. Note that the dashed lines in Fig. 8 virtually show the position and size of the subject S relative to the field of view F, and are not obtained as images.

[0037] The images at the top in FIG. 8 are images captured earlier than the images at the bottom. The five images I1, I2, I3, I4, and I5 in FIG. 8 are not necessarily the first to fifth images captured by the image sensor 44 of the imaging unit 24 by the processing device 16 in the flow described later (see FIG. 10), but for convenience they are designated by the symbols I1, I2, I3, I4, and I5. For example, one or more additional images may be captured between the images I1 and I2. That is, five images I1, I2, I3, I4, and I5 are shown in FIG. 8, but one or more images may be captured at different times between the top image I1 and the second image I2 from the top. This is the same for the images I2 and I3, the images I3 and I4, and the images I4 and I5. It is preferable that these images not shown are appropriately stored in the storage device 54. In this embodiment, for convenience of explanation, the image I1 is taken as the reference captured image. The reference captured image may be selected from, for example, the images I2, I3, I4, and I5.

[0038] Fig. 9 shows one surface-structure image Is of the subject S, constructed based on the image shown in Fig. 8. The horizontal length of the surface-structure image Is in Fig. 9 can be appropriately set according to the length of the subject S along the X-axis direction.

[0039] FIG. 10 shows a flow chart of processing by one or more processors 52 of processing device 16 of optical inspection system 10 according to this embodiment.

[0040] Here, the subject S used for inspection by the optical inspection device 14 according to this embodiment is exactly the same as the subject photographed by the optical inspection device 114 shown in Fig. 3. Also, the transport speed of the subject S by the transport unit 12 and the settings of the imaging unit 24 for the subject S are the same. Here, the optical inspection device 14 according to this embodiment has a different illumination for the surface of the subject S compared to the optical inspection device 114 in the case of normal photographing.

[0041] As shown in FIG. 10, first, while transporting the subject S by the transport section 12, the processing device 16 (one or more processors 52) acquires, by the optical inspection device 14, multiple images I1, I2, ..., In of each portion of the surface of the subject S when illumination light from the illumination section 22 is irradiated onto the surface of the subject S at different angles of incidence depending on the position of the subject S transported by the transport section 12 (step S11).

[0042] Of the light along the ZX plane incident on the surface of the subject S from the illumination unit 22 of the optical inspection device 14 shown in Figs. 1, 2, 5, and 6 according to this embodiment, the light incident on the surface of the subject S along the optical axis of the imaging unit 24 is almost regularly reflected because the surface of the subject S is flat (a plane parallel to the XY plane), and is incident on the image sensor 44 through the half mirror 34 and the imaging optical element 42. On the other hand, of the light incident on the surface of the subject S from the illumination unit 22 of the optical inspection device 14, when light that deviates from the optical axis of the imaging unit 24 is incident on the surface of the subject S, due to the law of reflection, it is difficult for the light to be incident on the imaging optical element 42 of the imaging unit 24. When such light that deviates from the optical axis of the imaging unit 24 is incident on the surface of the subject S, a part of the reflected light from the surface of the subject S becomes light that is not reflected toward the imaging optical element 42 of the imaging unit 24, and falls outside the field of view F of the imaging optical element 42 and the image sensor 44, for example. 1, 2, 5, and 6 of the optical inspection device 14 can obtain an image of the optical axis C of the imaging unit 24 in the X-axis direction and its vicinity, but it is difficult to obtain an image of a portion outside the optical axis C and its vicinity in the X-axis direction. That is, an image of the optical axis C of the imaging unit 24 and its vicinity in the X-axis direction can be obtained as an image of the surface of the subject S, but light is not incident on the imaging unit 24 for a portion outside the optical axis C and its vicinity in the X-axis direction, and the portion is imaged as black, for example.

[0043] That is, in the ZX plane of the optical inspection device 14, of the illumination light onto the surface of the subject S, the light that is reflected toward the imaging optical element 42 of the imaging unit 24 based on the law of reflection is imaged by the imaging unit 24 as a regular reflection component.

[0044] On the other hand, among the light along the YZ plane incident on the surface of the subject S from the illumination unit 22 (not shown) of the optical inspection device 14 shown in FIG. 6 according to this embodiment, the diffused light incident on the surface of the subject S along the optical axis C of the imaging unit 24 is incident on the image sensor 44 through the half mirror 34 and the imaging optical element 42. In addition, the light incident on the surface of the subject S deviating from the optical axis C of the imaging unit 24 is diffused light, and is therefore incident on the image sensor 44 through the half mirror 34 and the imaging optical element 42. For this reason, in the example shown in FIG. 6, the imaging unit 24 can obtain not only an image of the optical axis C of the imaging unit 24 and its vicinity in the Y-axis direction, but also an image of a portion deviating from the optical axis C along the Y-axis direction in addition to the optical axis C of the imaging unit 24 and its vicinity.

[0045] 5 and 6, when the subject S is a flat plate and the surface of the subject S is planar, the images I1, I2, I3, I4, and I5 of the surface of the subject S that can be acquired by the optical inspection device 14 according to this embodiment are part of the X-axis direction within the field of view of the image sensor 44, as shown in Fig. 8. On the other hand, in this embodiment, the image sensor 44 can obtain an image of the entire Y-axis direction within the corresponding X-axis range within the field of view F of the image sensor 44.

[0046] Here, the processing device 16 (one or more processors 52) knows the transport direction and transport speed of the subject S by the transport unit 12, and the position of the subject S on the transport unit 12. The processor 52 also knows the acquisition time and acquisition time interval (frame rate) of each of the images I1, I2, ..., In.

[0047] The processing device 16 (one or more processors 52) sets a reference image I1, and from each of the acquired partial images of the surface of the subject S (not limited to the images I2, I3, I4, and I5 shown in FIG. 8, but including images acquired between each image, here referred to as images I2, I3, I4, and I5 for convenience), the images I2, I3, I4, and I5 picked up according to the conveying speed of the conveying unit 12 are shifted by, for example, n×Δx (n is a natural number) and superimposed to construct at least one subject surface constituent image Is (step S12). At this time, for example, the first reference image I1 is not shifted, the second image is shifted by 1×Δx in the -X direction, and the third image is shifted by 2×Δx in the -X direction, and so on, shifting by (n-1)×Δx in the -X direction to generate one surface image Is of the subject S (see FIG. 9). Here, when constructing the subject surface image Is, the processing device 16 sets the shift amount of the captured images I2, I3, ..., In based on the frame rate of the image sensor 44 of the imaging unit 24, the transport speed of the subject S relative to the imaging unit 24, and the number of pixels of the image sensor 44 in the direction along the transport direction of the subject S relative to the image sensor 44 of the imaging unit 24.

[0048] In this manner, the processing device 16 according to this embodiment rearranges the multiple images I1, I2, I3, ... captured by the imaging unit 24 as shown in FIG. 11, and processes them as described below to generate a new subject surface image Is (see FIG. 9) according to the positional relationship of the subject S.

[0049] From the relationship between the transport direction, transport speed, and position of the subject S by the transport unit 12 and the time of the images I1, I2, ..., In, for example, the image extraction portion of image I2 is shifted by Δx in the -X axis direction with respect to image I1, and this is image I21. Similarly, the image extraction portion of image I3 is shifted by Δx in the -X axis direction with respect to image I2, and this is image I31. That is, the image extraction portion of image I3 is shifted by 2×Δx in the -X axis direction with respect to image I1. In this way, for example, images I2, I3, ..., In are sequentially shifted by Δx in the -X axis direction with respect to the previous image. That is, the processing device 16 creates images I21, I31, ..., In1.

[0050] Then, for example, the images I21, I31, ..., In1 that have been subjected to such image processing are added to the image I1. In this way, the processing device 16 can generate an image (constituent image) Is relating to the surface of the subject S. It should be noted that images I1, I2, etc. may partially overlap. In this case, the processing device 16 subtracts one of the overlapping parts or averages the pixel values ​​of the overlapping parts. For this reason, when constructing the subject surface image Is, the processing device 16 synthesizes the captured images I2, ..., In with the reference captured image I1 based on appropriate arithmetic operations.

[0051] The transport direction of the subject S is the X-axis direction, half of the field of view F of the imaging unit 24 is x0, and the number of pixels in the transport direction of the image sensor 44 is n m +1, and the frame rate of the image sensor 44 is f c Let the transport speed of the subject S be ν. For example, the N+1th image (where N is a natural number) can be overlaid by shifting it by Δx in the -X direction relative to the previous Nth image. The shift amount Δx for each captured image is Δx=(ν·n m ) / (2x0 f c ), which can be calculated using

[0052] As an example, x0 = 50 mm, n m = 1000pix, ν = 100m / s, f c= 50 fps. At this time, Δx = 20 pix is ​​obtained. Therefore, the processing device 16 can obtain a surface structure image Is (see FIG. 9) of the subject S by, for example, adding up a plurality of images I2, I3, ..., In shifted by Δx = 20 pix × (n-1) in the -X direction with respect to the reference image I1. Such an image Is is obtained by regular reflection light of illumination light that can be regarded as substantially parallel light, not diffuse light, in the ZX plane.

[0053] The surface structure image Is of the subject S according to this embodiment (see FIG. 9) can be obtained by illumination light from limited directions in the ZX plane. Therefore, the optical inspection system 10 according to this embodiment can obtain an image Is of the surface of the subject S, which is less affected by other light than the light directly facing the surface of the subject S. The positional relationship between the subject S and the optical inspection device 14 changes from moment to moment due to transportation. For this reason, the light directly facing the surface of the subject changes from moment to moment in its positional relationship with the optical inspection device 14. Therefore, the image In of the subject S at a certain time is different from the image In+1 of the subject S at another time. The processing device 16 then converts the image of the subject into x0,n m ,ν,f c By setting Δx in accordance with the relationship and performing image processing, for example, an entire image Is of the surface of the subject S can be obtained.

[0054] FIG. 12 shows an arrangement of images I1, I2, ..., I6 of the surface of a subject S, which is an actual plastic flat plate with scratches on its surface, captured at an appropriate frame rate using the optical inspection system 10 according to this embodiment. The images at the top were captured earlier than the images at the bottom. FIG. 13 shows one surface configuration image Is of the subject S, constructed based on the image shown in FIG. 12. As shown in FIG. 13, the optical inspection system 10 according to this embodiment can obtain an image equivalent to the actual surface image of the subject S (surface configuration image Is of the subject S).

[0055] As shown in FIG. 14, the processing device 16 can perform image processing (step S13) on the surface constituent image Is of the subject S shown in FIG. 13. When the processor 52 of the processing device 16 performs image processing for defect identification on the surface constituent image Is of the subject S, it performs arithmetic processing of convolving an image filter on the surface constituent image Is of the subject S. Here, DoG (Difference of Gaussian) processing is used as an example of image processing. As is clear from FIGS. 13 and 15, it can be seen that a defect identification image Id in which only the scratches d are made clear can be obtained from the surface constituent image Is of the subject S. By such image processing, the processing device 16 can extract defects shown in the surface constituent image Is of the subject S. Not only one defect identification image Id but multiple defect identification images Id can be obtained.

[0056] The image processing by the processing device 16 includes, for example, line detection, edge detection, frequency filtering, Canny filtering, Laplacian filtering, Gaussian filtering, DoG processing, etc. Any specific method may be used as long as it is image processing capable of extracting defects on the surface of the subject S.

[0057] Therefore, according to this embodiment, there are provided a processing device 16, an optical inspection system 10 having the processing device 16, a method for constructing a surface image Is of the subject S, and a program for constructing the surface image Is of the subject S, which perform processing to grasp the surface condition of a subject S having various shapes.

[0058] A method for constructing a surface image Is of a subject S in this embodiment includes irradiating illumination light directed in different directions within at least one plane onto the surface of the subject S, which moves relative to the illumination light, capturing an image of the surface of the subject S with an imaging unit 24 to obtain multiple captured images I1, I2, I3, I4, and I5 at multiple times including a reference captured image I1, and shifting and superimposing multiple captured images I2, I3, I4, and I5 selected from among the captured images I1, I2, I3, I4, and I5 to construct at least one surface image Is of the subject. The program for constructing a surface image Is of a subject S in this embodiment causes one or more processors 52 to execute a process of shifting and superimposing multiple captured images I2, I3, I4, I5 selected from captured images I1, I2, I3, I4, I5 obtained at multiple times, including a reference captured image I1, when illumination light directed in different directions within at least one plane is irradiated onto the surface of the subject S moving relative to the illumination light, and constructing at least one surface image Is of the subject. The processing device used to construct a subject surface image in this embodiment has one or more processors 52 that shift and superimpose multiple captured images I2, I3, I4, I5 selected from captured images I1, I2, I3, I4, I5 obtained by an imaging unit 24 when illumination light directed in different directions within at least one plane is irradiated onto the surface of a subject S moving relative to the illumination light, and constructs at least one subject surface image Is.

[0059] According to the embodiment, the optical inspection system 10 includes an illumination unit 22 that irradiates the surface of the subject S with illumination light each directed in a different direction, an imaging unit 24 (image sensor 44) that images the surface of the subject S at a predetermined frame rate, a transport unit 12 that moves the surface of the subject S in a predetermined direction relative to the illumination unit 22 and the imaging unit 24, and a processing device 16 that includes one or more processors 52. The one or more processors 52 of the processing device 16 are used as described above.

[0060] Therefore, according to this embodiment, it is possible to provide a processing device 16 that performs processing to grasp the condition of the surface of a subject having various shapes, an optical inspection system 10 having the processing device 16, a method for constructing a surface image Is of a subject S, and a program for constructing a surface image Is of a subject S.

[0061] According to an embodiment, when constructing at least one subject surface image Is, the processor 52 sets the shift amount of the captured images I2, I3, ..., In based on the frame rate of the imaging unit 24 (image sensor 44), the transport speed of the subject S relative to the imaging unit 24, and the number of pixels of the imaging unit 24 (image sensor 44) in a direction along the transport direction of the subject S relative to the imaging unit 24. The processor 52 calculates the shift amount Δx as Δx=(ν·n m ) / (2x0 f c ), where x0 is half the field of view of the imaging unit 24 (image sensor 44) along the transport direction of the subject S, n m +1: the number of pixels of the imaging unit 24 (image sensor 44) along the transport direction of the subject S, f c : the frame rate of the imaging section 24 (image sensor 44), and ν: the transport speed of the subject S relative to the imaging section 24 (image sensor 44).

[0062] When the processor 52 shifts and overlaps the captured images I2, I3, I4, and I5 to form at least one subject surface image Is, it synthesizes the captured images I1, which serves as a reference, based on, for example, four arithmetic operations. There are various possible methods for overlapping the images I1, I2, ..., In. For example, the pixel values ​​of the overlapping images can be compared, and the pixel with the largest pixel value can be used as the pixel value in the constituent image Is. The average value of the overlapping pixel values ​​can also be used. If the surface image Is of the subject S is constructed in this way, it is possible to obtain a constituent image Is in which the positional relationship of the subject S is correctly depicted.

[0063] The processor 52 performs image processing for defect identification on at least one object surface image Is to obtain a defect extraction image Id by extracting defects from the object surface image Is.

[0064] When performing image processing for defect identification, the processor 52 performs arithmetic processing on at least one object surface image Is by convolving an image filter.

[0065] Next, an example in which the entire surface of the subject S is inclined with respect to the XY plane will be described with reference to FIGS.

[0066] Fig. 16 shows an example in which the surface of the subject S is an inclined plane inclined with respect to the XY plane in the optical inspection device 14 shown in Fig. 5. The illumination unit 22 is omitted in Fig. 16. In the example shown in Fig. 16, the surface is formed as an inclined surface that becomes higher in the +X direction. Note that the illumination light along the YZ plane for the surface of the subject S is diffused light as shown in Fig. 6.

[0067] 16, the illumination light along the optical axis C of the imaging unit 24 in the ZX plane is unlikely to be incident on the imaging optical element 42 of the imaging unit 24 due to the law of reflection. On the other hand, of the illumination light to the subject S in the ZX plane, the light that is reflected toward the imaging optical element 42 of the imaging unit 24 based on the law of reflection is incident on the imaging optical element 42 of the imaging unit 24 as a regular reflection component and is imaged by the image sensor 44.

[0068] Fig. 17 shows examples of images I1, I2, ..., I6 that can be acquired by the optical inspection device 14 shown in Fig. 16 and Fig. 6. The images on the upper side are images taken earlier than the images on the lower side. Note that the dashed lines in Fig. 17 indicate the position and size of the subject S relative to the field of view F, and are not obtained as images.

[0069] FIG. 18 shows one surface structure image Is of the subject S constructed based on the above-mentioned flow (see FIG. 10). Even if the surface of the subject S is an inclined surface, the surface structure image Is of the subject S can be obtained in the same manner as described above. Even if the surface of the subject S, which is an actual plastic flat plate with scratches on its surface, is an inclined surface, the surface structure image Is of the subject S can be obtained as shown in FIG. 12 and FIG. 14. The processing device 16 can perform image processing on the surface structure image Is of the subject S. By such image processing, defects shown in the surface structure image Is of the subject S can be extracted (see FIG. 15).

[0070] Therefore, even in the case of a subject S having an inclined surface as shown in Figure 16, according to this embodiment, there are provided a processing device 16, an optical inspection system 10 having the processing device 16, a method for constructing a surface image Is of the subject S, and a program for constructing the surface image Is of the subject S, which perform processing to grasp the surface condition of the subject S having various shapes.

[0071] Next, an example in which the surface of the subject S is formed by combining a surface parallel to the XY plane and a surface inclined to the XY plane will be described with reference to FIGS.

[0072] Fig. 19 shows an example in which the surface of the subject S is continuous with a plane S1 parallel to the XY plane and an inclined plane S2 inclined with respect to the XY plane in the optical inspection device 14 shown in Fig. 5. The illumination unit 22 is omitted from Fig. 19. In the example shown in Fig. 19, the inclined plane S2 is formed as an inclined plane that becomes higher in the +X direction. Note that the illumination light along the YZ plane for the surface of the subject S is diffused light as shown in Fig. 6.

[0073] 19, illumination light along the optical axis C of the imaging unit 24 in the ZX plane is reflected specularly according to the law of reflection and enters the imaging optical element 42 of the imaging unit 24. Of the illumination light to the subject S in the ZX plane, light that is reflected toward the imaging optical element 42 of the imaging unit 24 based on the law of reflection is entered as a specular reflection component into the imaging optical element 42 of the imaging unit 24 and is imaged by the image sensor 44.

[0074] Figure 20 shows examples of images I1, I2, I3, I4, and I5 that can be acquired by the optical inspection device 14 shown in Figures 19 and 6. The images at the top were taken earlier than the images at the bottom. Note that the dashed lines in Figure 20 indicate the position and size of the subject S relative to the field of view F, and are not obtained as images.

[0075] FIG. 21 shows one constituent image Is of the subject S constructed based on the above-mentioned flow (see FIG. 10). Even if the surface of the subject S is a combination of a plane S1 and an inclined plane S2, a surface constituent image Is of the subject S can be obtained in the same manner as described above. Even if the surface of the subject S, which is an actual plastic flat plate with scratches on its surface, is an inclined surface, a surface constituent image Is of the subject S can be obtained as shown in FIG. 12 and FIG. 13. The processing device 16 can perform image processing on the surface constituent image Is of the subject S. By such image processing, the processing device 16 can extract defects shown in the surface constituent image Is of the subject S (see FIG. 14).

[0076] Therefore, even in the case of a subject S having a surface that combines a plane S1 and an inclined plane S2 as shown in Figure 18, according to this embodiment, there are provided a processing device 16, an optical inspection system 10 having the processing device 16, a method for constructing a surface image Is of the subject S, and a program for constructing the surface image Is of the subject S, which perform processing to grasp the surface condition of the subject S having various shapes.

[0077] 1, even if the surface of the subject S is wavy, an actual image of the surface of the subject S is acquired as a combination of flat and inclined surfaces as in the above example, and one surface image Is of the subject S is constructed. Therefore, according to the present embodiment, a processing device 16 that performs processing to grasp the surface condition of the subject S having various shapes, an optical inspection system 10 having the processing device 16, a method for constructing a surface image Is of the subject S, and a program for constructing a surface image Is of the subject S are provided.

[0078] (Modification) The subject S shown in FIG. 5 to FIG. 21 is illustrated as an example of a size where the entire subject S fits within one field of view F. For example, an example was described in which the captured images of the subject S are taken, and then the surface-structure image Is is formed using appropriate captured images I1, I2, ..., In. The processing device 16 may, for example, sequentially add the captured images of the subject S before taking the images, and finally form an image Is of a predetermined length. For example, a sheet-like member unwound from a roll may have a length along the conveying direction of, for example, tens or hundreds of meters, and the entire subject S may not fit within one field of view F. In such a case, the processing device 16 may, for example, perform a process to generate a surface-structure image Is of the subject S of a predetermined length so as to divide the inspection range of the surface of the subject S into a predetermined length along the conveying direction, based on the following flow shown in FIG. 22, and to grasp the surface condition of the subject S. The processing according to this embodiment can also be applied to a subject S whose overall length along the transport direction of the subject S is appropriately short, for example, such that the surface structure image of the subject S falls within the field of view F of the image sensor 44.

[0079] The transport unit 12 transports the subject S in the +X direction at a predetermined speed.

[0080] The processing device 16 acquires a partial image of the surface of the subject S when illumination light having an incident angle that varies depending on the position is irradiated onto the surface of the subject S during transport, and sets the acquired image as the first surface component image of the subject S (step S21).

[0081] Next, the processing device 16 judges whether or not the rear end of the subject S has passed through the field of view F of the imaging unit 24 along the conveying direction (step S22). This may be done by the processing device 16 detecting the output of a sensor disposed in the conveying unit 12, or by the extraction status of the image of the imaging unit 24. Here, it is assumed that this is the latter case.

[0082] If the rear end of the subject S has not passed through the field of view F of the imaging unit 24 (S22-No), the processing device 16 judges whether or not a subject surface image Is of, for example, a predetermined length or a desired number of pixels along the conveying direction has been acquired (step S23). Here, the subject surface image Is of a predetermined length or a desired number of pixels along the conveying direction essentially refers to the length or pixels of the range along the X-axis direction where light is incident on the image sensor 44. In other words, it refers to the desired length or desired number of pixels along the X-axis direction when the subject surface image Is is formed. For this reason, the length or pixels along the X-axis direction of the portion where light is not incident on the image sensor 44 and is not obtained as an image is excluded.

[0083] The length of the subject S along the transport direction can be calculated, for example, by setting a predetermined relationship between the number of pixels in the field of view F and the dimensions of the subject, and counting the number of times images are captured in this flow.

[0084] If the predetermined length or the desired number of pixels has not been reached (step S23-No), the processing device 16 performs the process of step S21 again, acquires a partial image of the surface of the subject S, and sets the acquired image as the second of the surface constituent images of the subject S. Note that the partial image of the surface of the first subject S and the partial image of the surface of the second subject S are shifted by Δx and superimposed on the partial image of the surface of the first subject S. At this time, it is preferable that the images partially overlap.

[0085] The processing device 16 repeats such operations, and when a subject surface constituent image (sample surface constituent image) Is of a desired length is obtained, for example, from the n-th image, the image at that point in time is set as the constituent image Is of the surface of the subject S (S24), and the process returns to step S21 again to obtain a new first image of the surface of the subject S. It is preferable that this new first image of the surface of the subject S overlaps part of one of the images of the subject surface image Is.

[0086] If the rear end of the subject S passes through the field of view F of the imaging unit 24 (S22-Yes), the processing device 16 configures the image generated in step S21 as a surface configuration image Is (step S25).

[0087] According to the optical inspection system 10 of this modification, it is possible to obtain an image (surface structure image Is of the subject S) equivalent to an image of the actual surface of the subject S. At this time, it is possible to obtain one or more surface structure images Is of an appropriate length of the subject S (S24). In addition, it is possible to obtain one surface structure image Is including the rear end of the subject S (S25).

[0088] Then, the processing device 16 can perform the above-mentioned image processing (see FIGS. 13 and 15) on the surface structure image Is of the subject S (step S13). By such image processing, defects shown in the surface structure image Is of the subject S can be extracted. The image processing by the processing device 16 is, for example, line detection or edge detection processing, frequency filtering, Canny filtering, Laplacian filtering, Gaussian filtering, DoG (Difference of Gaussian) processing, etc. Any specific method may be used as long as it is image processing that can extract defects on the surface of the subject S.

[0089] Therefore, according to this modified example, a processing device 16 that performs processing to grasp the surface condition of a subject S having various shapes, an optical inspection system 10 having the processing device 16, a method for constructing a surface image Is of a subject S, and a program for constructing a surface image Is of a subject S are provided.

[0090] Therefore, a method for constructing a surface image Is of a subject S according to a modified example of this embodiment includes irradiating illumination light directed in different directions in at least one plane onto the surface of the subject S, which moves relative to the illumination light, capturing an image of the surface of the subject S at a predetermined frame rate using the imaging unit 24 to obtain a reference captured image, and shifting each of the captured images following the reference captured image and superimposing it on an captured image captured before that captured image to construct at least one surface image Is of the subject.

[0091] In addition, in a modified example of this embodiment, when constructing at least one subject surface image Is, the shift amount of at least a plurality of captured images is set based on the frame rate in the imaging unit 24, the transport speed of the subject S relative to the imaging unit 24, and the number of pixels of the imaging unit 24 in a direction along the transport direction of the subject S relative to the imaging unit 24.

[0092] In a modification of this embodiment, at least one object surface image Is is subjected to image processing for defect identification.

[0093] The processing device 16 used to construct a subject surface image Is in accordance with a modified example of this embodiment has one or more processors 52 that obtain a reference captured image I1 at a predetermined frame rate by the imaging unit 24 when illumination light directed in different directions within at least one plane is irradiated onto the surface of the subject S moving relative to the illumination light, and that obtains captured images I2, I3, ..., In following the reference captured image I1, shifting each of them and superimposing them in order on captured images I1, I2, ... captured before captured images I2, I3, ... to construct at least one subject surface image Is.

[0094] A program for constructing a subject surface image Is in accordance with a modified example of this embodiment obtains a reference captured image I1 at a predetermined frame rate by the imaging unit 24 when illumination light directed in different directions within at least one plane is irradiated onto the surface of the subject S moving relative to the illumination light, and causes one or more processors 52 to execute a process of constructing at least one subject surface image Is by shifting captured images I2, I3, ..., In following the reference captured image I1 and superimposing them on images captured before captured images I2, I3, ... When constructing at least one subject surface image Is, the processor 52 sets the shift amount of the captured image based on the frame rate in the imaging unit 24, the transport speed of the subject S relative to the imaging unit 24, and the number of pixels of the imaging unit 24 in a direction along the transport direction of the subject S relative to the imaging unit 24.

[0095] The processor 52 calculates the shift amount Δx as Δx=(ν·n m ) / (2x0 f c ), where x0 is half the field of view of the imaging unit 24 (image sensor 44) along the transport direction of the subject S, n m +1: the number of pixels of the imaging unit 24 (image sensor 44) along the transport direction of the subject S, f c : the frame rate of the imaging section 24 (image sensor 44), and ν: the transport speed of the subject S relative to the imaging section 24 (image sensor 44).

[0096] When the processor 52 shifts and overlaps the captured images I2, I3, I4, and I5 to form at least one subject surface image Is, it synthesizes the captured images I1, which serves as a reference, based on arithmetic operations.

[0097] The processor 52 performs image processing for defect identification on at least one object surface image Is to obtain a defect extraction image Id by extracting defects from the object surface image Is.

[0098] When performing image processing for defect identification, the processor 52 performs arithmetic processing on at least one object surface image Is by convolving an image filter.

[0099] Second embodiment Next, an optical inspection system 10 according to a second embodiment will be described with reference to Fig. 23 to Fig. 27. This embodiment is a modified example of the optical inspection device 14 of the optical inspection system 10 according to the first embodiment, and the same members or members having the same functions as those described in the first embodiment are given the same reference numerals, and detailed descriptions thereof will be omitted.

[0100] Fig. 23 shows a schematic diagram of an optical inspection system 10 according to this embodiment. Fig. 24 shows a schematic diagram of an optical inspection device 14 according to this embodiment.

[0101] 23 and 24, in the optical inspection device 14 according to this embodiment, a sheet-like color aperture (wavelength selection unit) 26 that is perpendicular to the optical axis C and parallel to, for example, the XY plane is provided on the optical axis C between the subject S and the imaging optical element 42 of the imaging unit 24. The color aperture 26 is preferably provided, for example, between the half mirror 34 of the illumination unit 22 and the imaging optical element 42 of the imaging unit 24.

[0102] The color aperture 26 has at least two or more wavelength selection regions (three wavelength selection regions 72, 74, 76 in this embodiment). These are respectively called the first wavelength selection region 72, the second wavelength selection region 74, and the third wavelength selection region 76. The direction in which the first wavelength selection region 72, the second wavelength selection region 74, and the third wavelength selection region 76 are arranged is along the X-axis. That is, the first wavelength selection region 72, the second wavelength selection region 74, and the third wavelength selection region 76 intersect with the X-axis. The direction in which the first wavelength selection region 72, the second wavelength selection region 74, and the third wavelength selection region 76 extend is along the Y-axis. That is, the first wavelength selection region 72, the second wavelength selection region 74, and the third wavelength selection region 76 extend along the Y-axis. However, this is not a limitation, and the first wavelength selection region 72, the second wavelength selection region 74, and the third wavelength selection region 76 may intersect with the Y axis.

[0103] The first wavelength selective region 72 passes light having a wavelength spectrum including a first wavelength. Here, passing light means directing light from an object point to an image point by transmission or reflection. On the other hand, the first wavelength selective region 72 substantially blocks light having a second wavelength and a third wavelength different from the first wavelength. Blocking here means not passing light. In other words, it means not directing light from an object point to an image point. The second wavelength selective region 74 passes a wavelength spectrum including light having the second wavelength, but substantially blocks light having the first wavelength and the third wavelength. The third wavelength selective region 76 passes a wavelength spectrum including light having the third wavelength, but substantially blocks light having the first wavelength and the second wavelength. For this reason, the wavelength selective regions 72, 74, and 76 of the color aperture 26 selectively pass light having three different wavelength spectrums.

[0104] For example, the first wavelength is 450 nm blue light, the second wavelength is 650 nm red light, and the third wavelength is 550 nm green light, but the wavelengths are not limited to these and may be any wavelength.

[0105] Each pixel of the image sensor 44 is assumed to be capable of receiving at least a light beam of a first wavelength, a light beam of a second wavelength, and a light beam of a third wavelength. In this embodiment, each pixel of the image sensor 44 is assumed to have three color channels, red, blue, and green. That is, each pixel of the image sensor 44 is assumed to be capable of receiving blue light of a wavelength of 450 nm, red light of a wavelength of 650 nm, and green light of a wavelength of 550 nm in each independent color channel. However, each color channel does not need to be completely independent, and in reality, any color channel may have slight sensitivity to wavelengths other than the wavelength to which it has high sensitivity.

[0106] In this embodiment, the color aperture 26 is configured such that a first wavelength selection region 72, a second wavelength selection region 74, and a third wavelength selection region 76 are arranged in that order, for example, in the +X direction, to form one set, and multiple sets of these are arranged in the +X direction.

[0107] The operating principle of the optical inspection system 10 of this embodiment will be described.

[0108] For example, suppose that parallel light rays La and Lb are irradiated onto the surface of subject S as shown in Figure 24. In Figure 24, object point Oa is a mirror surface with no defects, while object point Ob has micron-sized uneven defects close to the wavelength of light. In this case, the directional distribution of light reflected at object point Oa (BRDF; Bidirectional Reflectance Distribution Function) has a narrow distribution. On the other hand, the BRDF at object point Ob has a wide distribution. In other words, object point Oa and object point Ob each have a different BRDF.

[0109] The illumination light La causes reflected light to be generated from the object point Oa. The reflected light from the object point Oa passes only through, for example, the second wavelength selection region 74 of the color aperture 26, and becomes light having a wavelength spectrum of, for example, a wavelength of 650 nm (red light). Alternatively, the reflected light passes through the first wavelength selection region 72 and the second wavelength selection region 74, and becomes light having a wavelength spectrum of, for example, a wavelength of 450 nm (blue light) and a wavelength of 650 nm (red light). Such light is then imaged by the imaging unit 24.

[0110] The illumination light Lb causes reflected light from the object point Ob. The reflected light from the object point Ob passes through, for example, a plurality of sets of the first wavelength selection region 72, the second wavelength selection region 74, and the third wavelength selection region 76 of the color aperture 26. For this reason, the light that passes through the first wavelength selection region 72 becomes light of the first wavelength (blue light), the light that passes through the second wavelength selection region 74 becomes light of the second wavelength (red light), and the light that passes through the third wavelength selection region 76 becomes light of the third wavelength (green light). Note that, of the reflected light from the object point Ob, the light of the second wavelength and the light of the third wavelength that enter the first wavelength selection region 72 are blocked, the light of the first wavelength and the light of the third wavelength that enter the second wavelength selection region 74 are blocked, and the light of the first wavelength and the light of the second wavelength that enter the third wavelength selection region 76 are blocked.

[0111] If the reflected light from the object point Oa can reach the imaging optical element 42, the object point Oa is reflected to a first image point Ia by the imaging optical element 42. In this embodiment, the reflected light from the object point Ob reaches the imaging optical element 42 and is reflected to a second image point Ib.

[0112] However, when acquiring one image, if the illumination light La faces the same direction as the illumination light Lb, the reflected light from the object point Oa cannot reach the imaging optical element 42. This is because the normal directions of the surface of the subject S at the object points Oa and Ob are different, and the reflection direction is determined according to the illumination direction and the normal direction. In other words, if the direction of the illumination light is not appropriately set according to the normal direction of the surface of the subject S, the reflected light cannot reach the imaging optical element 42. If the reflected light does not reach the imaging optical element 42, the object point Oa will not appear in the image. For this reason, if the reflected light from the object point Oa does not reach the imaging optical element 42, the surface condition of the object point Oa cannot be inspected with one image.

[0113] The object point Ob is projected onto the second image point Ib by the imaging optical element 42. However, if the first illumination light Lb faces the same direction as the illumination light La, the reflected light from the object point Ob cannot reach the imaging optical element 42. This is because the normal directions of the surface of the object S at the object points Oa and Ob are different, and the reflection direction is determined according to the illumination direction and the normal direction. In other words, if the direction of the illumination light is not appropriately set according to the normal direction of the surface of the object S, the reflected light cannot reach the lens. If the reflected light does not reach the imaging optical element 42, the object point Ob will not appear in the image. For this reason, if the reflected light from the object point Ob does not reach the imaging optical element 42, the condition of the surface S of the object point Ob cannot be inspected with one image.

[0114] As described above, by irradiating illumination light La and Lb having different directions onto object points Oa and Ob, which each have different normal directions, it is possible to obtain a single image I that can become part of the surface structure image Is of the subject S.

[0115] Therefore, the processing device 16 constructs one constituent image Is of the subject S from a plurality of acquired images, as described in the first embodiment.

[0116] At the first image point Ia, for example, only red light is received by the image sensor 44. Therefore, the processing device 16 recognizes that the light has passed through one type of wavelength selection region 74. Alternatively, at the first image point Ia, blue light and red light are received by the image sensor 44. Therefore, the processing device 16 recognizes that the light has passed through two types of wavelength selection regions 72, 74.

[0117] Meanwhile, blue light, red light, and green light are simultaneously received at the second image point Ib. Therefore, the processing device 16 recognizes that the light has passed through the three types of wavelength selection regions 72, 74, and 76. This process of estimating the number of colors is called a color number estimation process. The processor 52 of the processing device 16 can obtain the number of colors (the number of colors) received at each of the image points Ia and Ib through the color number estimation process.

[0118] The surface configuration image Is of the subject S has information on the number of colors acquired at each pixel. Therefore, the processing device 16 can extract defective parts by performing image processing based on the information on the number of colors at each pixel. In the example of this embodiment, when the processor 52 of the processing device 16 outputs that three colors have been received, it is determined that the object point corresponding to the light receiving position has a defect. Also, when the processor 52 of the processing device 16 outputs that only one color has been received, it is determined that the object point corresponding to the light receiving position does not have a defect. Also, when the processor 52 of the processing device 16 outputs that two colors have been received, it is determined that the object point corresponding to the light receiving position does not have a defect because there is a possibility that light passes through the boundary between the regions 72 and 74 or the boundary between the regions 74 and 76.

[0119] Light reflected in various directions due to the surface properties and microshapes of the subject S is generally called scattered light. As already mentioned above, the extent of the spread of the distribution of scattered light can be expressed by BRDF. In addition, it is considered that a BRDF with a large number of colors spreads, and the smaller the number of colors, the narrower the BRDF. In other words, if the number of colors at each image point can be obtained by color number estimation processing, the difference in BRDF at each object point can be identified. As a result, an image is obtained by capturing light from the subject S that has passed through a color aperture 26 having at least two different wavelength selection regions 72, 74, 76, and a color number estimation processing is performed to estimate the number of wavelength selection regions 72, 74, 76 through which the light has passed from the image, the number of colors is obtained, and the directional distribution of scattered light from the surface of the subject S can be identified based on the number of colors. Since the BRDF is correlated with the properties and microshapes of the surface S, this embodiment has the effect of being able to identify the differences in the properties and microshapes of the surface S of each object point Oa, Ob on the surface of the subject S. This allows the processing device 16 to identify the properties and microshape (surface condition) of the surface of the specimen S in a non-contact manner, without splitting the illumination.

[0120] Fig. 25 shows images I1, I2, I3, I4, and I5 of the surface of the subject S, a scratched plastic flat plate, photographed at an appropriate frame rate using the optical inspection system 10 according to this embodiment, which are then arranged (step S11). The processing device 16 shifts these images I1, I2, I3, I4, and I5 as described in the first embodiment, and adds them together, for example, to obtain the surface image Is shown in Fig. 26. That is, in this embodiment, based on the flow shown in Fig. 14, each image I1, I2, ..., In is appropriately shifted while carrying information on the presence or absence of such defects, and at least one surface image Is of the subject S is constructed (step S12).

[0121] Furthermore, based on at least one surface image Is of the subject S, the processor 52 performs image processing for defect identification on the at least one subject surface image Is to obtain a defect extraction image Id by extracting defects in the subject surface image Is (step S13). When performing image processing for defect identification, the processor 52 performs arithmetic processing on the at least one subject surface image Is by convolving an image filter.

[0122] For example, since the object point Oa is a mirror surface, the reflected light from the object point Oa has a BRDF with a narrow distribution. Even in this case, when passing through the boundary between the first wavelength selection region 72 and the second wavelength selection region 74, the light may be incident on the image sensor 44 with two colors. For this reason, it is preferable that the color aperture 26 is formed as a set of three regions. It is preferable that the widths of the regions 72, 74, and 76 are the same, but they may be different.

[0123] In this way, by providing a multi-color aperture (color filter) in the imaging unit 24, the color of the captured image can be changed according to the directional distribution (BRDF) of light reflected on the curved surface. In other words, BRDF information can be obtained as color information, and defects on the surface of the subject S can be identified based on this.

[0124] Specifically, defects are detected by performing image processing on the color information of the acquired image. Examples of image processing include line detection, edge detection, frequency filtering, Canny filtering, Laplacian filtering, Gaussian filtering, DoG (Difference of Gaussian) processing, etc. Any specific image processing method may be used as long as it can detect defects in the subject S.

[0125] Here, as an example, Fig. 27 shows a surface image Is of a subject S based on images I1, I2, ..., In captured by the optical inspection device 14 of the optical inspection system 10 of this embodiment, which shows scratches on the surface of a plastic plate, and a defect-identified image Id obtained by performing image processing on the image Is to identify the defects. DoG processing was used as an example for the image processing. As is clear from Figs. 26 and 27, it can be seen that only the scratches have been made clearer in the acquired images.

[0126] The processing device 16 according to this embodiment obtains one constituent image Is (see FIG. 26) based on a plurality of images I1, I2, ..., In (see FIG. 25) based on the flow (see FIG. 14) described in the first embodiment, and then performs image processing for identifying defects to obtain a defect-identified image Id (see FIG. 27). The processing device 16 configures the surface image Is of the subject S, and then performs image processing for defect identification on the surface image Is of the subject S, so that image processing only needs to be performed on one image Is, thereby reducing the cost of calculation and enabling high-speed detection. In addition, in order for the processing device 16 to obtain the defect-identified image Id, it is not necessarily necessary for the processing device 16 to hold or store all of the plurality of images I1, I2, ..., In captured by the imaging unit 24. That is, as long as the surface image Is of the subject S described in the first embodiment is held for the image data sequentially transferred from the imaging unit 24 to the processing device 16, a defect identification image Id can be obtained, so the processing device 16 does not need to hold or store all of the sequentially transferred image data in the storage device 54 of the processing device 16. Not having to hold or store all of the images obtained by the imaging unit 24 means that the processing costs involved in holding and storing can be reduced. Another advantage of the processing device 16 is that it can operate at a higher speed in proportion to the reduced processing costs.

[0127] According to this embodiment, the optical inspection device 14 of the optical inspection system 10 has a color aperture 26 provided between the subject S and the imaging unit 24. The color aperture 26 has at least two wavelength selection regions 72, 74, 76 that selectively pass light of different wavelength spectrums in at least one plane (ZX plane). The imaging unit 24 (image sensor 44) can receive light of different wavelength spectrums that pass through the wavelength selection regions 72, 74, 76. Then, the processing device 16 (processor 52) can output defects on the surface of the subject S based on the number of colors of the light of the wavelength spectrum received by the imaging unit 24.

[0128] According to this embodiment, it is possible to provide a processing device 16 used to construct a defect identification image Id of the surface of a subject S having various shapes, an optical inspection system 10 having the processing device, and a configuration program used to construct a defect identification image Id of the surface of a subject S.

[0129] (Modification) An optical inspection system 10 according to a modified example of the second embodiment will be described with reference to FIGS.

[0130] In the processing device 16 according to the second embodiment described above, a surface structure image Is (see FIG. 26) of the subject S is first constructed from each captured image I1, I2, ..., In, and then the processing device 16 performs image processing on the surface structure image Is of the subject S to construct one defect extraction image Id (see FIG. 27) in which defects are extracted.

[0131] As shown in FIG. 28, the processing device 16 constructs defect extraction images Id1, Id2, ..., Idn each time it acquires images I1, I2, ..., In, and appropriately overlays these to construct one defect extraction image Id without constructing one surface image Is of the subject S.

[0132] The processor 52 of the processing device 16 according to the modified example operates based on the flow shown in FIG.

[0133] The processing device 16 first acquires a plurality of images I1, I2, ..., In (step S31). The processing device 16 performs defect identification processing on each of the plurality of images I1, I2, ..., In to obtain a plurality of partial component images Id1, Id2, ..., Idn for defect identification (step S32). Therefore, according to this modification, the processor 52 performs image processing for defect identification on the captured images I1, I2, ..., In to extract defects in the captured images I1, I2, ..., In. In addition, when performing image processing for defect identification, the processor 52 performs arithmetic processing on the captured images by convolving an image filter.

[0134] Then, the processing device 16 appropriately shifts the obtained partial component images Id1, Id2, ..., Idn for defect identification, and performs, for example, four arithmetic operations to obtain one defect identification image Id (step S33). When the same defect is identified across multiple images I1, I2, ..., In, the processing device 16 directly constructs multiple partial component images (defect identification converted images) Id1, Id2, ..., Idn for defect identification from each image I1, I2, ..., In, and when these are added together to obtain one defect identification image Id, the signal of the identified defect is overlapped and increased. In other words, the processing device 16 has the advantage of being able to perform defect identification more robustly and with higher accuracy.

[0135] According to this modified example, the processing device 16 used to construct a defect identification image Id of the surface of the subject S has one or more processors 52 that, when illumination light directed in different directions is irradiated onto the surface of the subject S, which moves relative to the illumination light, obtain a reference captured image I1 at a predetermined frame rate by the imaging unit 24, perform image processing for defect identification on the reference captured image I1 to obtain a reference defect identification image (reference defect identification transformed image) Id1 corresponding to the reference captured image I1, and obtain captured images I2, ... following the reference captured image I1, and perform image processing for defect identification on each of them to turn the captured images I2, ... into defect identification images (defect identification transformed images) Id2, ... by shifting each of the defect identification images Id2, Id3, ... and superimposing the defect identification images Id2, Id3, ... following the reference defect identification image Id1 in order to construct a defect identification image Id of at least one surface of the subject S. The method for constructing a defect identification image Id in this embodiment includes irradiating illumination light directed in different directions within at least one plane onto the surface of the subject S, which moves relative to the illumination light, imaging the surface of the subject S with an imaging unit 24 to obtain multiple captured images at multiple times including a reference captured image I1, performing image processing for defect identification on the multiple captured images I2, I3, ..., In to generate defect identification converted images Id1, Id2, ..., Idn of the multiple captured images I1, I2, ..., In at multiple times, and shifting one or more defect identification converted images Id2, Id3, ..., Idn selected from the multiple defect identification converted images Id1, Id2, ..., Idn and superimposing them on a reference defect identification converted image Id1 corresponding to the reference captured image I1, to construct at least one defect identification image Id. In other words, the program for constructing a defect identification image Id in this embodiment causes one or more processors to execute the following steps: irradiating illumination light directed in different directions within at least one plane onto the surface of the subject S, which moves relative to the illumination light; imaging the surface of the subject S with the imaging unit 24 to obtain multiple captured images at multiple times including a reference captured image I1; performing image processing for defect identification on the multiple captured images I2, I3, ..., In to generate defect identification converted images Id1, Id2, ..., Idn of the multiple captured images I1, I2, ..., In at multiple times; shifting one or more defect identification converted images Id2, Id3, ..., Idn selected from the multiple defect identification converted images Id1, Id2, ..., Idn and superimposing them on a reference defect identification converted image Id1 corresponding to the reference captured image I1, to construct at least one defect identification image Id.

[0136] According to this modified example, it is possible to provide a processing device 16 used to construct a defect identification image Id of the surface of a subject S having various shapes, an optical inspection system 10 having the processing device, a method for constructing a defect identification image Id, and a construction program used to construct a defect identification image Id of the surface of a subject S.

[0137] Third embodiment Next, an optical inspection system 10 according to a third embodiment will be described with reference to Fig. 30 to Fig. 32. This embodiment is a modified example of the optical inspection system 10 according to the first and second embodiments, and the same members or members having the same functions as those described in the first and second embodiments are given the same reference numerals, and detailed descriptions thereof will be omitted. In this embodiment, an example will be described in which the processing device 16 performs a process of estimating an angle according to the position of the surface of the subject S in addition to or instead of the above-mentioned construction process of the surface constituent image Is of the subject S. Here, a description of the construction process of the surface constituent image Is of the subject S will be omitted.

[0138] 30, for example, specularly reflected light from the surface of the subject S being transported by the transport unit 12 changes depending on the inclination angle of the surface of the subject S. The incident position of the specularly reflected light on the image sensor 44 in the ZX plane changes depending on the inclination angle of the surface of the subject S with respect to the X-axis. Note that the illumination unit 22 is not shown in FIG.

[0139] Here, consider the optical inspection device 14 shown in Fig. 31. The light from the illumination unit 32 in Fig. 31 is parallel light traveling from the optical axis C of the imaging unit 24 along the optical axis C toward the surface of the subject S in the ZX plane, and is in the +x o is a parallel light beam traveling toward the surface of the subject S at a position of -x o The light is parallel light directed toward the surface of the subject S at the position of the light source 21. For this reason, the illumination unit 22 irradiates the surface of the subject S with illumination light directed in different directions in at least one plane (in the ZX plane). The distance between the imaging optical element 42 of the imaging unit 24 and the surface of the subject S is a c The angle formed by a virtual line segment connecting the imaging optical element 42 and half x0 of the field of view F of the imaging unit 24 along the X-axis direction and the optical axis of the imaging unit 24 is defined as θc. In addition, the light source 32 is assumed to be a point light source on the optical axis of the imaging unit 24. The distance between the light source 32 and the surface of the subject S is a l Let θc be the angle between the optical axis of the imaging unit 24 and half x0 of the field of view F of the imaging unit 24 along the X-axis direction. Let θl be the angle between a virtual line segment connecting the light source 32 and half x0 of the field of view F of the imaging unit 24 along the X-axis direction and the optical axis of the imaging unit 24. The inclination angle θs at a certain position on the surface of the subject S and the pixel position n at which the light reflected at that position appears on the image (on the image sensor 44) are expressed by the following relational expression. θs=1 / 2tan -1 {x0 / a c (2n / n m -1)}+ 1 / 2tan -1 {x0 / a l (2n / n m -1) Therefore, when light is input to the image sensor 44 within the field of view F and a pixel position n at which each captured image I1, I2, . . . , In is obtained is detected, x0, a c ,n m ,a l is known from the optical inspection device 14, the angle θs is calculated. The processing device 16 can determine the pixel position n from the pixel value, i.e., brightness, of each captured image I1, I2, ..., In. Therefore, there is a one-to-one correspondence between the pixel position n on each captured image I1, I2, ..., In and the inclination angle of the surface of the subject S.

[0140] For example, when θs is near 0°, the surface of the subject S is parallel to the XY plane, and the illumination light is irradiated onto the subject S along the optical axis of the imaging unit 24, the specular reflected light is incident on the position indicated by the symbol Ob (see FIG. 30) of the real field of view F on each of the captured images I1, I2, ..., In. Depending on the inclination angle of the surface of the subject S, the reflected light from the surface of the subject S is incident on the imaging unit 24. Depending on the direction of the normal direction of the inclined surface, the reflected light from the surface of the subject S is divided into the +X direction side and the -X direction side of the image sensor 44 of the imaging unit 24. For example, in the example of the inclined surface at the position indicated by the object point Oa shown in FIG. 30, the reflected light from the surface of the subject S is incident on the +X direction side of the image sensor 44 of the imaging unit 24. And, the larger the inclination angle of the inclined surface at the position indicated by the object point Oa, the more the reflected light from the surface of the subject S is incident on the +x direction side of the image sensor 44 of the imaging unit 24. o The smaller the inclination angle, the closer the reflected light from the surface of the subject S is to a position on the image sensor 44 of the imaging unit 24 that is closer to the optical axis C. Although not shown, when the inclined surface faces in the opposite direction to the direction at the position indicated by the object point Oa, the reflected light from the surface of the subject S is incident on the −X direction side of the image sensor 44 of the imaging unit 24.

[0141] As in the example shown in FIG. 5 to FIG. 21 using the optical inspection device 14 described in the first embodiment, when multiple images I1, I2, ..., In of the subject S being transported are captured by the optical inspection device 14, the appearance of the images I1, I2, ..., In changes depending on the inclination angle and the shape of the curved surface (inclination angle). In other words, the multiple images I1, I2, ..., In obtained by capturing the subject S being transported do not necessarily maintain the original positional relationship of the subject S. Then, based on this angle θs, the inclination angle of the surface of the subject S is estimated from the obtained images I1, I2, ..., In. In other words, the processing device 16 can estimate the inclination angle of the surface of the subject S based on the actually obtained images I1, I2, ..., In.

[0142] 32, the processor 52 of the processing device 16, for example, while transporting the subject S by the transport unit 12, acquires images of the subject S illuminated with illumination light having different angles of incidence depending on the position of the subject S (step S41). The acquired images I1, I2, ..., In each contain angle information of the surface of the subject S.

[0143] Then, the processor 52 of the processing device 16 outputs (estimates) the inclination angle of the surface of the subject S based on each of the images I1, I2, ..., In (step S42). Therefore, the optical inspection system 10 according to this embodiment can estimate, for example, the angle of the surface of the subject S. This allows the processor 52 of the processing device 16 to estimate, for example, whether the surface of the subject S is formed into a desired shape, as well as scratches that may exist on the surface of the subject S.

[0144] Therefore, the method for estimating the angle of the surface of the subject S in this embodiment includes irradiating the surface of the subject S with illumination light directed in different directions within at least one plane, imaging the surface of the subject S with the imaging unit 24 to obtain captured images I1, I2, ..., In, and identifying the positions of pixels in the captured images I1, I2, ..., In onto which the light is incident, and estimating the inclination angle of the surface of the subject S depicted in the captured images I1, I2, ..., In based on the positions of the pixels onto which the light is incident.

[0145] The inclination angle estimation program for the surface of the subject S in this embodiment causes one or more processors 52 to execute a process of estimating the inclination angle of the surface of the subject S depicted in the captured images I1, I2, ..., In based on the positions of pixels onto which light is incident in the captured images I1, I2, ..., In obtained by the imaging unit 24 when illumination light pointing in different directions in at least one plane is irradiated onto the surface of the subject S.

[0146] The processing device 16 for calculating the inclination angle of the surface of the subject S in this embodiment has one or more processors 52 that estimate the inclination angle of the surface of the subject S captured in the captured images I1, I2, ..., In based on the positions of pixels onto which light is incident, among the captured images I1, I2, ..., In obtained by the imaging unit 24 when the surface of the subject S is irradiated with illumination light each directed in a different direction within at least one plane.

[0147] According to the embodiment, the distance between the imaging unit 24 and the surface of the subject S is a c Let x0 be half the field of view F of the imaging unit 24 along the transport direction of the imaging unit 24 and the subject S, and the light source 32 of the illumination light be a point light source on the optical axis C of the imaging unit 24. Let a be the distance between the light source 32 and the surface of the subject S. l The inclination angle at a certain position on the surface of the object S is θ s Let the number of pixels of the image pickup unit 24 along the transport direction of the subject S be n m +1(n m is a natural number) and the pixel position where light from the surface of the subject S is incident on in the image captured by the imaging unit 24 is n, the processor 52 calculates θ s =1 / 2tan -1 {x0 / a c (2n / n m -1)}+1 / 2tan -1 {x0 / a l (2n / n m -1)}, the inclination angle of the surface of the subject S is estimated.

[0148] According to this embodiment, a method for estimating the angle of the surface of a subject S and a program for estimating the inclination angle of the surface of a subject S can be provided.

[0149] According to at least one of the embodiments described above, it is possible to provide a processing device that performs processing to grasp the condition of a subject's surface having various shapes or is used to construct a defect identification image of the subject's surface, an optical inspection system having a processing device, a method for constructing a subject's surface image, a method for constructing a defect identification image, a program for constructing a subject's surface image, a method for estimating the angle of the subject's surface, and a program for estimating the inclination angle of the subject's surface.

[0150] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0151] 10...optical inspection system, 12...transport section, 14...optical inspection device, 16...processing device, 22...illumination section, 24...imaging section, 32...light source, 34...half mirror (beam splitter), 36...illumination lens, 42...imaging optical element, 44...image sensor, 52...processor, 54...memory device, I1, I2, ..., In...captured image, Is...surface structure image, S...subject.

Claims

1. A processing device used for constructing a surface image of a subject, the processing device having one or more processors, the processing device comprising: a processing unit for irradiating illumination light directed in different directions within at least one plane onto the surface of a subject moving relatively to the illumination light, the processing unit irradiating the surface of the subject moving relatively to the illumination light with a plurality of captured images taken at multiple times including a reference captured image, the processing unit shifting one or more captured images selected from the plurality of captured images and superimposing the selected captured images on the reference captured image to construct at least one surface image of the subject.

2. A processing device used for constructing a surface image of a subject, the processing device having one or more processors, wherein when illumination light directed in different directions within at least one plane is irradiated onto the surface of a subject moving relatively to the illumination light, a reference captured image is obtained at a predetermined frame rate by an imaging unit, and captured images subsequent to the reference captured image are obtained and shifted to be superimposed in order on captured images captured prior to the reference captured image, thereby constructing at least one surface image of the subject.

3. When constructing the at least one surface image of the subject, the processor sets a shift amount of the captured image based on a frame rate of the imaging unit, a transport speed of the subject relative to the imaging unit, and a number of pixels of the imaging unit in a direction along a transport direction of the subject relative to the imaging unit. The processing device according to claim 1 or 2.

4. The processor sets the shift amount Δx as Δx=(ν・n ｍ ) / (2x ０ ・f ｃ ) x ０ : Half of the field of view of the imaging unit along the transport direction of the subject n ｍ +1: the number of pixels of the imaging unit along the transport direction of the subject f ｃ : Frame rate of the imaging unit v: transport speed of the subject relative to the imaging unit Set based on, The processing device according to claim 3 .

5. 3 . The processing device according to claim 1 , wherein the processor, when shifting and superimposing the captured images to construct the at least one subject surface image, synthesizes the captured images with the reference captured image based on four arithmetic operations.

6. The processing device according to claim 1 or claim 2, wherein the processor performs image processing for defect identification on at least one of the captured image and at least one of the object surface image, thereby extracting defects in at least one of the captured image and at least one of the object surface image.

7. The processing device according to claim 6 , wherein the processor performs arithmetic processing in which an image filter is convolved with at least one of the captured image and the at least one subject surface image when performing image processing for identifying the defect.

8. A processing device for use in constructing a defect identification image of a subject's surface, having one or more processors, which obtains a reference captured image at a predetermined frame rate by an imaging unit when illumination light directed in different directions is irradiated onto the surface of a subject moving relative to the illumination light, and then performs image processing for defect identification on the reference captured image to obtain a reference defect identification image corresponding to the reference captured image, and obtains captured images following the reference captured image and performs image processing for defect identification on each of the captured images to turn the captured images into defect identification images, shifts each of the defect identification images, and overlays the defect identification images following the reference defect identification image in order to construct a defect identification image of the surface of at least one subject.

9. A processing device for calculating the inclination angle of a surface of a subject, the processing device having one or more processors that estimates the inclination angle of the surface of the subject shown in an image obtained by an imaging unit when the surface of the subject is irradiated with illumination light directed in different directions within at least one plane, based on the position of a pixel into which light is incident.

10. The distance between the imaging unit and the surface of the object is a ｃ year, Half of the field of view F of the imaging unit along the conveying direction of the imaging unit and the object is defined as x ０ year, When the light source of the illumination light is a point light source on the optical axis of the imaging unit, the distance between the light source and the surface of the subject is a ｌ year, The inclination angle at a certain position on the surface of the object is θ ｓ year, The imaging unit determines the number of pixels of the object along the transport direction as n ｍ +1(n ｍ is a natural number), When the pixel position at the position where the light from the surface of the subject is incident in the image captured by the imaging unit is n, The processor, <h2 style=";text-align:left;direction:ltr">θ<h2 style=";text-align:left;direction:ltr"> ｓ <h2 style=";text-align:left;direction:ltr"> 1222<h2 style=";text-align:left;direction:ltr"> －１ <h2 style=";text-align:left;direction:ltr"> {x<h2 style=";text-align:left;direction:ltr"> ０ <h2 style=";text-align:left;direction:ltr"> / a<h2 style=";text-align:left;direction:ltr"> ｃ <h2 style=";text-align:left;direction:ltr"> (2)<h2 style=";text-align:left;direction:ltr"> ｍ <h2 style=";text-align:left;direction:ltr"> -11)}+ <h2 style=";text-align:left;direction:ltr">122<h2 style=";text-align:left;direction:ltr"> －１ <h2 style=";text-align:left;direction:ltr"> {x<h2 style=";text-align:left;direction:ltr"> ０ <h2 style=";text-align:left;direction:ltr"> / a<h2 style=";text-align:left;direction:ltr"> ｌ <h2 style=";text-align:left;direction:ltr"> (2)<h2 style=";text-align:left;direction:ltr"> ｍ <h2 style=";text-align:left;direction:ltr"> -11)} The processing device according to claim 9 , wherein the inclination angle of the surface of the object is estimated based on

11. an illumination unit that irradiates a surface of the subject with illumination light that travels in the different directions; an imaging unit that images the surface of the subject at a predetermined frame rate; a conveying unit that moves a surface of the subject in a predetermined direction relative to the illumination unit and the imaging unit; The processing device according to claim 1 or 2, An optical inspection system comprising:

12. a color aperture provided between the subject and the imaging unit; the color aperture has at least two wavelength selective regions in the at least one plane that selectively pass light of different wavelength spectrums; the imaging unit is capable of receiving light of the different wavelength spectrums passing through the wavelength selection region, The processing device is capable of outputting defects on the surface of the subject based on the number of colors of the light of the wavelength spectrum received by the imaging unit. The optical inspection system of claim 11.

13. Illuminating a surface of a subject that moves relative to the illumination light, the illumination light being directed in different directions in at least one plane; capturing an image of the surface of the subject with an imaging unit, and obtaining a plurality of captured images at a plurality of times including a reference captured image; one or more captured images selected from the plurality of captured images are shifted and superimposed on the reference captured image to construct at least one object surface image; 23. A method for constructing an object surface image, comprising:

14. Illuminating a surface of a subject that moves relative to the illumination light, the illumination light being directed in different directions in at least one plane; The surface of the subject is imaged at a predetermined frame rate by an imaging unit to obtain a reference image, and the imaged images following the reference image are shifted and superimposed on the imaged images taken before the reference image to form at least one image of the surface of the subject.

23. A method for constructing an object surface image, comprising:

15. When constructing the at least one surface image of the subject, a shift amount of at least a plurality of the captured images is set based on a frame rate of the imaging unit, a transport speed of the subject relative to the imaging unit, and a pixel count of the imaging unit in a direction along a transport direction of the subject relative to the imaging unit.

15. A method according to claim 13 or claim 14.

16. 15. A method according to claim 13 or claim 14, further comprising subjecting the at least one object surface image to image processing for defect identification.

17. Illuminating a surface of a subject that moves relative to the illumination light, the illumination light being directed in different directions in at least one plane; capturing an image of the surface of the subject with an imaging unit, and obtaining a plurality of captured images at a plurality of times including a reference captured image; performing image processing for defect identification on the plurality of captured images to generate a defect identification converted image of the plurality of captured images at the plurality of times; one or more defect identification converted images selected from the plurality of defect identification converted images are shifted and superimposed on a reference defect identification converted image corresponding to the reference captured image, thereby forming at least one defect identification image; A method for constructing a defect identification image, comprising:

18. A program for constructing a surface image of a subject, the program causing one or more processors to execute a process of shifting one or more captured images selected from a plurality of captured images taken at multiple times, including a reference captured image, obtained by an imaging unit when illumination light directed in different directions within at least one plane is irradiated onto the surface of a subject moving relatively to the illumination light, and constructing at least one surface image of the subject.

19. A program for constructing a surface image of a subject, the program causing one or more processors to execute a process of obtaining a reference captured image at a predetermined frame rate by an imaging unit when illumination light directed in different directions within at least one plane is irradiated onto a surface of a subject moving relatively to the illumination light, and shifting each captured image subsequent to the reference captured image and superimposing it on an captured image captured prior to the reference captured image, thereby constructing at least one surface image of the subject.

20. Illuminating a surface of a subject with illumination lights each directed in a different direction within at least one plane; capturing an image of the surface of the subject with an imaging unit to obtain a captured image, and identifying a position of a pixel on which light is incident in the captured image; estimating an inclination angle of the surface of the subject shown in the captured image based on the position of the pixel onto which the light is incident; A method for estimating an angle of a surface of an object, comprising:

21. A program for estimating the inclination angle of a surface of a subject, the program causing one or more processors to execute a process of estimating the inclination angle of the surface of the subject shown in an image obtained by an imaging unit when the surface of the subject is irradiated with illumination light directed in different directions within at least one plane, based on the position of a pixel into which light is incident.

Images