Inspection device and inspection method

Abstract

To determine lighting conditions and image processing parameters that allow defects on the surface of an object being inspected to be detected.SOLUTION: An image processing device displays on a display device original images of an object to be inspected, illuminated under each of a plurality of mutually different lighting conditions, and based on first input data from an input device, displays on the display device an original image that provides the best appearance of a defect image showing defects present on the surface of the object to be inspected from the plurality of original images displayed on the display device, and based on second input data from the input device, extracts good areas that do not contain defect images and defective areas that include defect images from the original image that provides the best appearance of the defect image, compares the good areas with the defective areas, extracts defect images from the defective areas, sequentially changes image processing parameters that specify the conditions at the time of image processing to perform image processing on the good areas and defective areas, and determines a lighting condition and an image processing parameter that provide the best appearance of the defect image in the image-processed defective areas.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The technology disclosed in this specification relates to an inspection device and an inspection method. [Background technology]

[0002] As disclosed in Patent Document 1, a visual inspection of an industrial product is carried out in the manufacturing process of the industrial product. In the visual inspection, the industrial product is imaged by an imaging device. The grayscale image of the industrial product captured by the imaging device is subjected to image processing to detect defects on the surface of the industrial product. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-024482 Summary of the Invention [Problem to be solved by the invention]

[0004] If the lighting conditions when an industrial product is imaged with an imaging device change, or if the image processing parameters that define the conditions for processing the grayscale image of the industrial product imaged with the imaging device change, the appearance of the defect in the image may change. In other words, depending on the lighting conditions or image processing parameters, the defect may or may not be detected.

[0005] When the user of the inspection device determines the illumination conditions and image processing parameters that enable defect detection through experience or trial and error, it is difficult for an unskilled user to determine the optimal illumination conditions and image processing parameters.

[0006] The techniques disclosed herein aim to determine lighting conditions and image processing parameters that will allow for the detection of defects on the surface of an object to be inspected. [Means for solving the problem]

[0007] This specification discloses an inspection apparatus including an illumination device that illuminates an object to be inspected, an imaging device that images the object to be inspected illuminated by the illumination device, a display device that provides display data to a user, an input device that generates input data when operated by the user, and an image processing device having a processor. The image processing device displays original images of the object to be inspected illuminated under each of a plurality of mutually different illumination conditions on the display device, and, based on first input data from the input device, displays on the display device an original image that provides the best visibility of a defect image showing a defect present on the surface of the object to be inspected from the plurality of original images displayed on the display device, extracts good areas that do not include a defect image and defective areas that include a defect image from the original image that provides the best visibility of the defect image, compares the good areas with the defective areas, extracts the defect images from the defective areas, sequentially changes image processing parameters that define conditions for image processing, and performs image processing on the good areas and the defective areas, and determines the illumination conditions and image processing parameters that provide the best visibility of the defect image in the image-processed defective areas. [Effects of the Invention]

[0008] The techniques disclosed herein make it possible to determine lighting conditions and image processing parameters that allow for the detection of defects on the surface of an object to be inspected. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram schematically illustrating an inspection device according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an inspection device according to an embodiment. [Figure 3] FIG. 3 is a flowchart illustrating an example of a defect detection method according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating the expansion process and the contraction process according to the embodiment. [Figure 5] FIG. 5 is a flowchart showing an example of a method for determining illumination conditions and image processing parameters according to the embodiment. [Figure 6] FIG. 6 is a diagram schematically illustrating an example of a display device according to the embodiment that displays original images of an inspection object captured under a plurality of different illumination conditions. [Figure 7] FIG. 7 is a diagram schematically illustrating an example of a display device that displays an original image selected by a user according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating the image processing parameter determination process performed by the image processing device according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating the image processing parameter determination process performed by the image processing device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment will be described with reference to the drawings. In the embodiment, a local coordinate system, which is an XYZ Cartesian coordinate system, is defined for the inspection device 1, and the positional relationship of each part will be described with reference to the local coordinate system. The direction parallel to the X axis in a horizontal plane is defined as the X axis direction. The direction parallel to the Y axis in the horizontal plane, which is perpendicular to the X axis, is defined as the Y axis direction. The direction parallel to the Z axis, which is perpendicular to the horizontal plane, is defined as the Z axis direction. A plane including the X axis and the Y axis will be referred to as the XY plane as appropriate. The XY plane is parallel to the horizontal plane. The Z axis is parallel to a vertical line. The Z axis direction is the up-down direction. The +Z direction is the upward direction, and the -Z direction is the downward direction.

[0011] [Inspection equipment] 1 is a diagram schematically illustrating an inspection device 1 according to an embodiment. The inspection device 1 is an appearance inspection device that inspects the appearance of an inspection object W. As shown in FIG. 1, the inspection device 1 includes a table 2, an imaging device 3, and an illumination device 4.

[0012] The table 2 supports an inspection object W. The inspection object W includes an industrial product. The industrial product includes a metal product. The industrial product may be a part, a work in progress, or a finished product. An example of an industrial product is an engine cylinder block.

[0013] The imaging device 3 images the inspection object W supported on the table 2. The imaging device 3 images the inspection object W from above. The imaging device 3 has an imaging optical system 5 and an image sensor 6. The imaging optical system 5 has an optical element. The optical axis AX of the imaging optical system 5 is parallel to the Z axis. The image sensor 6 acquires an image of the inspection object W via the imaging optical system 5. Examples of the image sensor 6 include a CCD (Couple Charged Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0014] The illumination device 4 illuminates the inspection object W supported on the table 2 with illumination light. The imaging device 3 captures an image of the inspection object W illuminated by the illumination device 4.

[0015] In the embodiment, the illumination device 4 has a coaxial illumination unit 7 and a side oblique illumination unit 8. The coaxial illumination unit 7 coaxially illuminates the inspection object W. Coaxial illumination refers to an illumination method in which illumination light is irradiated onto the inspection object W from a direction parallel to the optical axis AX. The side oblique illumination unit 8 illuminates the inspection object W from a side oblique direction. Side oblique illumination refers to an illumination method in which illumination light is irradiated onto the inspection object W from outside the optical axis AX.

[0016] The coaxial illumination unit 7 has a light source 9 and a half mirror 10. The light source 9 emits illumination light. An example of the light source 9 is a light emitting diode (LED). The light source 9 emits white light as the illumination light. At least a portion of the half mirror 10 is arranged on the optical axis AX. In the embodiment, the half mirror 10 is arranged inside the imaging optical system 5. The illumination light emitted from the light source 9 is reflected by the half mirror 10 and irradiated onto the inspection object W. The illumination light reflected by the inspection object W passes through the imaging optical system 5 and the half mirror 10, and then enters the image sensor 6.

[0017] The side oblique illumination unit 8 includes a light source 11 and a support member 12. The light source 11 emits illumination light. An example of the light source 11 is a light emitting diode (LED). The light source 11 emits white light as illumination light. The light source 11 is ring-shaped and arranged to surround the optical axis AX. In the embodiment, the side oblique illumination unit 8 performs ring illumination, which is a type of side oblique illumination. Ring illumination refers to an illumination method in which illumination light is irradiated onto the inspection object W from the ring-shaped light source 11 arranged to surround the optical axis AX. The optical axis AX of the imaging optical system 5 is arranged inside the annular light source 11. The support member 12 supports the light source 11. The support member 12 is arranged around the light source 11. The upper end of the support member 12 is fixed to the imaging device 3. The lower end of the support member 12 is arranged below the light source 11.

[0018] The side oblique illumination unit 8 has a plurality of light sources 11. The light sources 11 include a first light source 11A having a first inner diameter, a second light source 11B having a second inner diameter larger than the first inner diameter, and a third light source 11C having a third inner diameter larger than the second inner diameter. Of the plurality of light sources 11, the first light source 11A is disposed at a position farthest from the table 2, the second light source 11B is disposed at a position next farthest from the table 2 after the first light source 11A, and the third light source 11C is disposed at a position closest to the table 2.

[0019] The illumination device 4 illuminates the inspection object W under each of a plurality of illumination conditions that are different from one another. The illumination conditions include an incident angle θ of the illumination light incident on the inspection object W. The incident angle θ0 at which the illumination light emitted from the light source 9 of the coaxial illumination unit 7 is incident on the inspection object W, the incident angle θ1 at which the illumination light emitted from the first light source 11A is incident on the inspection object W, the incident angle θ2 at which the illumination light emitted from the second light source 11B is incident on the inspection object W, and the incident angle θ3 at which the illumination light emitted from the third light source 11C is incident on the inspection object W are all different.

[0020] The illumination device 4 irradiates the inspection object W with illumination light at each of a plurality of incident angles θ. When illumination light is emitted from the light source 9 of the coaxial illumination unit 7, illumination light is not emitted from each of the first light source 11A, the second light source 11B, and the third light source 11C. When illumination light is emitted from the first light source 11A, illumination light is not emitted from each of the second light source 11B, the third light source 11C, and the light source 9. When illumination light is emitted from the second light source 11B, illumination light is not emitted from each of the third light source 11C, the light source 9, and the first light source 11A. When illumination light is emitted from the third light source 11C, illumination light is not emitted from each of the light source 9, the first light source 11A, and the second light source 11B.

[0021] In the embodiment, there are four illumination conditions: a first illumination condition in which the inspection object W is illuminated with illumination light emitted from the light source 9, a second illumination condition in which the inspection object W is illuminated with illumination light emitted from the first light source 11A, a third illumination condition in which the inspection object W is illuminated with illumination light emitted from the second light source 11B, and a fourth illumination condition in which the inspection object W is illuminated with illumination light emitted from the third light source 11C.

[0022] [Image processing device] 2 is a block diagram showing the inspection device 1 according to the embodiment. As shown in FIG. 2, the inspection device 1 includes an image processing device 13, an imaging device 3, a display device 14, and an input device 15.

[0023] The image processing device 13 includes a computer. The image processing device 13 has a processor 16, a main memory 17, a storage 18, an input / output interface 19, and a communication interface 20.

[0024] The processor 16 includes a CPU (Central Processing Unit). The processor 16 may also include a GPU (Graphics Processing Unit).

[0025] The main memory 17 includes a non-volatile memory such as a read only memory (ROM) and a volatile memory such as a random access memory (RAM).

[0026] The storage 18 is a recording medium that records computer programs and data in a manner that can be read by the processor 16. Examples of the storage 18 include a hard disk or a semiconductor memory. In this embodiment, the storage 18 records an image processing program 21 and an operating system 22 as computer programs.

[0027] The processor 16 reads the image processing program 21 from the storage 18, loads it into the main memory 17, and executes processing in accordance with the image processing program 21. The image processing program 21 provides the functions of the image processing device 13.

[0028] The input / output interface 19 includes an input / output circuit. The imaging device 3, the display device 14, and the input device 15 are each connected to the input / output interface 19. The image processing device 13 acquires an image from the imaging device 3 via the input / output interface 19. The image processing device 13 transmits display data to the display device 14 via the input / output interface 19. The image processing device 13 acquires input data from the input device 15 via the input / output interface 19.

[0029] The communication interface 20 communicates via a communication network. An example of the communication interface 20 is a local area network (LAN) module. The image processing device 13 transmits data to an external computer via the communication interface 20 and the communication network. The image processing device 13 receives data from the external computer via the communication network and the communication interface 20. A computer program may be distributed to the image processing device 13 via the communication network.

[0030] The display device 14 provides display data to the user. Examples of the display device 14 include a flat panel display such as a liquid crystal display or an organic EL display.

[0031] The input device 15 generates input data when operated by a user. The input data generated by the input device 15 is transmitted to the image processing device 13. Examples of the input device 15 include a computer keyboard, a mouse, a button, a switch, and a touch panel.

[0032] [Defect detection method] The image processing device 13 detects defects on the surface of the inspection object W by performing image processing on the image of the inspection object W captured by the imaging device 3. Examples of defects on the surface of the inspection object W include scratches, stains, dirt, and dents that exist on the surface of the inspection object W. The image of the inspection object W captured by the imaging device 3 is transmitted to the image processing device 13 via the input / output interface 19. In the following description, the image of the inspection object W captured by the imaging device 3 will be referred to as the original image Ma as appropriate.

[0033] In the embodiment, the original image Ma of the inspection object W captured by the imaging device 3 is a grayscale image (monochrome image). The image processing device 13 performs image processing on the original image Ma of the inspection object W captured by the imaging device 3, thereby detecting defects on the surface of the inspection object W.

[0034] 3 is a flowchart showing an example of a defect detection method according to an embodiment. An inspection object W supported on a table 2 is illuminated by an illumination device 4 under predetermined illumination conditions. An imaging device 3 captures an image of the inspection object W illuminated under the predetermined illumination conditions. An image processing device 13 acquires an original image Ma of the inspection object W captured by the imaging device 3 (step SA1).

[0035] Next, the image processing device 13 performs binarization processing on the original image Ma acquired in step SA1 to generate a binary image Mb (step SA2).

[0036] Binarization is an image process that converts the original image Ma to be analyzed into a binary image Mb with only two values ​​(two colors): 1 (white) and 0 (black). If a defect exists on the surface of the inspection object W, binarization makes clear the boundary between the defect image Pa showing the defect and the background image Pb showing the surface of the inspection object W around the defect.

[0037] The image processing device 13 compares a predetermined threshold value H with the brightness value v of each of the pixels constituting the original image Ma, and converts the brightness value v of each of the pixels to 1 (white) or 0 (black). In the embodiment, the threshold value H is set to a lower limit value H min and upper limit H max The image processing device 13 determines whether the luminance value v of a pixel in the original image Ma is [H min ≦ v ≦ H max ] condition is satisfied, the luminance value v of the pixel is converted to 1 (white), and [H min ≦v ≦H max ] is not satisfied, the luminance value v of the pixel is converted to 0 (black).

[0038] Next, the image processing device 13 performs a noise removal process to remove noise from the binary image Mb generated in step SA2. In the embodiment, the image processing device 13 performs an opening process as the noise removal process (step SA3).

[0039] Even if the threshold H in the binarization process is optimally set, noise may still be present in the binary image Mb. There are two types of noise in the binary image Mb: missing portions, where a portion that should have been a defect image Pa has become the background image Pb, and over-detected portions, where a portion that should have been a background image Pb has become the defect image Pa. The shape of the noise portion is often thin and whisker-like. The area of ​​the noise portion is often small, like dust. In the embodiment, the image processing device 13 performs an opening process, which combines an expansion process and an erosion process, as a noise removal process. The opening process refers to a process in which an erosion process is performed multiple times on the binary image Mb, and then an expansion process is performed the same number of times as the erosion process. The image processing device 13 may also perform a closing process. The closing process refers to a process in which an expansion process is performed multiple times on the binary image Mb, and then an erosion process is performed the same number of times as the expansion process.

[0040] FIG. 4 is a diagram illustrating expansion processing and contraction processing according to an embodiment. FIG. 4 schematically shows pixels of a binary image Mb. As shown in FIG. 4, expansion processing refers to processing in which, when at least one of multiple surrounding pixels Y present around the target pixel X is a white pixel, the target pixel X is converted to a white pixel. Contraction processing refers to processing in which, when at least one of multiple surrounding pixels Y present around the target pixel X is a black pixel, the target pixel X is converted to a black pixel.

[0041] The surrounding pixels Y are pixels present in a matrix-shaped determination range of m rows and n columns (m×n) centered on the target pixel X. The size of the determination range may be a 3×3 range or a 5×5 range centered on the target pixel X. The size of the determination range is determined in advance. In the following description, the size of the determination range in the opening process will be referred to as the opening process pixel number Np, as appropriate.

[0042] The number of pixels Np for opening processing indicates the number of rows and columns of pixels in the determination range. As shown in Figure 4, if the size of the determination range is 3 x 3, the number of pixels Np for opening processing is 3 pixels. If the size of the determination range is 5 x 5, the number of pixels Np for opening processing is 5 pixels.

[0043] The image processing device 13 detects the presence or absence of defects on the surface of the inspection object W based on the processed image representing the binary image Mb that has been subjected to the opening process. Furthermore, if the image processing device 13 determines that there is a defect on the surface of the inspection object W, it detects the degree of the defect. The degree of the defect includes the size and shape of the defect (step SA4).

[0044] [How to determine lighting conditions and image processing parameters] As described above, the image processing device 13 performs binarization processing and opening processing on the original image Ma of the inspection object W captured by the imaging device 3 to detect defects.

[0045] If the illumination conditions when the inspection object W is imaged change, the appearance of the defect in the original image Ma may change. If the intensity of the illumination light or the angle of incidence of the illumination light changes, the minimum width or area of ​​the detectable defect may change. In other words, if the illumination conditions when the inspection object W is imaged are inappropriate, it may be difficult to detect the defect with high accuracy.

[0046] Furthermore, there is a possibility that the appearance of defects in the binary image Mb will change if there is a change in the image processing parameters that define the conditions for image processing of the original image Ma captured by the imaging device 3. The image processing parameters include a threshold H in the binarization process and a number Np of opening process pixels in the opening process.

[0047] If the threshold value H changes, the defect image Pa in the binary image Mb may become blurred or smeared. In other words, if the threshold value H is inappropriate, it may become difficult to detect defects with high accuracy.

[0048] If the number of opening process pixels Np changes, a part that should be the defect image Pa may become the background image Pb, or a part that should be the background image Pb may become the defect image Pa. In other words, if the number of opening process pixels Np is inappropriate, it may be difficult to detect defects with high accuracy.

[0049] In the embodiment, the image processing device 13 determines optimal illumination conditions when capturing an image of the inspection object W. The image processing device 13 also determines optimal image processing parameters including a threshold value H in the binarization process and the number of opening process pixels Np in the opening process. The method for determining the illumination conditions and image processing parameters will be described below.

[0050] FIG. 5 is a flowchart showing an example of a method for determining illumination conditions and image processing parameters according to the embodiment.

[0051] The image processing device 13 acquires original images Ma of the inspection object W captured under each of a plurality of mutually different illumination conditions. The image processing device 13 stores the original images Ma of the inspection object W captured under each of the plurality of mutually different illumination conditions. The image processing device 13 displays the original images Ma of the inspection object W captured under each of the plurality of mutually different illumination conditions on the display device 14 (step SB1).

[0052] 6 is a diagram schematically illustrating an example of a display device 14 that displays an original image Ma of an inspection object W captured under a plurality of different illumination conditions according to an embodiment. FIG. 6 shows an example in which a linear scratch exists as a defect on the surface of the inspection object W.

[0053] As described above, in the embodiment, there are four illumination conditions. Fig. 6 shows an example in which a first original image Ma1 showing an original image Ma of the inspection object W illuminated under the first illumination condition, a second original image Ma2 showing an original image Ma of the inspection object W illuminated under the second illumination condition, a third original image Ma3 showing an original image Ma of the inspection object W illuminated under the third illumination condition, and a fourth original image Ma4 showing an original image Ma of the inspection object W illuminated under the fourth illumination condition are displayed on the display device 14.

[0054] As shown in FIG. 6, when a defect exists on the surface of the inspection object W, the appearance of the defect varies depending on the illumination conditions. Due to differences in the light reflectance and shape of the surface of the inspection object W, when illumination light is irradiated onto the surface of the inspection object W where a defect exists, there are illumination conditions under which the defect appears well and illumination conditions under which the defect appears poorly. In the example shown in FIG. 6, under the first illumination condition, a first defect image Pa1 with a narrow line width is acquired as the defect image Pa showing the defect in the original image Ma. Under the second illumination condition, a second defect image Pa2 with a line width wider than that of the first defect image Pa1 is acquired as the defect image Pa. Under the third illumination condition, a third defect image Pa3 with a line width wider than that of the second defect image Pa2 is acquired as the defect image Pa. Under the fourth illumination condition, a fourth defect image Pa4 with a line width equal to that of the third defect image Pa3 is acquired as the defect image Pa. However, the brightness ratio between the fourth defect image Pa4 and the background image Pb is lower than the brightness ratio between the third defect image Pa3 and the background image Pb.

[0055] The user selects the original image Ma that best shows the defect image Pa from among the multiple (four) original images Ma (Ma1, Ma2, Ma3, Ma4) displayed on the display device 14. In the embodiment, the user selects the third original image Ma3 that includes the third defect image Pa3 that has the thickest line width and the largest contrast ratio with the background image Pb.

[0056] 6, in the embodiment, the image processing device 13 causes the display device 14 to display first guidance data 31 for prompting the user to select an original image Ma in which the defect image Pa is best visible from among a plurality of original images Ma (Ma1, Ma2, Ma3, Ma4) displayed on the display device 14. In the example shown in FIG. 6, the first guidance data 31 includes text data saying, "Please select the grayscale image in which the defect is most easily visible."

[0057] To select the third original image Ma3, the user operates the input device 15. If the input device 15 includes a touch sensor arranged on the display screen of the display device 14, the user taps on the display area of ​​the display screen of the display device 14 where the third original image Ma3 is displayed.

[0058] The input data (first input data) generated by operating the input device 15 is transmitted to the image processing device 13. The image processing device 13 acquires input data (first input data) indicating an original image Ma (third original image Ma3) selected by the user from among a plurality of original images Ma (Ma1, Ma2, Ma3, Ma4) of the inspection object W illuminated under each of a plurality of mutually different illumination conditions (step SB2).

[0059] 7 is a diagram schematically illustrating an example of a display device 14 that displays an original image Ma selected by a user according to an embodiment. As shown in FIG. 7, the image processing device 13 displays a third original image Ma3 selected by the user among four original images Ma (Ma1, Ma2, Ma3, Ma4) on the display device 14. That is, based on the input data (first input data) acquired from the input device 15 in step SB2, the image processing device 13 displays on the display device 14 an original image Ma (third original image Ma3) that best shows the defect image Pa, which shows a defect present on the surface of the inspection object W, among the multiple original images Ma (Ma1, Ma2, Ma3, Ma4) displayed on the display device 14 (step SB3).

[0060] In the third original image Ma3 in which the defective image Pa looks the best, the user specifies a good area 41 that does not include the defective image Pa and a defective area 42 that includes the third defective image Pa3.

[0061] 7, in the embodiment, the image processing device 13 causes the display device 14 to display second guidance data 32 for allowing the user to specify a good area 41 and a defective area 42 in a third original image Ma3 in which the defective image Pa looks the best. In the example shown in FIG. 7, the second guidance data 32 includes text data saying, "Please specify a good area and a defective area."

[0062] The user operates the input device 15 to specify the good area 41 and the defective area 42. If the input device 15 includes a touch sensor arranged on the display screen of the display device 14, the user moves his / her finger in a circular pattern on a part of the background image Pb where the defective image Pa is not present on the display screen of the display device 14 to specify the annular good area 41 so as not to include the defective image Pa. The user also moves his / her finger in a circular pattern on the display screen of the display device 14 to surround the third defective image Pa3 to specify the annular defective area 42 so as to include the third defective image Pa3.

[0063] The input data (second input data) generated by operating the input device 15 is transmitted to the image processing device 13. The image processing device 13 acquires input data (second input data) indicating the good area 41 and the defective area 42 specified by the user in the third original image Ma3 in which the defective image Pa looks the best (step SB4).

[0064] The image processing device 13 can identify a good area 41 that does not include the defective image Pa and a defective area 42 that includes the third defective image Pa3 based on the input data (second input data) from the input device 15 acquired in step SB4.

[0065] The user operates the input device 15 to cause the image processing device 13 to start image processing of the good area 41 and the defective area 42. In the embodiment, the image processing of the good area 41 and the defective area 42 includes processing to determine lighting conditions and image processing parameters. The user operates the input device 15 to cause the image processing device 13 to start processing to determine lighting conditions and image processing parameters.

[0066] In the embodiment, the input device 15 for starting the process of determining the lighting conditions and image processing parameters includes an automatic setting button 33 displayed on the display screen of the display device 14. The user taps the automatic setting button 33. Input data (third input data) generated by operating the automatic setting button 33 is transmitted to the image processing device 13. The image processing device 13 acquires the input data (third input data) for starting the process of determining the lighting conditions and image processing parameters generated by operating the automatic setting button 33 (step SB5).

[0067] The image processing device 13 starts image processing of the good area 41 and the defective area 42 based on the input data (third input data) from the input device 15 acquired in step SB5. That is, the image processing device 13 starts processing to determine image processing parameters based on the input data (third input data) from the input device 15.

[0068] 8 and 9 are diagrams illustrating the image processing parameter determination process performed by the image processing device 13 according to the embodiment. The image processing parameter determination process will be described below with reference to the flowchart in FIG. 5 and FIGS. 8 and 9.

[0069] The image processing device 13 extracts the good area 41 and the defective area 42 from the third original image Ma3 based on the input data (second input data) obtained from the input device 15 in step SB4 (step SB6).

[0070] Next, the image processing device 13 compares the good area 41 extracted in step SB6 with the defective area 42, and extracts a third defect image Pa3 from the defective area 42 (step SB7).

[0071] When extracting the third defective image Pa3 from the defective area 42, the image processing device 13 matches the size and shape of the good area 41 with the size and shape of the defective area 42, and associates the pixels of the good area 41 with the pixels of the defective area 42.

[0072] If the size of the good area 41 is smaller than the size of the defective area 42, the image processing device 13 enlarges the good area 41 so that the size of the good area 41 matches the size of the defective area 42. If the size of the good area 41 is larger than the size of the defective area 42, the image processing device 13 reduces the good area 41 so that the size of the good area 41 matches the size of the defective area 42.

[0073] 8 shows an example in which the size of a good area 41 specified by the user is smaller than the size of a defective area 42. The image processing device 13 enlarges the good area 41 specified by the user, indicated by a dotted leader line, to the good area 41 indicated by a solid leader line. The image processing device 13 enlarges the good area 41 so that the size of the good area 41 matches the size of the defective area 42.

[0074] Note that by enlarging the good area 41, the resolution of the good area 41 increases and the number of pixels in the good area 41 increases. The luminance value v of the pixels increased by enlarging the good area 41 may be determined by zero-order interpolation or linear interpolation. By reducing the good area 41, the resolution of the good area 41 decreases and the number of pixels in the good area 41 decreases. The luminance value v of the pixel combined by reducing the good area 41 may be determined by so-called thinning processing, or the average value of the luminance values ​​v of the multiple pixels to be combined may be used as the luminance value v of the combined pixel.

[0075] The image processing device 13 matches the size and shape of the good area 41 with the size and shape of the defective area 42, and then associates the pixels of the good area 41 with the pixels of the defective area 42. As an inter-pixel operation between the good area 41 and the defective area 42, the image processing device 13 performs a difference calculation between the brightness value v of the pixel in the good area 41 and the brightness value v of the pixel in the defective area 42 whose pixel values ​​(pixel positions) correspond to each other.

[0076] In the difference calculation, two extraction areas 43 are defined. Of the two extraction areas 43, one extraction area 43A (first extraction area) is defined in the good area 41 where the third defect image Pa3 does not exist, and the other extraction area 43B (second extraction area) is defined in the defective area 42 so as to surround the third defect image Pa3. The size and shape of the extraction area 43A are the same as those of the extraction area 43B. There is a one-to-one correspondence between the pixels of the extraction area 43A and the pixels of the extraction area 43B. The image processing device 13 extracts the defect image Pa (third defect image Pa3) from the defective area 42 based on the extraction area 43.

[0077] Image processing device 13 extracts third defect image Pa3 from defective area 42 by calculating the difference between luminance value v of pixels in extraction area 43A and luminance value v of pixels in extraction area 43B. As shown in Fig. 8, extraction area 43B for extracting third defect image Pa3 is slightly larger than third defect image Pa3. The edges of extraction area 43B are defined to surround third defect image Pa3.

[0078] In the following description, the image inside the extraction area 43A will be referred to as the good reference image Ra, and the image inside the extraction area 43B will be referred to as the defective reference image Rb. Furthermore, the good reference image Ra and the defective reference image Rb will be collectively referred to as the reference image R.

[0079] The good reference image Ra includes the background image Pb but does not include the defective image Pa. The defective reference image Rb includes the third defective image Pa3 and the background image Pb.

[0080] Next, the image processing device 13 extracts a partial image T from each of the original images Ma (Ma1, Ma2, Ma4) not selected by the user in step SB2, based on the extraction area 43 defined in step SB7 (step SB8).

[0081] 9, the partial image T includes a good partial image Ta extracted using the extraction area 43A and a defective partial image Tb extracted using the extraction area 43B. The good partial image Ta is an image of the original image Ma (Ma1, Ma2, Ma3, Ma4) that is inside the extraction area 43A. The defective partial image Tb is an image of the original image Ma (Ma1, Ma2, Ma3, Ma4) that is inside the extraction area 43B. The partial images T (Ta, Tb) are extracted from each of the first original image Ma1, the second original image Ma2, the third original image Ma3, and the fourth original image Ma4.

[0082] In a local coordinate system defined in the inspection device 1 (imaging device 3), the positions of the first original image Ma1, the second original image Ma2, the third original image Ma3, and the fourth original image Ma4 are the same. The pixel values ​​(pixel positions) of the good portion image Ta are the same in each of the first original image Ma1, the second original image Ma2, the third original image Ma3, and the fourth original image Ma4. The pixel values ​​(pixel positions) of the defective portion image Tb are the same in each of the first original image Ma1, the second original image Ma2, the third original image Ma3, and the fourth original image Ma4.

[0083] The good portion image Ta includes the background image Pb but does not include the defective image Pa. The defective portion image Tb includes the defective image Pa and the background image Pb. The good portion image Ta extracted from the third original image Ma3 is the good reference image Ra inside the extraction area 43A described with reference to FIG. 8. The defective portion image Tb extracted from the third original image Ma3 is the defective reference image Rb inside the extraction area 43B described with reference to FIG. 8.

[0084] The good partial image Ta is a part of the good area 41. The good area 41 includes the good partial image Ta. The defective partial image Tb is a part of the defective area 42. The defective area 42 includes the defective partial image Tb.

[0085] Next, the image processing device 13 performs binarization processing on each of the multiple (four) good portion images Ta and the multiple (four) defective portion images Tb generated in step SB8 (step SB9).

[0086] The image processing device 13 sequentially changes the threshold value H and performs binarization processing on the partial image T. The image processing device 13 sequentially changes the threshold value H and performs image processing on each of the multiple (four) good partial images Ta and the multiple (four) defective partial images Tb.

[0087] That is, the image processing device 13 compares the brightness value v of the pixel of the partial image T with each of a plurality of threshold values ​​H that are different from one another. As described above, in the embodiment, the threshold value H is set to a lower limit value H min and upper limit H max The image processing device 13 determines a lower limit H min and upper limit H max In the embodiment, the image processing device 13 performs binarization processing on the partial image T(Ta, Tb) by using each of a plurality of combinations of min and upper limit H max While changing the value of the normalized image, a plurality of (four) good portion images Ta and a plurality of (four) defective portion images Tb are binarized.

[0088] In the embodiment, the image processing device 13 sets the lower limit H min and upper limit H max The lower limit H min and upper limit H max Combination with [H min ,H max] has 325 patterns, namely, [0,10], [0,20], [0,30]...[0,250], [10,20], [10,30], [10,40]...[10,250], [20,30], [20,40], [20,50]...[20,250]...[230,240], [230,250], and [240,250]. Therefore, 325 patterns of binary images U are generated from one partial image T. As shown in FIG. 9, in this embodiment, there are four good partial images Ta and four defective partial images Tb. Therefore, 2600 patterns (325 x 8) of binary images U are generated.

[0089] In the following description, of the 2600 patterns of binary images U generated from the partial images T, the 1300 patterns of binary images U generated from the four good partial images Ta will be referred to as good binary images Ua, and the 1300 patterns of binary images U generated from the four defective partial images Tb will be referred to as defective binary images Ub.

[0090] The image processing device 13 compares the reference image R (Ra, Rb) generated in step SB7 with each of the multiple binary images U (Ua, Ub) generated in step SB9, and calculates a binary image U that satisfies the condition of overlapping heavily with the defective reference image Rb and overlapping less with the good reference image Ra (step SB10).

[0091] That is, the image processing device 13 calculates a combination of a good binary image Ua that has a low degree of match with the good reference image Ra generated in step SB7 and a defective binary image Ub that has a high degree of match with the defective reference image Rb generated in step SB7.

[0092] The defect reference image Rb is the image that best represents the appearance of the defect image Pb. If the defect reference image Rb and the defect binary image Ub match closely, the threshold H used to generate the defect binary image Ub can be considered to be a threshold H suitable for defect detection.

[0093] The image processing device 13 determines a threshold value H (lower limit H) when generating a binary image U that satisfies the conditions of overlapping a large amount with the defective reference image Rb and a small amount with the good reference image Ra. min and upper limit H max and the illumination conditions under which the partial image T of the binary image U before binarization processing was captured are determined (step SB11).

[0094] The illumination conditions and threshold value H acquired in step SB11 are appropriate illumination conditions and threshold value H for detecting defects. That is, the illumination conditions and threshold value H acquired in step SB11 are the illumination conditions and threshold value H that make the defect image Pa look the best in the defective portion image Tb. The threshold value H acquired in step SB11 is the threshold value H acquired by changing the threshold value in increments of 10. In step SB11, the appropriate threshold value H is roughly determined.

[0095] The image processing device 13 performs binarization processing on the partial image T before binarization processing of the binary image U calculated in step SB10, by changing the threshold value H in increments of 1 (step SB12).

[0096] For example, the lower limit H determined in step SB9 min and upper limit H max If the combination is [90,100], the image processing device 13 sets the lower limit H min and upper limit H max The combination of [90,100], [91,100], [92,100]...[99,100] is changed to the lower limit H min and upper limit H max The combination of is binarized.

[0097] Furthermore, the image processing device 13 performs opening processing on the binary image U obtained under the lighting conditions and threshold value H acquired in step SB11, by changing the number of pixels for opening processing (step SB13).

[0098] The image processing device 13 changes the number of pixels for opening processing to 0, 3, and 5. The number of pixels for opening processing being 0 means that opening processing is not performed. By performing opening processing while changing the number of pixels for opening processing, the degree of noise removal in the binary image U acquired in step SB12 changes.

[0099] The image processing device 13 compares each of the multiple binary images U obtained by changing the threshold value H in increments of 1 with the good binary image Ua and the defective binary image Ub after opening processing with mutually different numbers of opening processing pixels, with the good reference image Ra and the defective reference image Rb, and determines the illumination conditions, threshold value H, and number of opening processing pixels when a binary image U is generated that satisfies the conditions of having a large overlap with the defective reference image Rb and a small overlap with the good reference image Ra (step SB14).

[0100] That is, the image processing device 13 calculates a combination of a good binary image Ua that has a low degree of match with the good reference image Ra generated in step SB7 and a defective binary image Ub that has a high degree of match with the defective reference image Rb generated in step SB7.

[0101] The defect reference image Rb is the image that best represents the appearance of the defect image Pb. If the defect reference image Rb and the defect binary image Ub match closely, the threshold H used to generate the defect binary image Ub can be considered to be a threshold H suitable for defect detection.

[0102] The image processing device 13 determines a threshold value H (lower limit H) when generating a binary image U that satisfies the conditions of overlapping a large amount with the defective reference image Rb and a small amount with the good reference image Ra. min and upper limit H max and the illumination conditions under which the partial image T of the binary image U before binarization processing was captured are determined.

[0103] As described above, in step SB11, an appropriate threshold value H is roughly determined. The threshold value H determined in step SB14 is the threshold value H obtained by changing the threshold value in increments of 1. The illumination conditions and threshold value H determined in step SB14 are the illumination conditions and image parameters (threshold value H and opening processing pixel number Np) that provide the best visibility of the defect image Pa in the binarized defective portion image Tb (defect area 42).

[0104] The image processing device 13 causes the display device 14 to display the illumination conditions, the threshold value H for binarization processing, and the number of pixels Np for opening processing determined in step SB14 (step SB15).

[0105] The user can confirm the illumination conditions and image processing parameters (threshold H and number of pixels Np for opening processing) determined by the image processing device 13. After confirmation, the user may operate the input device 15 to fine-tune the image processing parameters.

[0106] [effect] As described above, according to the embodiment, the image processing device 13 displays on the display device 14 original images Ma of the inspection object W illuminated under each of a plurality of mutually different illumination conditions (step SB1), and based on the first input data from the input device 15, displays on the display device 14 the original image Ma in which the defect image Pa showing the defect present on the surface of the inspection object W is best viewed from the plurality of original images Ma displayed on the display device 14 (step SB3), and based on the second input data from the input device 15, selects the good area 41 not including the defect image Pa and the defect image Pa from the original image Ma in which the defect image Pa is best viewed. The good area 41 and the defective area 42 containing the image Pa are extracted (step SB6), the good area 41 and the defective area 42 are compared, and the defective image Pa is extracted from the defective area 42 (step SB7). The image processing parameters (threshold H and number of opening processing pixels Np) that define the conditions for image processing are sequentially changed to image process the good area 41 and the defective area 42 (steps SB9, SB12, SB13), and the lighting conditions and image processing parameters (threshold H and number of opening processing pixels Np) that give the best appearance to the defective image Pa in the image-processed defective area 42 are determined (step SB14).

[0107] According to the above configuration, the user selects the original image Ma in which the defect image Pa is best visible from among the multiple original images Ma (Ma, Mb, Mc, Md) displayed on the display device 14, as described with reference to Fig. 6, and then, in the third original image Ma3 in which the defect image Pa is best visible, specifies a good area 41 that does not include the defect image Pa and a defective area 42 that includes the defect image Pa, as described with reference to Fig. 7. After that, the user simply taps the automatic setting button 33, and the image processing device 13 outputs the illumination conditions and image processing parameters that provide the best visibility of the defect image Pa. This allows even an unskilled user to determine the optimal illumination conditions and image processing parameters for detecting defects on the surface of the inspection object W.

[0108] In an embodiment, the image processing device 13 defines an extraction area 43A (first extraction area) in the good area 41, defines an extraction area 43B (second extraction area) in the defective area 42 so as to surround the third defective image Pa3, and performs a differential calculation between the brightness value v of the pixel in the good area 41 and the brightness value v of the pixel in the defective area 42 to extract the third defective image Pa3 from the defective area 42.

[0109] According to the above configuration, the third defect image Pa3 can be extracted based on the good area 41 and the defective area 42 specified by the user.

[0110] In an embodiment, the image processing device 13 extracts a good portion image Ta, which is an image inside the extraction area 43A (first extraction area), from each of the multiple original images Ma (Ma1, Ma2, Ma3, Ma4), extracts a defective portion image Tb, which is an image inside the extraction area 43B (second extraction area), from each of the multiple original images Ma (Ma1, Ma2, Ma3, Ma4), and sequentially changes image processing parameters to image process each of the good portion image Ta and the defective portion image Tb.

[0111] According to the above configuration, the good portion image Ta, which is part of the good area 41, is image processed, and the defective portion image Tb, which is part of the defective area 42, is image processed. Since the good area 41 and the defective area 42 are each specified by the user, there is a possibility that the shapes, sizes, etc. of the good area 41 and the defective area 42 are not suitable for image processing. By image processing the good portion image Ta, which is part of the good area 41, and the defective portion image Tb, which is part of the defective area 42, the calculation load of the image processing is reduced, and the image processing is performed appropriately.

[0112] In an embodiment, the image processing device 13 displays first guidance data 31 on the display device 14 to allow the user to select the original image Ma that best shows the defect image Pa from among the multiple original images Ma displayed on the display device 14.

[0113] According to the above configuration, the user can select the original image Ma that best shows the defect image Pa in accordance with the first guidance data 31.

[0114] In the embodiment, the image processing device 13 causes the display device 14 to display second guidance data 32 for allowing the user to specify the good area 41 and the defective area 42 in the original image Ma in which the defective image Pa looks the best.

[0115] According to the above configuration, the user can specify the good area 41 and the defective area 42 in accordance with the second guidance data 32.

[0116] In the embodiment, the image processing device 13 starts image processing of the good area 41 and the defective area 42 based on the third input data from the input device 15 .

[0117] According to the above configuration, when the automatic setting button 33 is operated, the image processing device 13 starts the process of determining the lighting conditions and image processing parameters.

[0118] [Other embodiments] In the above-described embodiment, the number of illumination conditions is four. However, the number of illumination conditions may be two, three, or any number of five or more.

[0119] In the above-described embodiment, the original image of the inspection object W captured by the imaging device 3 is a grayscale image (monochrome image). The original image of the inspection object W may also be a color image. [Explanation of symbols]

[0120] 1...inspection device, 2...table, 3...imaging device, 4...illumination device, 5...imaging optical system, 6...image sensor, 7...coaxial illumination unit, 8...side oblique illumination unit, 9...light source, 10...half mirror, 11...light source, 11A...first light source, 11B...second light source, 11C...third light source, 12...support member, 13...image processing device, 14...display device, 15...input device, 16...processor, 17...main memory, 18...storage, 19...input / output interface, 20...communication interface, 21...image processing program, 22...operating system, 31...first guidance data, 32...second guidance data, 33...automatic setting button, 41...good area, 42...defective area, 43...extracted area, 43A...extracted area (first extracted area), 43B...extracted area (second extracted area), H...threshold, H min …Lower limit, H max ...upper limit, Ma...original image, Ma1...first original image, Ma2...second original image, Ma3...third original image, Ma4...fourth original image, Mb...binary image, Np...number of pixels for opening processing, Pa...defect image, Pa1...first defect image, Pa2...second defect image, Pa3...third defect image, Pa4...fourth defect image, Pb...background image, R...reference image, Ra...good reference image, Rb...defect reference image, T...partial image, Ta...good partial image, Tb...defective partial image, U...binary image, Ua...good binary image, Ub...defective binary image, W...inspection object.

Claims

1. an illumination device that illuminates the inspection object; an imaging device that images the inspection object illuminated by the illumination device; a display device for providing display data to a user; an input device that generates input data when operated by the user; an image processing device having a processor, The image processing device includes: displaying, on the display device, original images of the object illuminated under each of a plurality of mutually different illumination conditions; displaying, on the display device, an original image that shows the best appearance of a defect image that indicates a defect present on the surface of the inspection object, among the plurality of original images displayed on the display device, based on first input data from the input device; extracting a good area not including the defect image and a defect area including the defect image from an original image in which the defect image is most visible based on second input data from the input device; comparing the good area with the defective area to extract the defect image from the defective area; image processing parameters that define conditions for image processing are sequentially changed to perform image processing on the good area and the defective area; determining lighting conditions and image processing parameters that provide the best defect image appearance in the image-processed defect area; Inspection equipment.

2. The image processing device includes: defining a first extraction area in the good area; defining a second extraction area in the defect area so as to surround the defect image; calculating a difference between the luminance values ​​of pixels in the good area and the luminance values ​​of pixels in the defective area to extract the defect image from the defective area; The inspection device according to claim 1 .

3. The image processing device includes: extracting good portion images that are images inside the first extraction area from each of the original images of the inspection object illuminated under each of a plurality of mutually different illumination conditions; extracting a defective portion image, which is an image inside the second extraction area, from each of the original images of the inspection object illuminated under each of a plurality of mutually different illumination conditions; sequentially changing the image processing parameters to perform image processing on each of the good portion image and the defective portion image; The inspection device according to claim 2 .

4. The image processing device includes: displaying, on the display device, first guidance data for allowing the user to select an original image in which the defect image is best visible from among the plurality of original images displayed on the display device; The inspection device according to claim 1 .

5. The image processing device includes: displaying, on the display device, second guidance data for allowing the user to specify the good area and the defective area in the original image in which the defect image is most visible; The inspection device according to claim 1 .

6. The image processing device includes: starting image processing of the good area and the defective area based on third input data from the input device; The inspection device according to claim 1 .

7. The image processing parameters include a threshold value in binarization processing and a number of opening processing pixels in opening processing. The inspection device according to claim 1 .

8. an image processing device having a processor, displaying on a display device original images of the object of inspection illuminated under each of a plurality of mutually different illumination conditions; displaying, on the display device, an original image that shows the best appearance of a defect image that indicates a defect present on the surface of the inspection object, among the plurality of original images displayed on the display device, based on first input data from an input device; extracting a good area not including the defect image and a defect area including the defect image from an original image in which the defect image is most visible based on second input data from the input device; comparing the good area with the defective area to extract the defect image from the defective area; image processing parameters that define conditions for image processing are sequentially changed to perform image processing on the good area and the defective area; determining lighting conditions and image processing parameters that provide the best defect image appearance in the image-processed defect area; Testing method.

Images