Image processing system, setting method and recording medium

Abstract

The image processing system comprises: a selection unit that selects one target algorithm from a plurality of evaluation algorithms; a calculation unit that calculates, for each lighting pattern for evaluation, an evaluation value indicating suitability for image measurement using the target algorithm and at least one evaluation image corresponding to the lighting pattern for evaluation; and a mode determination unit that determines a lighting pattern for image measurement from a plurality of lighting patterns for evaluation based on the evaluation value. Thus, it is possible to more easily perform lighting setting of illumination in a manner suitable for image measurement using an appearance image of an object.

Description

Technical Field

[0001] The present disclosure relates to an image processing system, a setting method, and a recording medium. Background Art

[0002] In the field of factory automation (FA) and the like, image processing technology is used to capture an image of an object under illumination by light from an illumination device and to acquire information related to the object from the generated image data.

[0003] When using a lighting device with many settable conditions such as lighting color and direction, the best lighting mode is appropriately selected from a plurality of lighting modes. For example, Japanese Patent Publication No. 2003-270163 (Patent Document 1) discloses the following technology: based on a plurality of captured images obtained by photographing a condition setting object with changing lighting conditions, the lighting condition that optimizes the evaluation value is calculated, and the inspection object is inspected under the calculated lighting condition.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2003-270163

[0007] Patent Document 2: Japanese Patent Application Laid-Open No. 7-220058 Summary of the invention

[0008] Problems to be solved by the invention

[0009] In the technology disclosed in Patent Document 1, lighting conditions are optimized based on a unified evaluation value. However, in image measurement using an appearance image of an object, the evaluation method for suitability of image measurement may differ depending on the state of the object or the content of image processing on the appearance image. Therefore, the lighting conditions optimized based on a unified evaluation value may not be suitable for image measurement.

[0010] The present disclosure has been made in view of the above-mentioned problems, and an object of the present disclosure is to provide an image processing system, a setting method, and a recording medium that can more easily perform lighting setting of lighting so as to be suitable for image measurement using an appearance image of an object.

[0011] Means for solving problems

[0012] According to an example of the present disclosure, an image processing system for performing image measurement using an appearance image of an object includes: a photographing unit that photographs the object; and an illumination unit that is composed of a plurality of illumination elements for irradiating light to the object, and is capable of adjusting at least one of the luminous intensity and the luminous color for each illumination element. The image processing system also includes an image acquisition unit, a selection unit, a calculation unit, and a mode determination unit. The image acquisition unit irradiates illumination light from the illumination unit according to a plurality of different evaluation lighting modes, and acquires at least one evaluation image corresponding to each evaluation lighting mode from the photographing unit. The selection unit selects one object algorithm from a plurality of evaluation algorithms. The calculation unit calculates an evaluation value indicating suitability for image measurement for each evaluation lighting mode using the object algorithm and at least one evaluation image corresponding to the evaluation lighting mode. The mode determination unit determines a lighting mode for image measurement from a plurality of evaluation lighting modes based on the evaluation value.

[0013] According to the present disclosure, an evaluation algorithm suitable for image measurement is selected from a plurality of evaluation algorithms, thereby automatically determining the lighting mode most suitable for the image measurement as the lighting mode for measurement. As a result, the lighting setting of the lighting can be easily performed in a manner suitable for image measurement using an appearance image of an object.

[0014] In the above disclosure, the plurality of evaluation algorithms include a first algorithm and at least one second algorithm. The first algorithm outputs a first evaluation value that depends on a difference between at least one first region and at least one second region in at least one evaluation image as an evaluation value. The at least one second algorithm outputs a second evaluation value that depends on a uniformity of at least one second region as an evaluation value.

[0015] For example, in image measurement for detecting defects on the surface of an object, image processing for emphasizing the defective portion is performed. In the case of performing image processing that can emphasize the background together with the defective portion, a lighting mode that makes the background as uniform as possible is preferred. According to the above disclosure, the area containing the defect is set as the first area, the area of ​​the background that does not contain the defect is set as the second area, and the second algorithm is selected as the object algorithm, thereby making it possible to determine a lighting mode that makes the background uniform as a lighting mode for measurement. On the other hand, in the case of performing image processing that does not emphasize the background too much, a lighting mode that makes the difference between the defective portion and the background larger is preferred. In such a case, by selecting the first algorithm as the object algorithm, it is possible to determine a lighting mode that makes the difference between the defective portion and the background larger as a lighting mode for measurement.

[0016] In the above disclosure, at least one second algorithm includes a plurality of second algorithms having different contribution rates of uniformity to the second evaluation value.

[0017] According to the present disclosure, it is possible to select a second algorithm having a contribution ratio suitable for image measurement from among a plurality of second algorithms. Thus, a lighting pattern suitable for image measurement is determined as a lighting pattern for measurement.

[0018] In the above disclosure, at least one second algorithm outputs a second evaluation value using a threshold value when a value corresponding to a brightness variance of a plurality of pixels belonging to at least one second region is less than a threshold value, and outputs a second evaluation value using a value corresponding to the variance when a value corresponding to the variance is greater than the threshold value. The threshold value is set according to at least one second algorithm.

[0019] In the above disclosure, the image processing system further includes a user interface. The calculation unit determines the shape, size, and position of at least one first region according to input to the user interface. The calculation unit sets the shape and size of at least one second region to be the same as the shape and size of at least one first region. The calculation unit determines the position of at least one second region according to input to the user interface.

[0020] According to the present disclosure, the user can set the shape, size, and position of the first area without having to set the shape and size of the second area. As a result, the effort and time required for setting the second area can be reduced.

[0021] In the above disclosure, a plurality of evaluation algorithms use mutually different edge patterns and output, as evaluation values, third evaluation values ​​indicating edge similarities of pixels belonging to a designated area in at least one evaluation image.

[0022] According to the present disclosure, for example, when the surface of an object has stripes or patterns extending in a fixed direction, an evaluation lighting pattern that emphasizes the stripes or patterns is determined as a lighting pattern for image measurement by selecting an evaluation algorithm corresponding to the direction.

[0023] In the above disclosure, at least one evaluation image includes a plurality of evaluation images. The calculation unit calculates evaluation values ​​for each of the plurality of evaluation images. The mode determination unit extracts the lowest evaluation value with the lowest degree of fitness from the evaluation values ​​for the plurality of evaluation images for each evaluation lighting pattern. The mode determination unit determines the evaluation lighting pattern that has extracted the lowest evaluation value with the highest degree of fitness from the plurality of evaluation lighting patterns as the lighting pattern used for image measurement.

[0024] According to the present disclosure, a lighting pattern that is stable and highly suitable is determined as a lighting pattern for image measurement.

[0025] In the above disclosure, the plurality of lighting patterns that the illumination unit can adopt are divided into a plurality of groups in advance. The image acquisition unit selects at least one group from the plurality of groups and determines the plurality of lighting patterns belonging to the selected at least one group as the plurality of lighting patterns for evaluation.

[0026] According to the present disclosure, even if the total number of lighting patterns that the illumination unit can adopt is large, by selecting a group suitable for image measurement, it is possible to shorten the time required to determine the lighting pattern used for image measurement.

[0027] In the above disclosure, the plurality of lighting elements are radially arranged, for example, along the first to Nth radial directions at equal angle intervals. N is an integer greater than 2. The plurality of groups include: a first group consisting of a plurality of lighting modes in which the illumination intensity in the first to Nth radial directions is uniform; and a second group consisting of a plurality of lighting modes in which only the lighting elements arranged in one radial direction among the first to Nth radial directions are illuminated.

[0028] In the above disclosure, N is an even number greater than 4. The plurality of groups may further include a third group consisting of a plurality of lighting patterns for lighting only illumination elements arranged along a pair of radial directions opposite to each other among the first to Nth radial directions.

[0029] In the above disclosure, the plurality of groups may include: a first group consisting of a plurality of lighting modes in which the luminous color of the plurality of lighting elements is white; and a second group consisting of a plurality of lighting modes in which the luminous color of the plurality of lighting elements is a color other than white.

[0030] In the above disclosure, the image processing system further includes a user interface. The selection unit selects a target algorithm according to an input to the user interface.

[0031] According to the present disclosure, the user can select an evaluation algorithm suitable for image measurement as a target algorithm.

[0032] According to an example of the present invention, a setting method for setting the lighting of an illumination unit includes the following 1st to 4th steps, wherein the illumination unit is composed of a plurality of illumination elements for irradiating light to an object, and at least one of the luminous intensity and the luminous color can be adjusted for each illumination element. The 1st step is a step of irradiating illumination light from the illumination unit according to a plurality of different evaluation lighting modes, and obtaining at least one evaluation image corresponding to each evaluation lighting mode from a photographing unit that photographs the object. The 2nd step is a step of selecting one object algorithm from a plurality of evaluation algorithms. The 3rd step is a step of calculating, for each evaluation lighting mode, an evaluation value indicating the suitability for image measurement using an appearance image of the object using the object algorithm and at least one evaluation image corresponding to the evaluation lighting mode. The 4th step is a step of determining a lighting mode for image measurement from a plurality of evaluation lighting modes based on the evaluation value.

[0033] According to an example of the present disclosure, the program causes a computer to execute the above-mentioned setting method. According to these disclosures, it is also possible to perform lighting setting of the lighting more simply in a method suitable for image measurement using an appearance image of an object.

[0034] Effects of the Invention

[0035] According to the present disclosure, it is possible to more easily perform lighting setting of lighting in a manner suitable for image measurement using an appearance image of an object. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a diagram showing a configuration example of an image processing system.

[0037] Figure 2 1 is a schematic diagram showing a basic configuration of the image processing system 1 .

[0038] Figure 3 It is a diagram showing an XZ cross section of the lighting device.

[0039] Figure 4 It is a bottom view of the lighting device.

[0040] Figure 5 It is a schematic diagram showing the hardware structure of the control device.

[0041] Figure 6 This is a flowchart showing the flow of determining the measurement lighting pattern.

[0042] Figure 7 This is a diagram showing an example of a defect region and a background region set in an evaluation image.

[0043] Figure 8 This is a diagram for explaining the lighting pattern belonging to the first group.

[0044] Fig. 9 This is a diagram for explaining the lighting pattern belonging to the second group.

[0045] Fig.10 This is a diagram for explaining the lighting pattern belonging to the third group.

[0046] Fig.11 This is a diagram showing an example of a setting screen for optimizing the lighting pattern.

[0047] Fig.12 This is a diagram showing an example of a setting screen when the tab 61 for setting a defect area is operated.

[0048] Fig.13 Yes means Fig.12 FIG. 1 is a diagram showing an example of a setting screen when the edit button shown is operated.

[0049] Fig.14This is a diagram showing an example of a setting screen after the position and size of the frame line indicating the defect region range are changed.

[0050] Fig.15 This is a diagram showing an example of a setting screen when setting a background area.

[0051] Fig.16 This is a diagram showing another example of the setting screen when setting an additional background area and defect area.

[0052] Fig.17 This is a diagram showing an example of evaluation value data.

[0053] Fig.18 This is a diagram showing a list of evaluation values ​​displayed on the setting screen.

[0054] Fig.19 These are enlarged views of parts of four evaluation images obtained under illumination of four evaluation lighting patterns having different brightness.

[0055] Fig. 20 It means according to Fig.19 The graph shows the values ​​of σ' calculated for the evaluation images.

[0056] Fig.21 It means according to Fig.19 The diagram shows the evaluation value E calculated from the evaluation image.

[0057] Fig. 22 Yes Fig.19 The diagram shows the result of image processing on the evaluation image shown.

[0058] Fig.23 The diagram shows eight examples of edge patterns.

[0059] Fig.24 This is a diagram showing another configuration example of the image processing system. DETAILED DESCRIPTION

[0060] Embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, the same or corresponding parts in the drawings are denoted by the same reference numerals and their description will not be repeated.

[0061] §1 Application Examples

[0062] Reference Figure 1 , illustrating an application example of the present invention. Figure 1 It is a diagram showing a configuration example of an image processing system.

[0063] Figure 1The illustrated image processing system 1 is a system for performing image measurement using an appearance image of an object. The image processing system 1 includes: a camera 8 as an example of a photographing unit for photographing the object; an illumination device 4 as an example of an irradiation unit for irradiating light to the object; and a control device 100 for controlling the processing executed in the image processing system 1.

[0064] The lighting device 4 is composed of a plurality of lighting elements 40 for irradiating light to an object. Figure 1 In the example of FIG. 4 , the lighting device 4 is composed of 13 lighting elements 40. Figure 1 In FIG. 4 , some of the numbers of the lighting elements 40 are omitted. In addition, the number of the lighting elements 40 is not limited to Figure 1 In the example shown, it is sufficient if there are two or more. The sizes of the lighting elements 40 may be different from each other, or they may be the same. The lighting elements 40 may be arranged on the same plane, or they may be arranged on different planes, as long as they are arranged so that light can be irradiated to the object from each lighting element 40. The lighting device 4 can adjust the luminous intensity and luminous color for each lighting element 40. Here, the so-called ability to adjust the luminous intensity includes, of course, lighting or extinguishing each lighting element 40, and also includes adjusting the intensity of light irradiated from each lighting element 40 in stages.

[0065] The control device 100 typically has a structure that complies with a general computer architecture. The control device 100 includes an image acquisition unit 210, an evaluation algorithm selection unit 220, a calculation unit 230, and a lighting mode determination unit 240. Typically, the image acquisition unit 210, the evaluation algorithm selection unit 220, the calculation unit 230, and the lighting mode determination unit 240 are implemented by the CPU of the control device 100 executing a program stored in the control device 100 or the like.

[0066] The image acquisition unit 210 irradiates the object with illumination light from the illumination device 4 according to a plurality of different evaluation lighting patterns xi, and acquires at least one evaluation image R corresponding to each evaluation lighting pattern xi from the camera 8. The lighting pattern is a pattern that defines the light emission intensity and light emission color of each lighting element 40.

[0067] The evaluation algorithm selection unit 220 selects one evaluation algorithm from a plurality of evaluation algorithms as a target algorithm. The evaluation algorithm defines a calculation method for calculating an evaluation value indicating the suitability of a lighting pattern for image measurement using an evaluation image. The evaluation algorithm is represented by a function, a program, etc. for calculating the evaluation value.

[0068] A plurality of evaluation algorithms are preset according to various methods used in image measurement using an appearance image of an object. For example, in the case of performing image measurement for inspecting defects on the surface of an object, the following first and second algorithms are preset. The first algorithm is an algorithm for calculating a first evaluation value, which depends on the difference between a defective area including a defective portion and a background area not including the defective portion. The second algorithm is an algorithm for calculating a second evaluation value, which depends on the uniformity of the background area. In addition, the first evaluation value does not depend on the uniformity of the background area. The second evaluation value may also depend on the difference between the defective area and the background area.

[0069] In image measurement for detecting defects, image processing for emphasizing defective parts can be performed on appearance images. Various methods are known for such image processing, including image processing that can emphasize both defective parts and backgrounds and image processing that does not easily emphasize the background. For example, Sobel filter processing can be used as image processing that can emphasize the background. For example, Hough transform processing can be used as image processing that does not easily emphasize the background.

[0070] When image measurement including image processing that makes the background less likely to be emphasized is performed, a lighting condition in which the difference between the defect area and the background area is large is preferred, regardless of the uniformity of the background area. In this case, the first algorithm is selected. On the other hand, when image measurement using image processing that can emphasize the background is performed, a lighting condition in which the uniformity of the background area is high is preferred. In this case, the second algorithm is selected.

[0071] The calculation unit 230 calculates the evaluation value Pi indicating the suitability for image measurement for each evaluation lighting pattern xi using the target algorithm and the evaluation image R corresponding to the evaluation lighting pattern xi. Specifically, the calculation unit 230 calculates the feature quantity based on the evaluation image R and applies the calculated feature quantity to the target algorithm to calculate the evaluation value Pi corresponding to the evaluation lighting pattern xi. The evaluation value Pi indicates the degree to which the evaluation lighting pattern xi is suitable for image measurement.

[0072] The lighting pattern determination unit 240 determines a measurement lighting pattern for image measurement based on the evaluation value Pi calculated by the calculation unit 230. Typically, the lighting pattern determination unit 240 determines the evaluation lighting pattern xi corresponding to the evaluation value Pi indicating the highest suitability for image measurement as the measurement lighting pattern.

[0073] In the image processing system 1 according to the present embodiment, by selecting an evaluation algorithm suitable for image measurement from a plurality of evaluation algorithms, the lighting pattern most suitable for the image measurement is automatically determined as the lighting pattern for measurement. Thus, the lighting setting can be simply performed in a manner suitable for image measurement using the appearance image of the object.

[0074] §2 Specific Example

[0075] <A. Structure of Image Processing System>

[0076] Figure 2 It is a schematic diagram showing the basic structure of the image processing system 1. The image processing system 1 includes a control device 100, a camera 8, and a lighting device 4 as main components. The control device 100 and the camera 8 are connected so as to be able to communicate with each other data - wise. The lighting device 4 is controlled by the control device 100 via the camera 8.

[0077] Hereinafter, for the sake of convenience of explanation, the direction in which light is irradiated from the lighting device 4 is defined as the Z - axis, the left - right direction of the paper surface is set as the X - axis, and the axis perpendicular to the X - axis and the Z - axis is set as the Y - axis. In addition, the side irradiated with light is set as the lower side.

[0078] An opening 46 is provided in the upper part of the lighting device 4 so that the camera 8 can photograph the object W from the upper part of the lighting device 4. In addition, in the present embodiment, the camera 8 is provided in the upper part of the lighting device 4, but it may be provided as long as at least a part of the irradiation area of the lighting device 4 is included in at least a part of the shooting field of view of the camera 8, and it may also be provided on the side of the lighting device 4.

[0079] The camera 8 is a photographing unit that photographs a subject existing in the shooting field of view and generates an image. The camera 8 includes an optical system such as a lens and an aperture, and a light - receiving element such as a CCD (Charge Coupled Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) image sensor as main components.

[0080] The control device 100 can accept the setting of the content of image processing separately from the execution of image processing such as checking for defects and dirt on the object W, measuring the size, arrangement, orientation, etc. of the object W, and recognizing characters, graphics, etc. on the surface of the object W. The setting of the image processing content includes the setting of the shooting conditions when acquiring an image and the setting of the processing content to be executed on the image. The setting of the shooting conditions includes the lighting setting for the lighting device 4 and the camera setting for the camera 8. The control device 100 functions as a device for performing the lighting setting for the lighting device 4. In addition, a setting assistance device for performing the lighting setting for the lighting device 4 may be provided separately from the control device 100.

[0081] The control device 100 includes a display unit 101 and a touch panel 102 mounted on the display surface of the display unit 101 as a user interface. The display unit 101 is typically composed of a liquid crystal display and displays the setting content for the user, for example. The touch panel 102 functions as an input unit for inputting information related to various settings. For example, the user can operate the touch panel 102 based on the information displayed on the display unit 101, thereby inputting setting information related to the setting of the content of image processing and enabling various settings. In addition, although it is assumed that the input unit is composed of a touch panel, it may also be composed of a keyboard or a mouse, or both.

[0082] <B. Structure of the Lighting Device>

[0083] Refer to Figure 3 and Figure 4 , and the structure of the lighting device 4 will be described. Figure 3 is a diagram showing the XZ cross-section of the lighting device. Figure 4 is a bottom view of the lighting device.

[0084] Figure 3 The shape of the illustrated lighting device 4 is dome-shaped. In the lighting device 4, a plurality of light emitting units (hereinafter also referred to as "light sources") having different main wavelengths are provided at positions facing the object W. Specifically, a plurality of sets of light sources each composed of a red light source r, a green light source g, and a blue light source b are provided. In addition, in Figure 3 , the light source marked with a left downward diagonal line is the red light source r, the light source marked with a dot pattern is the green light source g, and the light source marked with a right downward diagonal line is the blue light source b, and some reference numerals are omitted.

[0085] The lighting device 4 includes a plurality of lighting elements 40. One or more sets of light sources are provided in each lighting element 40. As Figure 4As shown, the plurality of lighting elements 40 are radially arranged along the first radial direction D1 to the fourth radial direction D4 at intervals of 90 degrees. Specifically, the plurality of lighting elements 40 include: a central lighting element 41; lighting elements 42 (42U, 42M, 42L) arranged along the first radial direction D1 from the lighting element 41; lighting elements 43 (43U, 43M, 43L) arranged along the second radial direction D2 from the lighting element 41; lighting elements 44 (44U, 44M, 44L) arranged along the third radial direction D3 from the lighting element 41; and lighting elements 45 (45U, 45M, 45L) arranged along the fourth radial direction D4 from the lighting element 41.

[0086] The incident directions of the light emitted from the lighting elements 41 to 45 are different from each other. The incident direction refers to the direction centered on the Z axis. The light emitted from the light source of the lighting element 41 is reflected on the reflector 47 and travels parallel to the Z axis. The light emitted from the light source of the lighting element 42 travels parallel to the Z axis and then passes through the dome-shaped diffuser 48 (see FIG. 4 ). Figure 3 ), thereby moving in the first radial direction D1 (refer to Figure 4 ) in the direction inclined in the opposite direction of the Z axis. The light emitted from the light source of the illumination element 43 travels parallel to the Z axis, passes through the diffuser 48, and then travels in the second radial direction D2 (refer to Figure 4 ) in the direction inclined in the opposite direction of the Z axis. The light emitted from the light source of the illumination element 44 travels parallel to the Z axis, passes through the diffuser 48, and then travels in the third radial direction D3 (refer to Figure 4 ) in the direction inclined in the opposite direction of the Z axis. The light emitted from the light source of the illumination element 45 travels parallel to the Z axis, passes through the diffuser 48, and then travels in the fourth radial direction D4 (refer to Figure 4 ) in the direction of inclination on the opposite side.

[0087] The lighting element 41 is circular, and the lighting elements 42 to 45 are arc-shaped with the lighting element 41 as the center.

[0088] The illumination elements 42U, 43U, 44U, and 45U are arranged in a ring shape so that the distances (radius r1) from the central illumination element 41 are the same. The illumination elements 42M, 43M, 44M, and 45M are arranged in a ring shape so that the distances (radius r2>r1) from the central illumination element 41 are the same. The illumination elements 42L, 43L, 44L, and 45L are arranged in a ring shape so that the distances (radius r3>r2) from the central illumination element 41 are the same. The incident angles of light emitted from the illumination elements 42U to 45U with respect to the XY plane, the incident angles of light emitted from the illumination elements 42M to 45M with respect to the XY plane, and the incident angles of light emitted from the illumination elements 42L to 45L with respect to the XY plane are different from each other.

[0089] Thus, the lighting device 4 has 13 lighting elements 41, 42U to 45U, 42M to 45M, and 42L to 45L. However, the number of lighting elements included in the lighting device 4 is not limited to 13.

[0090] For example, in Figure 4 the illustrated lighting device 4, a plurality of lighting elements 40 are radially arranged along the first to fourth radial directions D1 to D4, but the plurality of lighting elements 40 may also be radially arranged along the first to Nth radial directions at equal angular intervals. N may be an integer of 2 or more.

[0091] In addition, in Figure 4 the illustrated lighting device 4, the number of lighting elements 40 arranged along each radial direction is set to 3, but this number may also be 2 or 4 or more.

[0092] In addition, in each lighting element, the red light source r, the green light source g, and the blue light source b do not need to be arranged in the same number respectively, and it is sufficient that each of the three light sources is arranged one or more. In addition, the ratios of the red light source r, the green light source g, and the blue light source b included in each lighting element may be the same or different. For example, more red light sources r may be arranged in one lighting element than in other lighting elements. In the present embodiment, the case where the red light source r, the green light source g, and the blue light source b are arranged in the same number is described.

[0093] <C. Hardware Structure of Control Device 100>

[0094] Figure 5 is a schematic diagram showing the hardware structure of the control device. As Figure 5 shown, the control device 100 typically has a structure that follows a general computer architecture, and various processes described later are implemented by the processor executing a pre-installed program.

[0095] More specifically, the control device 100 includes a processor 110 such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit), a RAM (Random Access Memory) 112, a display controller 114, a system controller 116, an I / O (Input Output) controller 118, a hard disk 120, a device interface 122, an input interface 124, a communication interface 128, and a memory card interface 130. These components are centered around the system controller 116 and are connected in such a way that they can communicate with each other.

[0096] The processor 110 exchanges programs (codes) and the like with the system controller 116 and executes the programs (codes) and the like in a predetermined order, thereby realizing target arithmetic processing.

[0097] The system controller 116 is connected to the processor 110 , the RAM 112 , the display controller 114 , the input interface 124 , and the I / O controller 118 via buses, exchanges data with each component, and controls the overall processing of the device 100 .

[0098] Typically, the RAM 112 is a volatile storage device such as a DRAM (Dynamic Random Access Memory), and stores programs read from the hard disk 120, camera images acquired by the camera 8, image processing results, and workpiece data including imaging conditions.

[0099] The display controller 114 is connected to the display unit 101 , and outputs signals for displaying various information to the display unit 101 in accordance with internal commands from the system controller 116 .

[0100] The input interface 124 is connected to the touch panel 102 , and transmits various information input from the touch panel 102 to the system controller 116 .

[0101] The I / O controller 118 controls data exchange with a recording medium or external device connected to the control device 100. More specifically, the I / O controller 118 is connected to the hard disk 120, the device interface 122, the communication interface 128, and the memory card interface 130.

[0102] The hard disk 120 is typically a non-volatile magnetic storage device, and stores various setting values ​​in addition to the control program 150 executed by the processor 110. The control program 150 installed in the hard disk 120 is circulated in a state stored in the memory card 136 or the like. In addition, a semiconductor storage device such as a flash memory or an optical storage device such as a DVD-RAM (Digital Versatile Disk Random Access Memory) may be used instead of the hard disk 120.

[0103] Figure 1 The image acquisition unit 210 , the evaluation algorithm selection unit 220 , the calculation unit 230 , and the lighting pattern determination unit 240 shown are implemented by the processor 110 executing the control program 150 .

[0104] The device interface 122 mediates data transmission between the camera 8 and the lighting device 4 and the processor 110. The device interface 122 outputs an instruction according to the shooting conditions commanded by the processor 110 to the camera 8 and the lighting device 4. In addition, the device interface 122 acquires image data obtained by photographing the object W and mediates data transmission between the processor 110 and the camera 8.

[0105] The communication interface 128 mediates data transmission between the processor 110 and other personal computers, server devices, etc. not shown. The communication interface 128 typically consists of Ethernet (registered trademark), USB (Universal Serial Bus), etc.

[0106] The memory card interface 130 mediates data transmission between the processor 110 and the memory card 136 as a recording medium. The control program 150 etc. executed by the control device 100 are stored in the memory card 136 and circulated, and the memory card interface 130 reads the control program from the memory card 136. The memory card 136 consists of a general semiconductor storage device such as SD (Secure Digital), a magnetic recording medium such as a flexible disk, or an optical recording medium such as a CD-ROM (Compact Disk ReadOnly Memory). Alternatively, a program downloaded from a distribution server etc. can be installed in the control device 100 via the communication interface 128.

[0107] When using a computer having a structure following a general computer architecture as described above, in addition to the application for providing the functions of the present embodiment, an OS (Operating System) for providing the basic functions of the computer can also be installed. In this case, the control program 150 of the present embodiment can also execute processing by calling necessary modules in the program modules provided as part of the OS in a prescribed order and / or timing.

[0108] Furthermore, the control program 150 of the present embodiment can also be provided as part of another program. In this case, the program itself does not include the modules included in the other program having the above-described combination, but cooperates with the other program to execute processing. That is, as the control program of the present embodiment, it can also be in a form embedded in such another program.

[0109] Alternatively, instead, a part or all of the functions provided by the execution of the control program 150 can be installed as a dedicated hardware circuit.

[0110] <D. Outline of the method for determining the lighting mode>

[0111] In the present embodiment, the control device 100 evaluates each evaluation lighting pattern based on the evaluation images R obtained by photographing in mutually different evaluation lighting patterns, and determines the measurement lighting pattern for image measurement based on the evaluation result.

[0112] Figure 6 is a flowchart showing the process of determining the measurement lighting pattern. The control device 100 obtains the evaluation image Ri photographed in each evaluation lighting pattern xi (step S1). The control device 100 selects one target algorithm from a plurality of evaluation algorithms (step S2).

[0113] Next, the control device 100 calculates the evaluation value Pi of the evaluation lighting pattern xi using the target algorithm and the evaluation image Ri (step S3). The control device 100 determines the measurement lighting pattern based on the evaluation value Pi for each evaluation lighting pattern xi (step S4).

[0114] <E. Example of evaluation algorithm>

[0115] Next, an example of the evaluation algorithm will be described. Here, an evaluation algorithm for calculating an evaluation value that represents the suitability of each evaluation lighting pattern for image measurement for defect inspection will be described.

[0116] In order to inspect the defect of the object W, a lighting pattern that can obtain an image in which the defect part is conspicuous is preferable. Therefore, for the evaluation image R, at least one defect region including the defect part and at least one background region not including the defect part are preset.

[0117] Figure 7 is a diagram showing an example of the defect region and the background region set in the evaluation image. In Figure 7 the illustrated evaluation image R, m defect regions H0 to H m-1 and n background regions h0 to h n-1 are set. In addition, m and n only need to be integers of 1 or more. That is, the number of defect regions set in the evaluation image R is not particularly limited and may be 1 or 2 or more. The number of background regions set in the evaluation image R is not particularly limited and may be 1 or 2 or more.

[0118] Figure 7 the illustrated defect regions H0 to H m-1 and the background regions h0 to h n-1 all have the same shape and size. Specifically, the defect regions H0 to H m-1 and the background regions h0 to h n-1 are rectangular shapes with a length of lx in the x-axis direction and a length of ly in the y-axis direction. However, the defect regions H0 to H m-1and background area h0~h n-1 The shapes and sizes can be different from each other.

[0119] The position of each pixel constituting the evaluation image R is represented by an xy coordinate value with the upper left corner of the evaluation image R as the origin. p The coordinates of the upper left vertex are (X p , Y p ). Background area h q The coordinates of the upper left vertex are (x q ,y q ). The pixel brightness (brightness) at coordinates (x, y) is represented by f(x, y).

[0120] In image measurement for defect detection, image processing for emphasizing defective parts is performed on appearance images, and the background is also emphasized according to the image processing method. Therefore, a first algorithm that depends on the difference between the defective area and the background area and a second algorithm that calculates an evaluation value that depends on the uniformity of the background area are pre-set.

[0121] The first algorithm and the second algorithm calculate the evaluation value E shown in the following formula (1). That is, the evaluation value E is calculated for each of the defect areas H0 to H m-1 Calculated (σ0 / σ')~(σ m-1 The minimum value among them is taken as the evaluation value E.

[0122]

[0123] The σ of the numerator of formula (1) p is the defect area H p and background area h0~h n-1 Therefore, the first algorithm and the second algorithm can calculate the evaluation value E that depends on the difference between the defect area and the background area. p For example, the following formula (2) is satisfied.

[0124]

[0125] In formula (2), b represents the background area h0~h n-1 The average value of the brightness of the pixels is expressed by the following formula (3).

[0126]

[0127] The denominator σ' in equation (1) represents the background area h0~h n-1 Uniformity. In the first algorithm, a predetermined threshold κ is substituted for σ'. κ is set to 1, for example. Thus, when the first algorithm is used, an evaluation value E that does not depend on the uniformity of the background area is calculated.

[0128] In the second algorithm for calculating the evaluation value E that depends on the uniformity of the background area, σ' is calculated as follows. Here, the calculation method of σ' when setting multiple second algorithms with different contribution rates of the uniformity of the background area to the evaluation value E is described. The contribution rate of the uniformity of the background area to the evaluation value E is represented by the background uniformity level λ. The background uniformity level λ can take any integer from 1 to L. L is 5, for example. In the case of L=5, 5 second algorithms corresponding to the background uniformity levels λ=1 to 5 are pre-set.

[0129] First, the background area h0~h is calculated according to the following formula (4): n-1 The variance V of the brightness values ​​of all pixels.

[0130]

[0131] Using the variance V calculated according to the equation (4), σ' is calculated according to the following equation (5).

[0132]

[0133] As shown in formula (5), at the value (λ / L)V corresponding to the variance V 1 / 2 When σ' is smaller than the threshold value κ, σ' is fixed (limited) to the threshold value κ so as not to be smaller than the threshold value κ. That is, at the value (λ / L)V corresponding to the variance V 1 / 2 If it is less than the threshold value κ, the threshold value κ is used to calculate the evaluation value E. On the other hand, when the value (λ / L)V corresponding to the variance V 1 / 2 When the value is greater than the threshold value κ, σ' is determined to be the value (λ / L)V corresponding to the variance V 1 / 2 That is, at the value (λ / L)V corresponding to the variance V 1 / 2 When the value is above the threshold value κ, (λ / L)V is used. 1 / 2 The evaluation value E is calculated. In addition, the threshold κ is preset according to the second algorithm and is 1, for example.

[0134] Thus, in the second algorithm, the evaluation value E is calculated depending on the variance V representing the uniformity of the background area. Furthermore, the larger the background uniformity level λ is (the closer it is to L), the larger σ', the denominator of equation (1), is (the greater the contribution of the uniformity of the background area to the evaluation value E).

[0135] The brightness of the pixels constituting the evaluation image is represented by a red component, a green component, and a blue component. The processor 110 substitutes the brightness of the red component into the above-mentioned equations (2) to (5) to calculate σ Rp and σ' R The processor 110 calculates σ by substituting the brightness of the green component into the above equations (2) to (5).Gp and σ' G The processor 110 calculates σ by substituting the luminance of the blue component into the above formulas (2) to (5). Bp and σ' B The processor 110 uses σ Rp +σ Gp +σ Bp as σ p and substitutes it into formula (1). Similarly, the processor 110 uses σ' R +σ' G +σ' B as σ' and substitutes it into formula (1).

[0136] The higher the fitness of the image measurement, the larger the evaluation value E calculated by formula (1). However, the evaluation value is not limited to the evaluation value E calculated by formula (1), and it can also be smaller when the fitness of the image measurement is higher. In the following, the evaluation value E calculated by formula (1) is taken as an example for explanation.

[0137] <F. Examples of Lighting Patterns for Evaluation>

[0138] When the brightness of each lighting element 40 can be adjusted to 7 levels (including extinguished), and the color of each lighting element 40 can be adjusted to any one of 7 colors: white, red, green, blue, cyan, magenta, and yellow, the lighting states that each lighting element 40 can take are 43 obtained by adding the extinguished state to 6×7 = 42 lighting states. When the lighting device 4 has 13 lighting elements 40, the total number of lighting patterns that the lighting device 4 can take is 43 13 , which becomes huge. Therefore, in the present embodiment, the lighting patterns that the lighting device 4 can take are pre-divided into multiple groups, and the lighting patterns belonging to at least one group selected from the multiple groups are determined as the lighting patterns for evaluation.

[0139] Figure 8 is a diagram illustrating the lighting patterns belonging to the first group. Figure 8 The first group illustrated consists of multiple lighting patterns with uniform irradiation intensity in the first to fourth radial directions D1 to D4.

[0140] As Figure 8 shown, in the first group, as combinations of lit lighting elements 40 (hereinafter referred to as "element combinations"), there are 9 types from (b) to (j). In the element combination shown in (b), all lighting elements 40 are lit. In the element combination shown in (c), the distance from the lighting element 41 is r3 (refer to Figure 4) lighting elements 42L to 45L are lit. In the element combination shown in (d), the distances from the lighting element 41 are r2 and r3 respectively (refer to Figure 4 ) are lit. In the element combination shown in (e), the lighting elements 40 other than the central lighting element 41 are lit. In the element combination shown in (f), the lighting elements 40 other than the central lighting element 41 are lit. Figure 4 ) lighting elements 42M to 45M are lit. In the element combination shown in (g), the distances from the lighting element 41 are r1 and r2 respectively (refer to Figure 4 ) are lit. In the element combination shown in (h), only the central lighting element 41 is lit. In the element combination shown in (i), the distance from the lighting element 41 is r1 (refer to Figure 4 ) are lit. In the element combination shown in (j), the lighting elements 41, 42U to 45U are lit. In addition, the brightness and color of the lighting elements that are lit are adjusted to be the same.

[0141] Fig. 9 This is a diagram for explaining the lighting pattern belonging to the second group. Fig. 9 The second group illustrated is composed of a plurality of lighting patterns for lighting only the illumination elements arranged in one radial direction among the first radial direction D1 to the fourth radial direction D4.

[0142] like Fig. 9 As shown, there are 16 element combinations (b) to (q) in the second group. In the element combination shown in (b), the elements are arranged in the first radial direction D1 (refer to Figure 4 ) are all illuminated, and in the element combinations shown in (c) to (e), the illumination elements 42L, 42M, and 42U are illuminated separately. In the element combination shown in (f), the illumination elements 42L, 42M, and 42U are arranged in the second radial direction D2 (refer to Figure 4 ) are all illuminated, and in the element combinations shown in (g) to (i), the illumination elements 43L, 43M, and 43U are illuminated separately. In the element combination shown in (j), the illumination elements 43L, 43M, and 43U are arranged in the third radial direction D3 (refer to Figure 4 ) are all illuminated, and in the element combinations shown in (k) to (m), the illumination elements 44L, 44M, and 44U are illuminated separately. In the element combination shown in (n), the illumination elements 44L, 44M, and 44U are arranged in the fourth radial direction D4 (refer to Figure 4All of the lighting elements 45L, 45M, and 45U on (o) are lit, and in the element combinations shown in (o) to (q), the lighting elements 45L, 45M, and 45U are lit respectively. In addition, the brightness and color of the lit lighting elements are adjusted to be all the same.

[0143] Fig.10 It is a diagram for explaining the lighting patterns belonging to Group 3. Fig.10 The exemplified Group 3 is composed of a plurality of lighting patterns in which only the lighting elements arranged along a pair of diametrically opposed radial directions among the first to fourth radial directions D1 to D4 are lit.

[0144] As Fig.10 shown, there are 12 kinds of element combinations (b) to (m) in Group 3. Specifically, in the element combinations shown in (b) to (g), among the lighting elements 42L, 42M, 42U arranged on the first radial direction D1 (refer to Figure 4 ) and the lighting elements 44L, 44M, 44U arranged on the third radial direction D3, the selected lighting elements are lit. The lit lighting elements among 44L, 44M, 44U are symmetric with respect to the central lighting element 41 to the lit lighting elements among 42L, 42M, 42U. In the element combinations shown in (h) to (m), among the lighting elements 43L, 43M, 43U arranged on the second radial direction D2 (refer to Figure 4 ) and the lighting elements 45L, 45M, 45U arranged on the fourth radial direction D4, the selected lighting elements are lit. The lit lighting elements among 43L, 43M, 43U are symmetric with respect to the central lighting element 41 to the lit lighting elements among 45L, 45M, 45U. In addition, the brightness and color of the lit lighting elements are adjusted to be all the same.

[0145] In addition, when a plurality of lighting elements 40 are arranged radially along the first to Nth radial directions at equal angular intervals, as long as N is an even number of 4 or more, Group 3 in which only the lighting elements arranged along a pair of diametrically opposed radial directions are lit can be set. As long as N is an integer of 2 or more, Group 2 in which only the lighting elements arranged in one radial direction are lit can be set.

[0146] <G. Setting Screen for Optimizing Lighting Patterns>

[0147] Refer to Fig.11 to explain an example of a setting screen for optimizing lighting patterns. Fig.11 It is a diagram showing an example of a setting screen for optimizing lighting patterns.

[0148] Fig.11The illustrated setting screen 60 is displayed on the display unit 101 by the processor 110 executing the control program 150. Input to the setting screen 60 is performed using the touch panel 102. That is, the processor 110 performs processing corresponding to the input to the touch panel 102 as a user interface.

[0149] Fig.11 The illustrated setting screen 60 has three tabs 61 to 63. Tab 61 is selected when setting the defect area. Tab 62 is selected when setting various measurement parameters for optimization. Tab 63 is selected when setting the output condition of information indicating the measurement lighting pattern determined by optimization. Fig.11 , a setting screen 60 is shown when a tab 62 is selected.

[0150] The setting screen 60 includes an input field 64 for selecting an evaluation algorithm. One of integers from 0 to L can be input in the input field 64. Here, "0" is input when the first algorithm is selected. Any integer from 1 to L is input when the second algorithm is selected. "1" to "L" input to the input field 64 represents the background uniformity level λ. The processor 110 selects a target algorithm from the first algorithm and L types of second algorithms according to the input to the input field 64.

[0151] The setting screen 60 includes an input field 65 and check boxes 66 to 68 for selecting an evaluation lighting mode among the lighting modes that the lighting device 4 can adopt.

[0152] Check boxes 66 and 67 are used to select Figure 8 to Figure 10 If both check boxes 66 and 67 are not checked, select Figure 8 When check box 66 is checked and check box 67 is not checked, select Figure 8 Group 1 and Fig.10 When check box 66 is not checked and check box 67 is checked, select Figure 8 Group 1 and Fig. 9 When both check boxes 66 and 67 are checked, all of the first to third groups are selected.

[0153] The check box 68 is used to set the emission color of the illuminated lighting element 40. When the check box 68 is not checked, the emission color of the illuminated lighting element 40 is set to white only. When the check box 68 is checked, the emission color of the illuminated lighting element 40 is adjusted to six colors other than white, namely red, green, blue, cyan, magenta, and yellow.

[0154] The input field 65 is used to set the variation range of the brightness of the illuminated lighting element 40. For example, the brightness of each lighting element 40 can be adjusted in 127 levels. When "20" is input in the input field 65, the brightness of the illuminated lighting element 40 is adjusted to 6 levels of "20", "40", "60", "80", "100", and "120". When "30" is input in the input field 65, the brightness of the illuminated lighting element 40 is adjusted to 4 levels of "30", "60", "90", and "120".

[0155] For example, when the check boxes 66 to 68 are not checked and "20" is input in the input field 65, Figure 8 The nine combinations of elements (b) to (j) shown are adjusted to six levels of brightness in sequence. Therefore, the processor 110 selects 9×6=54 lighting patterns as lighting patterns for evaluation.

[0156] When only the check box 66 is checked and "20" is input in the input field 65, Figure 8 The nine combinations of elements (b) to (j) shown and Fig.10 The 12 combinations of elements (b) to (m) shown are adjusted to 6 levels of brightness in sequence. Therefore, the processor 110 selects 21×6=126 lighting patterns as lighting patterns for evaluation.

[0157] When only the check box 68 is checked and "20" is input in the input field 65, Figure 8 The nine combinations of elements (b) to (j) shown are adjusted to six levels of brightness and six luminous colors, respectively. Therefore, the processor 110 selects 9×6×6=324 lighting patterns as lighting patterns for evaluation.

[0158] The setting screen 60 includes a button 69 for adding a defect area, a button 70 for adding a background area, and a button 71 for deleting a defect area or a background area. Details of the processing performed by the processor 110 when the buttons 69 to 71 are operated will be described later.

[0159] The setting screen 60 includes a button 72 for starting a process of determining a measurement lighting mode that is most suitable for image measurement from the evaluation lighting modes. When the button 72 is operated, the processor 110 causes the lighting device 4 to irradiate illumination light according to the plurality of evaluation lighting modes determined by the operation of the input field 65 and the check boxes 66 to 68, and obtains an evaluation image corresponding to each evaluation lighting mode from the camera 8. The processor 110 calculates an evaluation value E for each evaluation lighting mode using the obtained evaluation image and the target algorithm selected by the input to the input field 64.

[0160] The processor 110 calculates the evaluation value E each time an evaluation image is acquired, and displays the calculated evaluation value E in the display column 73. In addition, when the processor 110 has calculated the evaluation values ​​E for all evaluation lighting patterns, it displays the maximum evaluation value E and the lighting pattern No. for identifying the evaluation lighting pattern for which the maximum evaluation value E is calculated in the display column 74.

[0161] The processor 110 repeatedly calculates the evaluation value E each time the button 72 is operated. Thus, a plurality of evaluation values ​​E are obtained for the same sample or different samples.

[0162] The setting screen 60 includes a button 76 for switching the display content of the area 75. In the area 75, either the evaluation image obtained from the camera 8 or the list of evaluation values ​​E calculated by the processor 110 is displayed. When the button 76 is operated when the evaluation image is displayed, the processor 110 switches the display content of the area 75 to the list of evaluation values ​​E. When the button 76 is operated when the list of evaluation values ​​E is displayed, the processor 110 switches the display content of the area 75 to the evaluation image.

[0163] The setting screen 60 includes a button 77 for deleting unnecessary data. The processor 110 deletes data designated in the list of evaluation values ​​E displayed in the area 75 .

[0164] The setting screen 60 includes check boxes 78 and 79 for selecting one of a plurality of evaluation lighting modes and an input field 80. When one of the check boxes 78 and 79 is checked, the processor 110 receives input to the input field 80. In addition, the processor 110 sorts the list of evaluation values ​​E displayed in the area 75 in the order of the magnitude of the evaluation values ​​E in accordance with the fact that the check box 79 is checked.

[0165] The processor 110 displays an evaluation image corresponding to the evaluation lighting mode of the lighting mode No. input to the input field 80 in the area 75. Furthermore, the processor 110 displays a schematic diagram showing the position of the lighting element 40 lit in the evaluation lighting mode of the lighting mode No. input to the input field 80 in the area 81. Furthermore, the processor 110 displays the evaluation value E calculated for the evaluation lighting mode of the lighting mode No. input to the input field 80 in the display field 73. Thus, the user can easily confirm the evaluation value E corresponding to each evaluation lighting mode, the evaluation image, and the position of the lighting element 40 lit.

[0166] The setting screen 60 includes a check box 82 for recording the evaluation image. When the check box 82 is checked, the processor 110 stores the acquired evaluation image in the storage area.

[0167] The setting screen 60 includes a data storage area 83 for storing the calculated evaluation value E. In the area 83, a storage folder and a storage file name are specified. When the processor 110 finishes calculating the evaluation value E for each evaluation lighting pattern according to the operation of the button 72, it generates evaluation value data representing a list of the calculated evaluation values E. The processor 110 assigns the file name specified in the area 83 to the generated evaluation value data and saves the evaluation value data in the folder specified in the area 83. Details of the evaluation value data will be described later.

[0168] In addition, when the checkbox 82 is selected, the evaluation image is also saved in the storage folder specified in the area 83.

[0169] Furthermore, the setting screen 60 includes a button 84 for performing a test measurement and an area 85 for inputting determination conditions. When the evaluation value E does not satisfy the determination conditions input to the area 85, the processor 110 gives an error notification. When this error notification is not required, all the available ranges of the evaluation value E can be set in the area 85.

[0170] The setting screen 60 includes an OK button 86 and a Cancel button 87. Based on the operation of the OK button 86, the processor 110 determines the evaluation lighting pattern corresponding to the evaluation value E (maximum evaluation value) with the highest fitness as the measurement lighting pattern. Based on the operation of the Cancel button 87, the processor 110 ends the process without reflecting the operation result of the setting screen 60 to the setting of the lighting device 4.

[0171] <H. Setting of Defect Area and Background Area>

[0172] Next, with reference to Figure 12 to Figure 16 , a method for setting the defect area and the background area will be described. Fig.12 is a diagram showing an example of the setting screen when the label 61 for setting the defect area is operated.

[0173] As Fig.12 shown, when the label 61 is operated, the processor 110 displays a sample image obtained from the camera 8 in the area 75 of the setting screen 60. Before operating the label 61, the user sets a sample of the object W. Thus, a sample image S corresponding to the sample is displayed in the area 75 of the setting screen 60. Fig.12 The exemplified sample image S has a defective part 90.

[0174] The setting screen 60 includes an edit button 91 for editing the defect area. By operating the edit button 91, the processor 110 starts editing the defect area.

[0175] Fig.13 is a diagram showing Fig.12FIG. 1 is a diagram showing an example of a setting screen when the edit button shown is operated. Fig.13 The illustrated setting screen 60 includes a region shape selection button group 92 and a size editing area 93. When a button included in the region shape selection button group 92 is operated, the processor 110 displays a frame line 94 having a shape (rectangular, circular, sectoral, etc.) corresponding to the button in an overlapping manner on the sample image S. The frame line 94 indicates the range of the defect region.

[0176] Furthermore, the processor 110 edits the position and size of the frame line 94 according to the operation of the button and / or input box included in the size editing area 93. In addition, the processor 110 may also display a change button 95 for changing the position or size of the frame line 94, and change the position and size of the frame line 94 according to the operation of the change button 95.

[0177] Fig.14 FIG. 1 is a diagram showing an example of a setting screen after the position and size of the frame line indicating the range of the defect area are changed. Fig.14 As shown, the user adjusts the position and size of the frame line 94 so that the defective portion 90 is contained within the frame line 94.

[0178] When the OK button 96 is operated, the processor 110 sets the area surrounded by the frame line 94 as a defect area. On the other hand, when the CANCEL button 97 is operated, the processor 110 deletes the data being edited.

[0179] Fig.15 This is a diagram showing an example of a setting screen when setting a background area. Fig.15 The illustrated setting screen 60 is displayed in response to the case where the tab 62 is operated after the defect area is set. When the button 70 for adding the background area is operated, the processor 110 overlaps and displays the frame line 98a having the same shape and size as the frame line 94 indicating the range of the defect area that has been set on the sample image S. The frame line 98a indicates the range of one background area. The processor 110 accepts a drag operation on the frame line 98a and moves the frame line 98a according to the drag operation. The user only needs to move the frame line 98a in a manner that does not include the defective part. The processor 110 sets the area surrounded by the frame line 98a as the background area. In this way, the user can set the background area only by appropriately setting the position of the frame line 98a.

[0180] Fig.16 FIG. 2 is a diagram showing another example of a setting screen when setting an additional background area and a defect area. Fig.16In the illustrated setting screen 60, frame lines 98a and 98b for setting the background area are displayed. Each time the operation button 70 is operated, the processor 110 overlays the frame line 98 for setting the background area on the sample image S, and sets the area surrounded by each frame line 98 (98a, 98b) as the background area. Depending on the sample image S, the brightness, contrast, etc. may vary according to the position. Therefore, the user sets a plurality of background areas arranged at different positions.

[0181] When the sample image S contains a plurality of defective parts, the user only needs to operate the button 69 for adding a defective area. According to the operation of the button 69, the processor 110 overlays a frame line 94a having the same shape and size as the frame line 94 representing the range of the set defective area on the sample image S. The user changes the position and size of the frame line 94a. The processor 110 sets the range surrounded by the frame line 94a as the range of the new defective area.

[0182] Furthermore, by operating the button 71 in a state where any of the frame lines 94a, 98a, and 98b is selected, the processor 110 deletes the selected frame line. Thus, the user can delete unnecessary frame lines.

[0183] <I. Evaluation value data>

[0184] Fig.17 is a diagram showing an example of evaluation value data. As Fig.17 shown, the evaluation value data is data in which the lighting pattern No. for identifying the evaluation lighting pattern, the detailed information of the evaluation lighting pattern, and the evaluation value E are associated for each evaluation lighting pattern. The detailed information indicates the illuminated lighting element, the brightness of the lighting element, and the emission color of the lighting element. As described above, when the button 72 (refer to Fig.11 ) on the setting screen 60 is operated multiple times, an evaluation image is acquired each time it is operated, and the evaluation value E is calculated for the acquired evaluation image. Therefore, the evaluation value data represents the evaluation value E for each time.

[0185] If the button 76 is operated in the setting screen 60 as Fig.11 shown, the processor 110 displays a list of the evaluation values E in the area 75 of the setting screen 60 based on the evaluation value data.

[0186] Fig.18 is a diagram showing a list of the evaluation values displayed in the setting screen. In Fig.18 (a), a list of the evaluation values E when the checkbox 79 is not selected in the setting screen 60 as Fig.11 shown is shown, and in Fig.18 (b), a list of the evaluation values E when the checkbox 79 is selected is shown. In Fig.18shows a list of eight evaluation values E (“Data0” to “Data7”) respectively calculated based on eight operations on the button 72 (refer to Fig.11 ) of the setting screen 60. “P-No.” represents the lighting pattern No. for identifying the evaluation lighting pattern.

[0187] As shown in Fig.18 (a), the evaluation values E are displayed in the order of the lighting pattern No. Thus, the user can grasp the suitability of each evaluation lighting pattern for image measurement. By selecting the checkbox 79, the user can sort the evaluation values E in descending order as shown in Fig.18 (b). Thus, it is easy to determine the evaluation lighting pattern with high suitability for image measurement.

[0188] In addition, when the evaluation value data includes multiple evaluation values E for each evaluation lighting pattern, the processor 110 performs sorting according to the following steps. The processor 110 extracts the evaluation value with the lowest suitability for image measurement (the minimum evaluation value) from the multiple evaluation values E for each evaluation lighting pattern. The processor 110 rearranges the lighting pattern No. and the evaluation values in descending order of the suitability for image measurement based on the extracted minimum evaluation value. That is, the processor 110 rearranges the lighting pattern No. and the evaluation values in descending order of the minimum evaluation value.

[0189] That is, when the processor 110 obtains multiple evaluation images for each evaluation lighting pattern, the evaluation value E is calculated for each of the multiple evaluation images. Then, the processor 110 extracts the evaluation value with the lowest suitability (hereinafter referred to as the “lowest evaluation value”) from the evaluation values E for the multiple evaluation images for each evaluation lighting pattern. The processor 110 determines the evaluation lighting pattern with the highest suitability of the extracted lowest evaluation value as the measurement lighting pattern from the multiple evaluation lighting patterns.

[0190] <J. Processing Example of Optimizing the Lighting Pattern>

[0191] Next, with reference to Figure 19 to Figure 22 , a processing example of optimizing the lighting pattern will be described. Fig.19 is an enlarged view of a part of four evaluation images respectively obtained under the illumination of four evaluation lighting patterns with different brightnesses. In (a) to (d), the evaluation images corresponding to the lighting pattern Nos. 29, 7, 44, and 40 are shown respectively. (a) shows a relatively dark evaluation image with low background uniformity. In contrast, in the order of (b), (c), and (d), the brightness increases and the background uniformity improves. However, as the background uniformity improves, the difference between the defect part 90 and the background becomes smaller.

[0192] Fig. 20 It shows that according to Fig.19 The values ​​of σ' calculated from the evaluation images are shown in FIG. Fig. 20 The value of σ' when the threshold κ is set to 1 is shown in Fig. 20 In the figure, "λ=0" indicates the first algorithm for calculating the evaluation value that depends on the difference between the defect area and the background area and does not depend on the uniformity of the background area. That is, σ' is fixed to the threshold value κ (=1) for all evaluation lighting patterns. On the other hand, "λ=1" to "λ=5" indicate the second algorithm when the background uniformity level is set to λ=1 to 5, respectively.

[0193] σ′ corresponding to lighting pattern No. 29 has a low uniformity of the background in the evaluation image (see Fig.19 (a)), so even if the background uniformity level λ=1, it takes a value larger than the threshold κ(=1), and increases as the background uniformity level λ becomes larger.

[0194] The uniformity of the background in the evaluation images corresponding to lighting patterns No. 7 and 44 is higher than that in the evaluation image corresponding to lighting pattern No. 29 (see Fig.19 (b)(c)). Therefore, σ' corresponding to lighting patterns No. 7 and 44 is fixed to threshold κ (=1) in background uniformity levels λ=1 and 2, and takes a value greater than threshold κ when background uniformity level λ is 3 or more. In addition, the uniformity of the background in the evaluation image corresponding to lighting pattern No. 44 is higher than the uniformity of the background in the evaluation image corresponding to lighting pattern No. 7. Therefore, when the background uniformity level λ is 3 or more, σ' corresponding to lighting pattern No. 44 is smaller than σ' corresponding to lighting pattern No. 7.

[0195] The uniformity of the background in the evaluation image corresponding to lighting pattern No. 40 is higher than the uniformity of the background in the evaluation image corresponding to lighting pattern No. 44 (see Fig.19 Therefore, σ' corresponding to lighting pattern No. 40 is fixed to the threshold value κ (=1) in all background uniformity levels λ.

[0196] Fig.21 It means according to Fig.19 The evaluation value E calculated by the evaluation image shown in FIG. Fig.21 Graph 2 shows the evaluation value E when the threshold κ is set to 1.

[0197] When the first algorithm (λ=0) is selected as the target algorithm, the evaluation value E corresponding to the lighting pattern No. 29 becomes the largest, and the lighting pattern No. 29 is determined as the lighting pattern for measurement. Fig. 20As shown, σ', which depends on the uniformity of the background, is fixed to a threshold value κ (= 1) for all evaluation lighting patterns. Therefore, the evaluation value E depends only on the numerator σ according to the above formula (1). p σ p It is a value indicating the difference between the defect area and the background area. Therefore, the evaluation value E corresponding to lighting pattern No. 29, which obtains an evaluation image with a large difference between the defect area and the background, is the largest.

[0198] When the second algorithm with background uniformity level λ=1, 2 is selected as the target algorithm, the evaluation value E corresponding to lighting pattern No. 7 becomes the largest, and lighting pattern No. 7 is determined as the lighting pattern for measurement. Fig. 20 As shown in FIG. 1 , in the background uniformity level λ=1, 2, σ' corresponding to lighting pattern No. 29 is greater than the threshold κ (=1). σ' is the denominator of the above formula (1). Therefore, the evaluation value E corresponding to lighting pattern No. 29 becomes smaller, and the difference between the defect area and the background area is smaller than σ p The evaluation value E corresponding to the lighting pattern No. 7 which is the second largest is the largest.

[0199] When the second algorithm with background uniformity level λ=3, 4 is selected as the target algorithm, the evaluation value E corresponding to lighting pattern No. 44 becomes the largest, and lighting pattern No. 44 is determined as the lighting pattern for measurement. Fig. 20 As shown in FIG. 1 , in the background uniformity levels λ=3 and 4, σ′ corresponding to lighting pattern No. 7 is larger than the threshold κ (=1). Therefore, the evaluation value E corresponding to lighting pattern No. 7 becomes smaller. p σ' corresponding to the next largest lighting pattern No. 44 is also larger than the threshold κ (=1), but smaller than σ' corresponding to lighting pattern No. 7. Therefore, the evaluation value E corresponding to lighting pattern No. 44 is the largest.

[0200] When the second evaluation algorithm with background uniformity level λ=5 is selected as the target algorithm, the evaluation value E corresponding to lighting pattern No. 40 becomes the largest, and lighting pattern No. 40 is determined as the lighting pattern for measurement. Fig. 20 As shown, in the background uniformity level λ=5, only σ' corresponding to lighting pattern No. 40 is fixed to threshold κ(=1), and σ' corresponding to other evaluation lighting patterns is much larger than threshold κ(=1). Therefore, the evaluation value E corresponding to lighting pattern No. 40 is the largest.

[0201] As described above, the lighting pattern for evaluation that maximizes the evaluation value E varies depending on the selected target algorithm. Therefore, by selecting an evaluation algorithm according to image measurement, a lighting pattern suitable for image measurement can be determined as the lighting pattern for measurement.

[0202] Fig. 22 It represents a graph showing the result of image processing performed on the Fig.19 evaluated image shown. The images after performing Sobel filtering processing as image processing are shown in (a) to (d). The images after performing Hough transform processing as image processing are shown in (e) to (h). (a) and (e) represent the images after image processing performed on the evaluated image corresponding to lighting pattern No. 29. (b) and (f) represent the images after image processing performed on the evaluated image corresponding to lighting pattern No. 7. (c) and (g) represent the images after image processing performed on the evaluated image corresponding to lighting pattern No. 44. (d) and (h) represent the images after image processing performed on the evaluated image corresponding to lighting pattern No. 40.

[0203] As Fig. 22 shown, when performing Sobel filtering processing, even the edges contained in the background are emphasized. As a result, in the case of using an image with low background uniformity, it is difficult to distinguish the defective part from the background, and the defective part cannot be detected with high precision. Therefore, in the case of performing image measurement including Sobel filtering processing, the user can select the second algorithm with a background uniformity level λ = 5. Thus, lighting pattern No. 40 with high background uniformity is determined as the lighting pattern for measurement.

[0204] On the other hand, when performing Hough transform processing, there is almost no influence on the background, but the defective part tends to become thinner. As a result, in the case of using an image with a small difference between the defective part and the background, the defective part cannot be detected with high precision. Therefore, in the case of performing image measurement including Hough transform processing, the user can select the first algorithm. Thus, lighting pattern No. 29 with a large difference between the defective part and the background is determined as the lighting pattern for measurement.

[0205] <K. Function / Effect>

[0206] As described above, the image processing system 1 of the present embodiment performs image measurement using an appearance image of an object W. The image processing system 1 includes a camera 8 that captures an image of the object W, and an illumination device 4 that includes a plurality of illumination elements 40 for irradiating light to the object W, and is capable of adjusting at least one of the light emission intensity and the light emission color for each illumination element 40. The image processing system 1 further includes an image acquisition unit 210, an evaluation algorithm selection unit 220, a calculation unit 230, and a lighting mode determination unit 240. The image acquisition unit 210 irradiates illumination light from the illumination device 4 according to a plurality of different evaluation lighting modes, and acquires at least one evaluation image corresponding to each evaluation lighting mode from the camera 8. The evaluation algorithm selection unit 220 selects one target algorithm from a plurality of evaluation algorithms. The calculation unit 230 calculates an evaluation value indicating suitability for image measurement for each evaluation lighting mode using the target algorithm and the evaluation image corresponding to the evaluation lighting mode. The lighting pattern determination unit 240 determines a lighting pattern to be used for image measurement from among a plurality of lighting patterns for evaluation based on the evaluation value.

[0207] According to the above configuration, an evaluation algorithm suitable for image measurement is selected from a plurality of evaluation algorithms, thereby automatically determining the lighting mode most suitable for the image measurement as the measurement lighting mode. As a result, the lighting setting of the illumination can be easily performed in a manner suitable for image measurement using an appearance image of the object W.

[0208] The plurality of evaluation algorithms include: a first algorithm whose output depends on an evaluation value of a difference between at least one defect region and at least one background region in an evaluation image; and at least one second algorithm whose output depends on an evaluation value of uniformity of at least one background region.

[0209] As described above, in the image measurement for detecting the surface defects of the object W, image processing for emphasizing the defective portion is performed. When performing image processing that can emphasize the background together with the defective portion, a lighting pattern that makes the background as uniform as possible is preferred. In such a case, by selecting the second algorithm as the target algorithm, a lighting pattern that makes the background uniform can be determined as the lighting pattern for measurement.

[0210] On the other hand, when performing image processing that does not emphasize the background, a lighting mode that makes the difference between the defect area and the background area larger is preferred. In such a case, by selecting the first algorithm as the target algorithm, the lighting mode that makes the difference between the defect area and the background area larger can be determined as the lighting mode for measurement.

[0211] At least one second algorithm includes a plurality of second algorithms having different contribution rates of background uniformity to the evaluation value.

[0212] According to the above configuration, it is possible to appropriately select a second algorithm suitable for the contribution rate of image measurement from a plurality of second algorithms. Thus, a lighting pattern suitable for image measurement is determined as the lighting pattern for measurement.

[0213] The second algorithm calculates the value corresponding to the brightness variance V of a plurality of pixels belonging to the background area (for example, (λ / L) V 1 / 2 ) is less than the threshold κ, the evaluation value is calculated using the threshold κ. The second algorithm calculates the evaluation value when the value corresponding to the variance V (for example, (λ / L)V 1 / 2 ) is greater than the threshold κ, a value corresponding to the variance V (e.g., (λ / L)V 1 / 2 The threshold κ is set according to the second algorithm.

[0214] When the variance V is used to calculate the evaluation value, the closer the variance V is to 0, the greater the influence of the variance V on the evaluation value. For example, when the evaluation value is calculated by a formula including the variance V as the denominator, as the variance V approaches 0, the evaluation value approaches infinity. However, according to the above structure, when the uniformity of the background is high to a certain extent, the evaluation value is calculated using the threshold κ, and when the uniformity of the background is low, the evaluation value is calculated using a value corresponding to the variance V. Thus, an appropriate evaluation value is calculated.

[0215] The image processing system 1 further includes a touch panel 102 as a user interface. The calculation unit 230 determines the shape, size, and position of the defect area according to the input to the touch panel 102. In addition, the calculation unit 230 sets the shape and size of the background area to be the same as the shape and size of the defect area, and determines the position of the background area according to the input to the touch panel 102. Thus, the user does not need to set the shape and size of the background area by setting the shape, size, and position of the defect area. As a result, the labor and time required for setting the background area are reduced.

[0216] The calculation unit 230 calculates evaluation values ​​for each of the plurality of evaluation images. The lighting mode determination unit 240 extracts the lowest evaluation value with the lowest degree of fitness from the evaluation values ​​for the plurality of evaluation images for each lighting mode for evaluation. The lighting mode determination unit 240 determines the lighting mode for evaluation, from the plurality of lighting modes for evaluation, the lowest evaluation value with the highest degree of fitness extracted, as the lighting mode for measurement. Thus, a lighting mode for evaluation that is stable and has a high degree of fitness is determined as the lighting mode for measurement.

[0217] The plurality of lighting modes that the lighting device 4 can adopt are pre-divided into a plurality of groups. The image acquisition unit 210 selects at least one group from the plurality of groups, and determines the plurality of lighting modes belonging to the selected at least one group as a plurality of lighting modes for evaluation. Thus, even if the total number of lighting modes that the lighting device 4 can adopt is large, by selecting a group suitable for image measurement, the time required to determine the lighting modes for measurement can be shortened.

[0218] The plurality of lighting elements 40 are arranged radially, for example, along the first to Nth radial directions at equal angular intervals. N is an integer of 2 or more. The plurality of groups include: a first group composed of a plurality of lighting modes in which the irradiation intensities in the first to Nth radial directions are uniform; and a second group composed of a plurality of lighting modes in which only the lighting elements arranged in one of the first to Nth radial directions are lit.

[0219] According to the above structure, by setting an object near the center of the plurality of lighting elements 40 arranged radially and selecting the first group, the irradiation direction of the illumination light on the object becomes uniform. Therefore, in the case of performing image measurement where the irradiation direction of the preferred illumination light is uniform, it is only necessary to select the first group. On the other hand, in the case of performing image measurement where the irradiation direction of the preferred illumination light has a deviation, it is only necessary to select the second group.

[0220] Alternatively, the plurality of groups may further include a third group composed of a plurality of lighting modes in which only the lighting elements arranged along a pair of radially opposite directions among the first to Nth radial directions are lit. Here, N is an integer of 4 or more.

[0221] In addition, the plurality of groups may also include: a group composed of a plurality of lighting modes in which the emission colors of the plurality of lighting elements are white; and a group composed of a plurality of lighting modes in which the emission colors of the plurality of lighting elements are colors other than white.

[0222] The evaluation algorithm selection unit 220 selects an object algorithm according to the input to the touch panel 102. Thus, the user can select an evaluation algorithm suitable for image measurement as the object algorithm.

[0223] <L. Modified Example>

[0224] (Modified Example 1)

[0225] In the above description, the processor 110 (evaluation algorithm selection unit 220) selects an object algorithm according to the input to the touch panel 102. However, the processor 110 (evaluation algorithm selection unit 220) may also automatically select an object algorithm according to the image measurement being performed.

[0226] For example, the control device 100 stores in advance a table that associates category information indicating the category with identification information for identifying an evaluation algorithm suitable for the category for each category of image measurement. The table is created in advance by experiments, etc. The processor 110 (evaluation algorithm selection unit 220) reads the identification information corresponding to the category of image measurement to be performed from the table, and selects the evaluation algorithm identified by the read identification information as the target algorithm.

[0227] (Variant 2)

[0228] The method for calculating the evaluation value is not limited to the above method. For example, when the defect area and the background area are all of the same shape and size, σ in equation (1) p The following formula (6) may be used for calculation instead of formula (2).

[0229]

[0230] In equation (6), b(i, j) represents the background area h0~h n-1 The average value of the brightness value of the pixel (i, j) in is expressed by the following formula (7).

[0231]

[0232] In addition, the background area h0~h n-1 The variance V of the brightness values ​​of all pixels can be calculated according to the following (8) instead of equation (4).

[0233]

[0234] Using the variance V calculated according to equation (8), calculate σ' according to equation (5) above. Alternatively, σ in equation (1) p The following formula (9) may be used for calculation instead of formula (2).

[0235]

[0236] In formula (9), a represents the defect area H p The average value of the brightness in is expressed by the following formula (10).

[0237]

[0238] The σ calculated using formula (9) p Indicates defect area H p The standard deviation of the defect area H p contains the defective part, so in the lighting mode that makes the defective part conspicuous, σ p As it increases, the evaluation value also increases.

[0239] (Variant 3)

[0240] In the above description, the threshold value κ is set in advance to, for example, 1. However, the threshold value κ may be set based on the evaluation image corresponding to the evaluation lighting pattern. For example, the threshold value κ is set as follows.

[0241] The processor 110 (calculation unit 230) calculates the variance V for each evaluation lighting pattern according to the above-mentioned formula (4) or formula (8). After that, the processor 110 (calculation unit 230) may determine the threshold value κ in a manner that satisfies both the following conditions a and b, for example. In the case where the threshold value κ can take multiple values, for example, the central value of the multiple values ​​is determined as the threshold value κ. (Condition a) In 2 to 20% of all evaluation lighting patterns, the value of (1 / L)V 1 / 2 ≥κ.

[0242] (Condition b) In 80% to 100% of all evaluation lighting patterns, V 1 / 2 ≥κ.

[0243] Alternatively, the processor 110 (calculation unit 230) may determine the threshold value κ according to the input to the touch panel 102. In this way, the user can set the threshold value κ to a desired value.

[0244] (Variant 4)

[0245] In the above description, the plurality of evaluation algorithms include a first algorithm for calculating an evaluation value that depends on the difference between the defect region and the background region, and a second algorithm for calculating an evaluation value that depends on the uniformity of the background region. However, the plurality of algorithms are not limited to these algorithms.

[0246] For example, multiple evaluation algorithms may use different edge patterns to calculate evaluation values ​​indicating edge similarity of pixels belonging to a designated area in an evaluation image. The evaluation values ​​indicating edge similarity are calculated using the method disclosed in Japanese Patent Application Laid-Open No. 7-220058 (Patent Document 2), for example.

[0247] Fig.23 is a diagram showing eight examples of edge modes. Fig.23As shown, a window of 3×3=9 pixels is set for each edge pattern. The processor 110 (calculation unit 230) makes the window overlap with the 3×3 pixels in the evaluation image, and calculates the fitness of the edge pattern and the image data in the window. The processor 110 (calculation unit 230) calculates the fitness while moving the window in the horizontal direction and the vertical direction in the designated area of ​​the evaluation image, and calculates the total value of the calculated fitness. The designated area can be the entire area of ​​the evaluation image or a part of the area. The total value of the calculated fitness represents the edge similarity of the pixels belonging to the designated area.

[0248] The processor 110 (calculation unit 230) calculates the luminance difference Q (brightness difference) between the central pixel and each of the eight peripheral pixels in the window. The calculation unit 230 calculates the fitness using the luminance difference Q as follows.

[0249] Fig.23 The fuzzy model of each edge mode expressed using the membership function is shown. Fig.23 In the figure, N stands for Negative, Z stands for zero, and P stands for Positive.

[0250] The membership function N takes a value of 1 when the brightness difference Q is a negative value to less than Qb, takes a value of 0 when it is greater than 0, and takes a value from 1 to 0 that changes linearly corresponding to the value in the range from -Qb to 0.

[0251] The membership function P takes a value of 0 when the brightness difference Q is less than 0, takes a value of 1 when it is greater than a certain positive value Qb, and takes a value between 0 and 1 that changes linearly corresponding to the value in the range of 0 to Qb.

[0252] The membership function Z has a function value of 0 in the range where the absolute value of the brightness difference Q is greater than a certain value Qa, and a function value of 1 in the range where the absolute value is less than Qb. Furthermore, in the range where the absolute value of the brightness difference Q is greater than Qb and less than Qa, the function value becomes a value between 1 and 0 that changes linearly corresponding to the value.

[0253] Fig.23 The blur model shown in (a) represents an edge pattern that is brighter in the lower right direction and darker in the upper left direction, and is slanted to the upper right. Fig.23 The blur model shown in (b) represents an edge pattern that is brighter on the right, darker on the left, and extends in the vertical direction. Fig.23 The blur model shown in (c) represents an edge pattern that is brighter in the upper right direction and darker in the lower left direction, and is slanted to the lower right. Fig.23 The blur model shown in (d) represents an edge pattern that is brighter at the top and darker at the bottom and extends in the horizontal direction. Fig.23The blur model shown in (e) represents an edge pattern that is brighter in the upper left direction and darker in the lower right direction, and is slanted to the upper right. Fig.23 The blur model shown in (f) represents an edge pattern that is brighter on the left, darker on the right, and extends in the vertical direction. Fig.23 The blur model shown in (g) represents an edge pattern that is brighter in the upper left direction and darker in the lower right direction, and is slanted to the lower right. Fig.23 The blur model shown in (h) represents an edge pattern that is brighter at the bottom, darker at the top, and extends in the horizontal direction.

[0254] For example, the following third to sixth algorithms are preset as a plurality of evaluation algorithms, and the user selects one of the third to sixth algorithms as the target algorithm. Fig.23 (a) and Fig.23 The total value of the fitness calculated by the edge mode shown in (e) is output as the evaluation value. Fig.23 (b) and Fig.23 The total value of the fitness calculated by the edge mode shown in (f) is output as the evaluation value. The fifth algorithm uses Fig.23 (c) and Fig.23 The total value of the fitness calculated by the edge mode shown in (g) is output as the evaluation value. The sixth algorithm uses Fig.23 (d) and Fig.23 The total value of the fitness calculated by the edge mode shown in (h) is output as the evaluation value.

[0255] A plurality of stripes along one direction are formed on the surface of the object W. In image measurement, image processing is performed to emphasize the stripes. In this case, an evaluation lighting pattern that makes the stripes conspicuous is preferred. Therefore, the user only needs to select an evaluation algorithm according to the direction of the stripes.

[0256] For example, if the stripes are vertical, the fourth algorithm is selected. The fourth algorithm uses Fig.23 (b) and Fig.23 The edge pattern represented by (f) is used to calculate the fitness, and the total value of the calculated fitness is output as the evaluation value. That is, the evaluation value represents the edge similarity in the vertical direction. Therefore, by selecting the fourth algorithm, the evaluation lighting pattern that makes the vertical stripes conspicuous is determined as the measurement lighting pattern.

[0257] (Variant 5)

[0258] In the above description, the shape of the lighting device 4 is set to be dome-shaped. However, the shape of the lighting device 4 is not limited to the dome shape, and may be, for example, a plane or a straight line. In addition, a half mirror may be used to reflect the illumination light to illuminate the object. For example, the light sources may be arranged in a matrix shape in the lighting device 4, and the lighting device 4 may be divided into a plurality of lighting elements.

[0259] In addition, Figure 3 In the illustrated lighting device 4, all the emission directions of the lights from the lighting elements 40 are directed toward the center. However, the emission directions of the lights from the plurality of lighting elements are not limited thereto, and may be directed toward different directions.

[0260] The light emitted by each lighting element can also be given different polarization characteristics. Specifically, by installing 0-degree, 45-degree, 90-degree, and 135-degree linear polarization filters on the emission surface of the lighting element (outside the diffuser), five characteristics including unpolarized light can be provided.

[0261] §3 Another application example

[0262] Reference Fig.24 , another application example of the present invention is described. Fig.24 This is a diagram showing another configuration example of the image processing system.

[0263] In the technology disclosed in Patent Document 1, the lighting angle that optimizes the evaluation value is calculated based on the captured images obtained by capturing the object at multiple lighting angles. Here, multiple lighting angles are predetermined. However, in the case of a lighting device that uses a large number of settable conditions such as the color and direction of the lighting, there are multiple lighting modes, and if all of these lighting modes are evaluated, the optimization will take time. On the other hand, in the case of evaluating only the predetermined lighting modes among the multiple lighting modes, there may be a situation where a lighting mode that has not been evaluated is more appropriate. As a result, the optimization becomes insufficient.

[0264] The present disclosure has been made in view of such problems, and an object of the present disclosure is to provide an image processing system, a setting method, and a program capable of more appropriately performing lighting setting of lighting in a manner suitable for image measurement using an appearance image of an object.

[0265] Fig.24 The illustrated image processing system 1A and Figure 1 Compared with the image processing system 1 shown in the figure, a control device 100A is provided instead of the control device 100 .

[0266] The control device 100A typically has a configuration that complies with a general-purpose computer architecture. The control device 100A is different from the above-described control device 100 in that it does not include the evaluation algorithm selection unit 220 , and includes an image acquisition unit 210A instead of the image acquisition unit 210 .

[0267] The plurality of lighting modes that the lighting device 4 can adopt are pre-divided into a plurality of groups. The image acquisition unit 210A selects at least one group from the plurality of groups according to the input to the user interface, and determines the plurality of lighting modes belonging to the selected at least one group as a plurality of evaluation lighting modes. Then, the image acquisition unit 210A irradiates the object with illumination light from the lighting device 4 according to the selected plurality of evaluation lighting modes xi, and acquires at least one evaluation image R corresponding to each evaluation lighting mode xi from the camera 8.

[0268] In addition, Fig.24 In the image processing system 1A shown, the calculation unit 230 may calculate the evaluation value Pi indicating the suitability for the image measurement using a predetermined evaluation algorithm and the evaluation image R corresponding to the evaluation lighting pattern xi.

[0269] in addition, Fig.24 The specific example of the image processing system 1A shown is the same as the above-mentioned "§2 specific example". That is, the plurality of groups include, for example Figure 8 Group 1 shown, Fig. 9 Group 2 shown, Fig.10 The third group shown. In addition, the plurality of groups may include a group consisting of a plurality of lighting patterns in which the luminous color of the plurality of illumination elements 40 is white, and a group consisting of a plurality of lighting patterns in which the luminous color of the plurality of illumination elements 40 is a color other than white.

[0270] according to Fig.24 The image processing system 1A shown can select a group corresponding to the image measurement from a plurality of groups. Thus, the time required for optimization can be shortened compared to the case where all of the plurality of lighting modes that the lighting device 4 can adopt are evaluated. Furthermore, since the group to be evaluated can be selected from a plurality of groups, it is possible to avoid that the lighting mode to be evaluated is not the evaluation object. Thus, an image processing system can be realized that can perform lighting setting of the lighting more appropriately in a manner suitable for image measurement using the appearance image of the object.

[0271] §4 Notes

[0272] As described above, this embodiment includes the following disclosure.

[0273] (Structure 1)

[0274] An image processing system (1) performs image measurement using an appearance image of an object (W), wherein the image processing system (1) comprises:

[0275] a photographing unit (8) for photographing the object (W);

[0276] an illumination unit (4) comprising a plurality of illumination elements (40, 41, 42 to 45, 42L to 45L, 42M to 45M, 42U to 45U) for irradiating the object (W) with light, and capable of adjusting at least one of the luminous intensity and the luminous color for each illumination element;

[0277] an image acquisition unit (110, 210) which irradiates illumination light from the illumination unit (4) according to a plurality of different evaluation lighting patterns, and acquires at least one evaluation image corresponding to each evaluation lighting pattern from the imaging unit;

[0278] A selection unit (110, 220) that selects one target algorithm from a plurality of evaluation algorithms;

[0279] a calculation unit (110, 230) for calculating, for each lighting pattern for evaluation, an evaluation value indicating suitability of measurement on the image using the target algorithm and the at least one evaluation image corresponding to the lighting pattern for evaluation; and

[0280] A mode determination unit (110, 240) determines a lighting mode used for the image measurement from among the plurality of lighting modes for evaluation based on the evaluation value.

[0281] (Structure 2)

[0282] In the image processing system (1) described in Structure 1,

[0283] The multiple evaluation algorithms include: a first algorithm, which outputs a first evaluation value that depends on the difference between at least one first region and at least one second region in the at least one evaluation image as the evaluation value; and at least one second algorithm, which outputs a second evaluation value that depends on the uniformity of the at least one second region as the evaluation value.

[0284] (Structure 3)

[0285] In the image processing system (1) described in Structure 2,

[0286] The at least one second algorithm includes a plurality of second algorithms having different contribution rates of the uniformity to the second evaluation value.

[0287] (Structure 4)

[0288] In the image processing system (1) described in structure 2 or 3,

[0289] The at least one second algorithm outputs the second evaluation value using the threshold value when the value corresponding to the brightness variance of the plurality of pixels belonging to the at least one second region is less than the threshold value, and outputs the second evaluation value using the value corresponding to the variance when the value corresponding to the variance is greater than the threshold value,

[0290] The threshold is set based on the at least one second algorithm.

[0291] (Structure 5)

[0292] In the image processing system (1) described in any one of Structures 2 to 4,

[0293] The image processing system (1) also has a user interface (102).

[0294] The calculation unit (110, 230) determines the shape, size and position of the at least one first region according to input to the user interface (102),

[0295] The calculation unit (110, 230) sets the shape and size of the at least one second region to be the same as the shape and size of the at least one first region,

[0296] The calculation unit (110, 230) determines the position of the at least one second area according to input to the user interface (102).

[0297] (Structure 6)

[0298] In the image processing system (1) described in Structure 1,

[0299] The plurality of evaluation algorithms use different edge patterns and output, as the evaluation value, a third evaluation value indicating edge similarity of pixels belonging to a designated area in the at least one evaluation image.

[0300] (Structure 7)

[0301] In the image processing system (1) according to any one of structures 1 to 6,

[0302] The at least one evaluation image includes a plurality of evaluation images,

[0303] The calculation unit calculates the evaluation value for each of the plurality of evaluation images.

[0304] The mode determination unit (110, 240) extracts, for each lighting mode for evaluation, a lowest evaluation value having the lowest degree of suitability from among the evaluation values ​​for the plurality of evaluation images,

[0305] The pattern determination unit (110, 240) determines, from among the plurality of lighting patterns for evaluation, the lighting pattern for evaluation that extracts the lowest evaluation value having the highest degree of suitability as the lighting pattern to be used for the image measurement.

[0306] (Structure 8)

[0307] In the image processing system (1) described in any one of Structures 1 to 7,

[0308] The plurality of lighting modes that the lighting unit (4) can adopt are pre-divided into a plurality of groups.

[0309] The image acquisition unit (110, 210) selects at least one group from the plurality of groups, and determines a plurality of lighting patterns belonging to the selected at least one group as the plurality of lighting patterns for evaluation.

[0310] (Structure 9)

[0311] In the image processing system (1) described in Structure 8,

[0312] The plurality of lighting elements (40, 41, 42 to 45, 42L to 45L, 42M to 45M, 42U to 45U) are radially arranged along the first to Nth radial directions at equal angle intervals.

[0313] N is an integer greater than 2,

[0314] The plurality of groups include:

[0315] A first group, which is composed of a plurality of lighting patterns with uniform illumination intensity in the first to Nth radial directions; and

[0316] The second group is composed of a plurality of lighting patterns for lighting only the illumination elements arranged in one radial direction among the first to N-th radial directions.

[0317] (Structure 10)

[0318] In the image processing system (1) described in Structure 9,

[0319] N is an even number greater than 4,

[0320] The plurality of groups further include a third group consisting of a plurality of lighting patterns for lighting only illumination elements arranged along a pair of radial directions opposite to each other among the first to N-th radial directions.

[0321] (Structure 11)

[0322] In the image processing system (1) described in Structure 8,

[0323] The plurality of groups include:

[0324] A first group, which is composed of a plurality of lighting patterns in which the light emission color of the plurality of lighting elements is white; and

[0325] The second group is composed of a plurality of lighting patterns in which the light emission colors of the plurality of illumination elements are colors other than white.

[0326] (Structure 12)

[0327] In the image processing system (1) described in Structure 1,

[0328] The image processing system (1) also has a user interface (102).

[0329] The selection unit (110, 220) selects the target algorithm according to input to the user interface (102).

[0330] (Structure 13)

[0331] A setting method for setting lighting of an illumination unit (4), wherein the illumination unit (4) is composed of a plurality of illumination elements (40, 41, 42 to 45, 42L to 45L, 42M to 45M, 42U to 45U) for irradiating light to an object (W), and at least one of the luminous intensity and the luminous color can be adjusted for each illumination element, wherein the setting method comprises the following steps:

[0332] The illumination unit (4) irradiates illumination light according to a plurality of different evaluation lighting patterns, and obtains at least one evaluation image corresponding to each evaluation lighting pattern from an imaging unit (8) that images the object (W);

[0333] Selecting one object algorithm from multiple evaluation algorithms;

[0334] for each lighting pattern for evaluation, using the object algorithm and the at least one evaluation image corresponding to the lighting pattern for evaluation, calculating an evaluation value indicating suitability for image measurement using the appearance image of the object; and

[0335] A lighting pattern used for the image measurement is determined from among the plurality of lighting patterns for evaluation based on the evaluation value.

[0336] (Structure 14)

[0337] A program causing a computer to execute the setting method described in structure 13.

[0338] (Structure 15)

[0339] An image processing system (1A) performs image measurement using an appearance image of an object (W), wherein the image processing system (1A) comprises:

[0340] an imaging unit (8) for imaging the object (W); and

[0341] An illumination unit (4) is composed of a plurality of illumination elements (40, 41, 42 to 45, 42L to 45L, 42M to 45M, 42U to 45U) for irradiating the object (W) with light, and is capable of adjusting at least one of the luminous intensity and the luminous color for each illumination element.

[0342] The plurality of lighting modes that the lighting unit (4) can adopt are pre-divided into a plurality of groups.

[0343] The image processing system (1A) further comprises:

[0344] an image acquisition unit (110A, 210) which selects at least one group from the plurality of groups, irradiates illumination light from the illumination unit (4) respectively according to a plurality of evaluation lighting patterns belonging to the at least one group, and acquires at least one evaluation image corresponding to each evaluation lighting pattern from the imaging unit (8);

[0345] a calculation unit (110, 230) for calculating, for each lighting pattern for evaluation, an evaluation value indicating a degree of suitability of measurement on the image using the at least one evaluation image corresponding to the lighting pattern for evaluation; and

[0346] A mode determination unit (110, 240) determines a lighting mode used for the image measurement from among the plurality of lighting modes for evaluation based on the evaluation value.

[0347] (Structure 16)

[0348] In the image processing system (1A) described in structure 15,

[0349] The plurality of lighting elements are radially arranged along the first to Nth radial directions at equal angle intervals.

[0350] N is an integer greater than 2,

[0351] The plurality of groups include:

[0352] A first group, which is composed of a plurality of lighting patterns with uniform illumination intensity in the first to Nth radial directions; and

[0353] The second group is composed of a plurality of lighting patterns for lighting only the illumination elements arranged in one radial direction among the first to N-th radial directions.

[0354] (Structure 17)

[0355] In the image processing system (1A) described in Structure 16,

[0356] N is an even number greater than 4,

[0357] The plurality of groups further include a third group consisting of a plurality of lighting patterns for lighting only illumination elements arranged along a pair of radial directions opposite to each other among the first to N-th radial directions.

[0358] (Structure 18)

[0359] In the image processing system (1A) described in structure 15,

[0360] The plurality of groups include:

[0361] A first group, which is composed of a plurality of lighting patterns in which the light emission color of the plurality of lighting elements is white; and

[0362] The second group is composed of a plurality of lighting patterns in which the light emission colors of the plurality of illumination elements are colors other than white.

[0363] (Structure 19)

[0364] In the image processing system (1A) described in any one of Structures 15 to 18,

[0365] The image processing system (1A) also has a user interface (102).

[0366] The image acquisition unit (110A, 210) selects the at least one group according to input to the user interface (102).

[0367] (Structure 20)

[0368] A setting method for setting lighting of an illumination unit (4), wherein the illumination unit (4) is composed of a plurality of illumination elements (40, 41, 42 to 45, 42L to 45L, 42M to 45M, 42U to 45U) for irradiating light to an object (W), and at least one of the luminous intensity and the luminous color can be adjusted for each illumination element, wherein the setting method comprises the following steps:

[0369] The plurality of lighting modes that the lighting unit (4) can adopt are pre-divided into a plurality of groups.

[0370] At least one group is selected from the plurality of groups, illumination light is irradiated from the illumination unit (4) respectively according to a plurality of evaluation lighting patterns belonging to the at least one group, and at least one evaluation image corresponding to each evaluation lighting pattern is acquired from the imaging unit (8) for imaging the object (W);

[0371] for each lighting pattern for evaluation, using the at least one evaluation image corresponding to the lighting pattern for evaluation, calculating an evaluation value indicating suitability of measurement on the image; and

[0372] A lighting pattern used for the image measurement is determined from among the plurality of lighting patterns for evaluation based on the evaluation value.

[0373] (Structure 21)

[0374] A program (150) causes a computer to execute the setting method described in structure 20.

[0375] Although the embodiments of the present invention have been described, the embodiments disclosed this time should be considered to be illustrative in all aspects and not restrictive. The scope of the present invention is indicated by the claims, and it is intended to include all modifications within the meaning and scope equivalent to the claims.

[0376] Description of symbols

[0377] 1, 1A: image processing system; 4: lighting device; 8: camera; 40, 41, 42 to 45, 42L to 45L, 42M to 45M, 42U to 45U: lighting element; 46: opening; 47: reflector; 48: diffuser; 60: setting screen; 61, 62, 63: label; 64, 65, 80: input field; 66 to 68, 78, 79, 82: check box; 69 to 72, 76, 77, 84, 86: button; 73, 74: display field; 75, 81, 83, 85: area; 90: defective part; 91: edit button; 92: area shape selection button group; 93: size editing area; 94, 94a, 98, 98a, 98b: frame line; 95: change button; 96: confirm button; 97: cancel button; 100, 100A: control device; 101: display unit; 102: touch panel; 110: processor; 112: RAM; 114: display controller; 116: system controller; 118: I / O controller; 120: hard disk; 122: device interface; 124: input interface; 128: communication interface; 130: memory card interface; 136: memory card; 150: control program; 210, 210A: image acquisition unit; 220: evaluation algorithm selection unit; 230: calculation unit; 240: lighting mode determination unit; D1: first radial direction; D2: second radial direction; D3: third radial direction; D4: fourth radial direction; H0~H m-1 : defect area; R, Ri: evaluation images; S: sample image; W: object; h0~h n-1 : Background area; xi: Lighting pattern used for evaluation.

Claims

1. An image processing system for performing image measurement using an appearance image of an object, wherein: The image processing system has: an imaging unit configured to image the object; an illumination unit, which is composed of a plurality of illumination elements for illuminating the object with light, and is capable of adjusting at least one of a light emission intensity and a light emission color for each illumination element; an image acquisition unit configured to irradiate illumination light from the illumination unit according to a plurality of different evaluation lighting patterns, and acquire at least one evaluation image corresponding to each evaluation lighting pattern from the imaging unit; a selection unit that selects one target algorithm from a plurality of evaluation algorithms; a calculation unit that calculates, for each lighting pattern for evaluation, an evaluation value indicating a degree of suitability of measurement on the image using the target algorithm and the at least one evaluation image corresponding to the lighting pattern for evaluation; as well as a mode determination unit that determines a lighting mode used for the image measurement from among the plurality of lighting modes for evaluation based on the evaluation value, The multiple evaluation algorithms include: a first algorithm, which outputs a first evaluation value that depends on the difference between at least one first region and at least one second region in the at least one evaluation image as the evaluation value; and at least one second algorithm, which outputs a second evaluation value that depends on the uniformity of the at least one second region as the evaluation value.

2. The image processing system according to claim 1, wherein: The at least one second algorithm includes a plurality of second algorithms having different contribution rates of the uniformity to the second evaluation value.

3. The image processing system according to claim 1 or 2, wherein: The at least one second algorithm outputs the second evaluation value using the threshold value when the value corresponding to the brightness variance of the plurality of pixels belonging to the at least one second region is less than the threshold value, and outputs the second evaluation value using the value corresponding to the variance when the value corresponding to the variance is greater than the threshold value, The threshold is set based on the at least one second algorithm.

4. The image processing system according to claim 1 or 2, wherein: The image processing system also has a user interface. The calculation unit determines the shape, size and position of the at least one first region according to an input to the user interface, The calculation unit sets the shape and size of the at least one second region to be the same as the shape and size of the at least one first region, The calculation unit determines a position of the at least one second area according to an input to the user interface.

5. The image processing system according to claim 1, wherein: The plurality of evaluation algorithms further include two or more third algorithms that use different edge patterns and output, as the evaluation value, a third evaluation value indicating edge similarity of pixels belonging to a designated area in the at least one evaluation image.

6. The image processing system according to claim 1, wherein: The image processing system also has a user interface. The selection unit selects the target algorithm according to an input to the user interface.

7. An image processing system for performing image measurement using an appearance image of an object, wherein: The image processing system has: an imaging unit configured to image the object; an illumination unit, which is composed of a plurality of illumination elements for illuminating the object with light, and is capable of adjusting at least one of a light emission intensity and a light emission color for each illumination element; an image acquisition unit configured to irradiate illumination light from the illumination unit according to a plurality of different evaluation lighting patterns, and acquire at least one evaluation image corresponding to each evaluation lighting pattern from the imaging unit; a selection unit that selects one target algorithm from a plurality of evaluation algorithms; a calculation unit that calculates, for each lighting pattern for evaluation, an evaluation value indicating a degree of suitability of measurement on the image using the target algorithm and the at least one evaluation image corresponding to the lighting pattern for evaluation; as well as a mode determination unit that determines a lighting mode used for the image measurement from among the plurality of lighting modes for evaluation based on the evaluation value, The at least one evaluation image includes a plurality of evaluation images, The calculation unit calculates the evaluation value for each of the plurality of evaluation images. The mode determination unit extracts, for each lighting pattern for evaluation, a lowest evaluation value having the lowest degree of suitability from among the evaluation values ​​for the plurality of evaluation images; The pattern determination unit determines, from among the plurality of lighting patterns for evaluation, the lighting pattern for evaluation that extracts the lowest evaluation value having the highest degree of suitability as the lighting pattern to be used for the image measurement.

8. An image processing system for performing image measurement using an appearance image of an object, wherein: The image processing system has: an imaging unit configured to image the object; an illumination unit, which is composed of a plurality of illumination elements for illuminating the object with light, and is capable of adjusting at least one of a light emission intensity and a light emission color for each illumination element; an image acquisition unit configured to irradiate illumination light from the illumination unit according to a plurality of different evaluation lighting patterns, and acquire at least one evaluation image corresponding to each evaluation lighting pattern from the imaging unit; a selection unit that selects one target algorithm from a plurality of evaluation algorithms; a calculation unit that calculates, for each lighting pattern for evaluation, an evaluation value indicating a degree of suitability of measurement on the image using the target algorithm and the at least one evaluation image corresponding to the lighting pattern for evaluation; as well as a mode determination unit that determines a lighting mode used for the image measurement from among the plurality of lighting modes for evaluation based on the evaluation value, The plurality of lighting modes that the lighting unit can adopt are pre-divided into a plurality of groups. The image acquisition unit selects at least one group from the plurality of groups, and determines a plurality of lighting patterns belonging to the selected at least one group as the plurality of evaluation lighting patterns.

9. The image processing system according to claim 8, wherein: The plurality of lighting elements are radially arranged along the first to Nth radial directions at equal angle intervals. N is an integer greater than 2, The plurality of groups include: A first group, which is composed of a plurality of lighting patterns with uniform illumination intensity in the first to Nth radial directions; and The second group is composed of a plurality of lighting patterns for lighting only the illumination elements arranged in one radial direction among the first to N-th radial directions.

10. The image processing system according to claim 9, wherein: N is an even number greater than 4, The plurality of groups further include a third group consisting of a plurality of lighting patterns for lighting only illumination elements arranged along a pair of radial directions opposite to each other among the first to N-th radial directions.

11. The image processing system according to claim 8, wherein: The plurality of groups include: A first group, which is composed of a plurality of lighting patterns in which the light emission color of the plurality of lighting elements is white; and The second group is composed of a plurality of lighting patterns in which the light emission colors of the plurality of illumination elements are colors other than white.

12. A setting method for setting lighting of an illumination unit, wherein the illumination unit is composed of a plurality of illumination elements for irradiating light to an object, and at least one of the luminous intensity and the luminous color can be adjusted for each illumination element, wherein: The setting method has the following steps: irradiating illumination light from the illumination unit according to a plurality of different evaluation lighting patterns, and acquiring at least one evaluation image corresponding to each evaluation lighting pattern from an imaging unit that images the object; Selecting one object algorithm from multiple evaluation algorithms; For each lighting pattern for evaluation, using the object algorithm and the at least one evaluation image corresponding to the lighting pattern for evaluation, calculate an evaluation value indicating suitability for image measurement using the appearance image of the object; as well as determining a lighting pattern used for the image measurement from among the plurality of lighting patterns for evaluation based on the evaluation value; The multiple evaluation algorithms include: a first algorithm, which outputs a first evaluation value that depends on the difference between at least one first region and at least one second region in the at least one evaluation image as the evaluation value; and at least one second algorithm, which outputs a second evaluation value that depends on the uniformity of the at least one second region as the evaluation value.

13. A recording medium storing a computer-readable program in a nonvolatile manner, wherein: The computer-readable program causes a computer to execute the setting method according to claim 12 .

Images